Fabrication of ultra-high aspect ratio exchangeable and customizable tips for atomic force microscopy (AFM) using lateral focused ion beam (FIB) milling is presented. While on-axis FIB milling does allow high aspect ratio (HAR) AFM tips to be defined, lateral milling gives far better flexibility in terms of defining the shape and size of the tip. Due to beam-induced deformation, it has so far not been possible to define HAR structures using lateral FIB milling. In this work we obtain aspect ratios of up to 45, with tip diameters down to 9 nm, by a deformation-suppressing writing strategy. Several FIB milling strategies for obtaining sharper tips are discussed. Finally, assembly of the HAR tips on a custom-designed probe as well as the first AFM scanning is shown.