Ultrafast magnetization reversal driven by femtosecond laser pulses has been shown to be a promising way to write information. Seeking to improve the recording density has raised intriguing fundamental questions about the feasibility of combining ultrafast temporal resolution with sub-wavelength spatial resolution for magnetic recording. Here we report on the experimental demonstration of nanoscale sub-100 ps all-optical magnetization switching, providing a path to sub-wavelength magnetic recording. Using computational methods, we reveal the feasibility of nanoscale magnetic switching even for an unfocused laser pulse. This effect is achieved by structuring the sample such that the laser pulse, via both refraction and interference, focuses onto a localized region of the structure, the position of which can be controlled by the structural design. Time-resolved photo-emission electron microscopy studies reveal that nanoscale magnetic switching employing such focusing can be pushed to the sub-100 ps regime.