A single metallic nanodisk is the simplest plasmonic nanostructure, but it is robust enough to generate a Fano resonance in the forward and backward scattering spectra by the increment of nanodisk height in the symmetric and asymmetric dielectric environment. Thanks to the phase retardation effect, the non-uniform distribution of electric field along the height of aluminum (Al) nanodisk generates the out-of-plane higher-order modes, which interfere with the dipolar mode and subsequently result in the Fano-lineshape scattering spectra. Meanwhile, the symmetry-breaking effect by the dielectric substrate and the increment of refractive index of the symmetric dielectric environment further accelerate the phase retardation effect and contribute to the appearance of out-of-plane modes. The experimental results on the periodic Al nanodisk arrays with different heights confirm the retardation-induced higher modes in the asymmetric and symmetric environment. The appearance of higher modes and blueshifted main dips in the transmission spectra prove the dominant role of out-of-plane higher modes on the plasmonic resonances of the taller Al nanodisk.