An approach to expanding the instantaneous bandwidth of a photonic sampling analog-to-digital converter (ADC) for receiving linear frequency modulation waveforms (LFMWs) is proposed and experimentally demonstrated based on up-sampling and filtering in the fractional Fourier domain. Through twice zero interpolation, the equivalent sampling rate is quadrupled, which also quadruples the nominal instantaneous bandwidth of the photonic sampling ADC. In addition, with the assistance of bandpass filtering in the fraction Fourier domain, the image signals and the harmonic distortions generated in the interpolation process are filtered out. As a result, the effective instantaneous bandwidth of the photonic sampling ADC is doubled. In the experiment, the instantaneous bandwidth of a photonic sampling ADC with a sampling rate of 5 GSa/s for receiving LFMWs is increased from 2.5 GHz to 5 GHz by using the proposed method. Input LFMWs within the frequency range of 24-27 GHz and 30-33 GHz, i.e., with an instantaneous bandwidth of 3 GHz, are digitized without frequency-domain aliasing. Besides, the ability of the proposed method to enhance the ranging accuracy in a broadband radar system is demonstrated. This method reduces the hardware complexity of the photonic sampling ADC for receiving broadband LFMWs in radar systems.