We introduce the concept of a quantum pulse gate (QPG), a method for accessing the intrinsic broadband spectral mode structure of ultrafast quantum states of light. This mode structure can now be harnessed for applications in quantum information processing. We propose an implementation in a PPLN waveguide, based on spectrally engineered sum frequency generation (SFG). It allows us to pick well-defined spectral broadband modes from an ultrafast multi-mode state for interconversion to a broadband mode at another frequency. By pulse-shaping the bright SFG pump beam, different orthogonal broadband modes can be addressed individually and extracted with high fidelity.