A strong limitation of linear optical quantum computing is the probabilistic operation of two-quantum-bit gates based on the coalescence of indistinguishable photons. A route to deterministic operation is to exploit the single-photon nonlinearity of an atomic transition. Through engineering of the atom-photon interaction, phase shifters, photon filters and photon-photon gates have been demonstrated with natural atoms. Proofs of concept have been reported with semiconductor quantum dots, yet limited by inefficient atom-photon interfaces and dephasing. Here, we report a highly efficient single-photon filter based on a large optical nonlinearity at the single-photon level, in a near-optimal quantum-dot cavity interface. When probed with coherent light wavepackets, the device shows a record nonlinearity threshold around 0.3 ± 0.1 incident photons. We demonstrate that 80% of the directly reflected light intensity consists of a single-photon Fock state and that the two- and three-photon components are strongly suppressed compared with the single-photon one.