In a quantum network based on atoms and photons, a single atom should control the photon state and, reciprocally, a single photon should allow the coherent manipulation of the atom. Both operations require controlling the atom environment and developing efficient atom-photon interfaces, for instance by coupling the natural or artificial atom to cavities. So far, much attention has been drown on manipulating the light field with atomic transitions, recently at the few-photon limit. Here we report on the reciprocal operation and demonstrate the coherent manipulation of an artificial atom by few photons. We study a quantum dot-cavity system with a record cooperativity of 13. Incident photons interact with the atom with probability 0.95, which radiates back in the cavity mode with probability 0.96. Inversion of the atomic transition is achieved for 3.8 photons on average, showing that our artificial atom performs as if fully isolated from the solid-state environment.