Over the last decades, the use of magnetic nanoparticles in research and commercial applications has increased dramatically. However, direct detection of trace quantities remains a challenge in terms of equipment cost, operating conditions and data acquisition times, especially in flowing conditions within complex media. Here we present the in-line, non-destructive detection of magnetic nanoparticles using high performance atomic magnetometers at ambient conditions in flowing media. We achieve sub-picomolar sensitivities measuring ~30 nm ferromagnetic iron and cobalt nanoparticles that are suitable for biomedical and industrial applications, under flowing conditions in water and whole blood. Additionally, we demonstrate real-time surveillance of the magnetic separation of nanoparticles from water and whole blood. Overall our system has the merit of in-line direct measurement of trace quantities of ferromagnetic nanoparticles with so far unreached sensitivities and could be applied in the biomedical field (diagnostics and therapeutics) but also in the industrial sector.