CD4+ T cell counts are important tests used to stage HIV-positive patients, enabling clinicians to make informed antiretroviral treatment decisions and to monitor the therapeutic outcomes. However, state-of-the-art CD4 counting methods based on flow cytometry are not applicable in resource-limited settings, due to their high cost and technical requirements. In previous work, we reported the development of a cell isolation microchip that can be used at the point of care for CD4 counts. In that microfluidic chip, CD4+ T cells were separated from 10 microL of whole blood, and enumerated via either light microscopy or impedance sensing. The microchip counts matched flow cytometry results in the intermediate CD4 count range, between 200-800 cells/microL, but displayed a positive bias at absolute CD4 counts below 200 cells/microL, due largely to monocyte contamination. To enhance the performance in the low CD4 count range, we report here an improved design of a two-stage microfluidic device to deplete monocytes from whole blood, followed by CD4+ T cell capture. Using the double-stage device combined with a high viscosity rinsing solution, we obtained microchip CD4 counts comparable to flow cytometry results in the full clinically relevant range. In addition to CD4 counting, the strategy of contaminant depletion prior to target cell isolation can be easily adapted to immunoaffinity capture of other cell types that lack a unique surface marker from a complex biological fluid.