Some HIV controllers (HICs) experience CD4+T cell count loss and/or lose their ability to control HIV. In this study, we investigated the rate of immunologic and/or virologic progression (ImmP/VirP) and its determinants in the ANRS CO21/CODEX cohort. Immunologic progression was defined as a lasting fall in CD4+T cell count below 350/mm(3) or more than 200/mm(3) with a baseline count below 600/mm(3). Virologic progression was defined as a HIV viral load (VL) above 2000 copies/mL on two consecutive determinations. Clinical characteristics, immune activation, ultrasensitive HIV VL and total HIV DNA were analyzed. Disease progression was observed in 15 of the 217 patients followed up between 2009 and 2013 (ImmP, n = 10; VirP, n = 5). Progressors had higher ultrasensitive HIV RNA levels at inclusion (i.e. 1-2 years before progression) than non-progressors. ImmP had also lower CD4+T cell nadir and CD4+T cell count at inclusion, and VirP had higher HIV DNA levels in blood. T cell activation and IP10 levels at inclusion were significantly higher in ImmP than in non-progressors. In summary, the lasting loss of CD4+T cells, residual HIV replication and basal levels of immune activation appear to be major determinants of progression in HICs. These factors should be considered for adjusting their follow-up.