We have investigated the relation between local structure, valence and carrier type with magnetism in the Zn-Co-O system. Thin films ranging from wurtzite Zn(1-x)Co(x)O (Co:ZnO) to ZnCo2O4 spinel were grown on c-sapphire substrates. On the one hand, the unprecedented doping of x = 0.6 Co in ZnO enables to study the structural and magnetic properties well-above the coalescence limit. On the other hand, the ZnCo2O4 spinel provides a p-type environment. We find a strong correlation between local structure, valence and carrier type throughout the Zn-Co-O system. In contrast to earlier publications neither 60% Co:ZnO nor ZnCo2O4 exhibit any sign of ferromagnetic order despite of the high concentration of magnetic ions and a p-type carrier background. Instead, antiferromagnetic exchange is found to be the predominant magnetic interaction in the Zn-Co-O system.