Clustering chemistry is a key point in the design and synthesis of the secondary building units that comprise metal-organic frameworks (MOFs) based on group IV metals. In this work, the first stages of the zirconium-carboxylate clustering process in alcohol/water mixtures are studied in detail using the monocarboxylic benzoic and hydroxybenzoic acids to avoid the polymerization. Mass spectroscopy measurements performed on the reactions revealed the presence of hexa- and pentanuclear species even at low pH values and also evidenced the acid-base nature and pH dependence of the transformation between both species. The control on the chemistry governing the equilibria between these species has allowed us to isolate six new compounds in the solid state. The single-crystal X-ray diffraction analysis revealed that they are closely related to the well-known [Zr6(O)4(OH)4(OOC)12] secondary building unit found in many MOFs by removing carboxylic ligands in the case of the hexameric species ([Zr6(O)4(OH)4(OOC)8(H2O)8]4+) or by additionally removing one of the metal centers in the case of the pentameric entities ([Zr5(O)2(OH)6(OOC)4(H2O)11(alcohol)]6+). Going in detail, the unsaturated hexameric clusters exhibit different dispositions of their eight carboxylate ligands in such a way that the remaining four carboxylate-free positions are arranged according to a square planar or tetrahedral symmetry. It should be highlighted that the pentameric complexes imply an unprecedented core nuclearity in zirconium clusters and thus their isolation provides a novel building block for the design of metal-organic materials.