The state of aggregation at room temperature of tert-butanol (TBH) and perfluoro tert-butanol (TBF) liquid mixtures is assessed by vibrational spectroscopy (Raman and infrared) and X-ray diffraction and analyzed using density functional theory (DFT) and molecular dynamics (MD) simulations. It is shown that larger clusters (mostly tetramers) of TBH are destroyed upon dilution with TBF. Small oligomers, monomers, and mainly heterodimers are present at the equimolar concentration. At variance with slightly interacting solvents, the signature of hetero-oligomers is shown by the appearance of a new broad band detected in the infrared region. The same spectral observation is detected for mixtures of other hydrogenated alcohols (methanol and 1-butanol). The new infrared feature is unaffected by dilution in a polar solvent (CDCl3) in a high-concentration domain, allowing us to assign it to the signature of small hetero-oligomers. MD simulations are used to assess the nature of the species present in the mixture (monomers and small hetero-oligomers) and to follow the evolution of their population upon the dilution. Combining MD simulations with DFT calculations, the infrared spectral profile is successfully analyzed in equimolecular mixtures. This study shows that TBF is a structure breaker of hydrogen-bonded alcohol networks and that the TBF (donor)-TBH (acceptor) heterodimer is the dominant species in an extended range of concentration, centered in the vicinity of the equimolar fraction.