The paramagnetic NMR study of HoM2N@C80-Ih and Ho2MN@C80-Ih nitride cluster fullerenes (M = Sc, Lu, Y) reveals strong dependence of Ho-induced paramagnetic shifts (δ(para)) in (13)C NMR spectra on the size of the diamagnetic metal in the cluster. In particular, the δ(para) value in HoY2N@C80 is almost doubled in comparison to that in HoSc2N@C80. X-ray magnetic circular dichroism studies show that all Ho-nitride cluster fullerenes have the same magnetic ground state of Ho(3+). Point-charge ligand-field splitting calculations show that the increase of the M(3+) radius in going from Sc to Y results in a considerable increase of the energy splitting between different Jz states. This leads to a 19% higher magnetic anisotropy of Ho(3+) in HoY2N@C80 than in HoSc2N@C80 at 300 K. Variations of the molecular geometry and cluster dynamics with the size of the cluster are found to have even greater influence on δ(para) values. This work shows that the magnetic properties of the species confined inside the fullerene cages can be tuned using the geometrical factors such as the cluster and the cage size.