Alkali aluminoborate glasses have recently been shown to exhibit a high threshold for indentation cracking compared to other bulk oxide glasses. However, to enable the use of these materials in engineering applications, there is a need to improve their hardness by tuning the chemical composition. In this study, we substitute alkaline earth for alkali network-modifying species at fixed aluminoborate base glass composition and correlate it with changes in the structure, mechanical properties, and densification behavior. We find that the increase in field strength (i.e., the charge-to-size ratio) achieved by substituting alkaline earth oxide from BaO to MgO manifests itself in a monotonic increase in several properties, such as atomic packing density, glass-transition temperature, densification ability, indentation hardness, and crack resistance. Although the use of alkaline earth oxides as modifier enables higher hardness values (increasing from 2.0 GPa for Cs to 5.8 GPa for Mg), their crack resistance is generally lower than that of the corresponding alkali aluminoborate glasses. We discuss the origin of this compromise between hardness and crack resistance in terms of the ability of the glass networks to undergo structural transformations and self-adapt under stress. We show that the extent of volume densification scales linearly with the number of pressure-induced coordination number changes of B and Al.