By introduction of K+, Rb+, and Cs+ cations into the classical commercial nonlinear optical crystal LiB3O5 (LBO), the series of novel mixed-alkali-metal borates Li2.6K0.4[B5O8(OH)2] (K-LBO), Li2.85Rb0.15[B5O8(OH)2] (Rb-LBO), and Li2.9Cs0.1[B5O8(OH)2] (Cs-LBO) have been obtained under hydrothermal conditions. The steric hindrance effect generated by the introduction of large alkali-metal cations and partial substitution of small Li+ cations broke the three-dimensional (3-D) framework of [B3O7]5- borate-oxygen clusters in LBO and resulted in a structure rearrangement to produce infrequent [B10O26]22- 2-D layers. The unique layered structure induced an increase in birefringence in A-LBOs (A = K, Rb, Cs), which is favorable for phase matching during second-harmonic generation. All three compounds are potential deep-ultraviolet nonlinear optical materials, which was proved by UV-vis-NIR diffuse reflectance spectroscopy and second-harmonic-generation measurements.