In this paper, a data augmentation aided complex-valued network is proposed for underwater acoustic (UWA) orthogonal frequency division multiplexing (OFDM) channel estimations, wherein empirical mode decomposition based data augmentation is proposed to solve the current dilemma in the deep learning embedded UWA-OFDM communications: data scarcity and data-sampling difficulties in real-world applications. In addition, the significance of high-frequency component augmentation for the UWA channel and how it positively influences the following model training are discussed in detail and demonstrated experimentally in this paper. In addition, the complex-valued network is specially designed for the complex-formatted UWA-OFDM signal, which can fully utilize the relationship between its real and imaginary parts with half of the spatial resources of its real-valued counterparts. The experiments with the at-sea-measured WATERMARK dataset indicate that the proposed method can perform a near-optimal channel estimation, and its low resource requirements (on dataset and model) make it more adaptable to real-world UWA applications.