Extensive ocean noise records have kurtoses markedly different from the Gaussian distribution and therefore exhibit non-Gaussianity, which influences the performance of many sonar signal processing methods. To model the amplitude distribution, this paper studies a Bayesian Gaussian mixture model (BGMM) and its associated learning algorithm, which exploits the variational inference method. The most compelling feature of the BGMM is that it automatically selects a suitable number of effective components and then can approximate a sophisticated distribution in practical applications. The probability density functions (PDFs) of three types of noise in different frequency bands collected in the South China Sea-ambient noise, ship noise, and typhoon noise-are modeled and the goodness of fit is examined by applying the one-sample Kolmogorov-Smirnov test. The results demonstrate that: (i) Ambient noise in the low-frequency band may be slightly non-Gaussian, ship noise in each considered band is apparently non-Gaussian, and typhoons affect the noise in the low-frequency band to make it apparently non-Gaussian, while the noise in the high-frequency band is less affected and appears to be Gaussian. (ii) BGMM has higher goodness of fit than the Gaussian or Gaussian mixture model. (iii) In the non-Gaussian case, despite some components having small mixing coefficients, they are of great significance for describing the PDF.