In this work, a fusion algorithm is proposed for improving the accuracy and stability of passive sound source localization. Different from the traditional algorithm that contains a single-plane cross array, here, the fusion algorithm is used to overcome the position blur in the process of localization. First, the two-plane four-element cross array model is established. Based on this model, the method is defined to judge the position where the sound source is located. According to the localization principle, we derive the calculation formula of the sound source position, based on a single four-element planar array. Then, the elevation angle sine value is introduced into the coordinate formula as the weighted coefficient by analyzing the indirect measurement error, and the fusion algorithm is employed to conduct the sound source localization, based on the two-plane four-element cross array. Finally, the relationships are obtained, between the time delay estimation error, the elevation angle, the horizontal angle, and the localization performance. Besides, the validity of this algorithm is validated by measuring the ranging and direction-finding accuracy. The results show that the distance error rate is within 2%, and the angle error rate is within 3%, which means a good localization effect. The proposed algorithm is expected to be widely used in thunderstorm cloud detection for its quick measurement and high precision.