In underwater imaging, water waves cause severe geometric distortions and blurring of the acquired short-exposure images. Corrections for these distortions have been tackled reasonably well by previous efforts but still need improvement in the estimation of pixel shift maps to increase restoration accuracy. This paper presents a new algorithm that efficiently estimates the shift maps from geometrically distorted video sequences and uses those maps to restore the sequences. A nonrigid image registration method is employed to estimate the shift maps of the distorted frames against a reference frame. The sharpest frame of the sequence, determined using a sharpness metric, is chosen as the reference frame. A k-means clustering technique is employed to discard too-blurry frames that could result in inaccuracy in the shift maps' estimation. The estimated pixel shift maps are processed to generate the accurate shift map that is used to dewarp the input frames into their nondistorted forms. The proposed method is applied on several synthetic and real-world video sequences, and the obtained results exhibit significant improvements over the state-of-the-art methods.