The integration of the Inertial Navigation System (INS) and the Global Positioning System (GPS) is widely applied to seamlessly determine the time-variable position and orientation parameters of a system for navigation and mobile mapping applications. For optimal data fusion, the Kalman filter (KF) is often used for real-time applications. Backward smoothing is considered an optimal post-processing procedure. However, in current INS/GPS integration schemes, the KF and smoothing techniques still have some limitations. This article reviews the principles and analyzes the limitations of these estimators. In addition, an on-line smoothing method that overcomes the limitations of previous algorithms is proposed. For verification, an INS/GPS integrated architecture is implemented using a low-cost micro-electro-mechanical systems inertial measurement unit and a single-frequency GPS receiver. GPS signal outages are included in the testing trajectories to evaluate the effectiveness of the proposed method in comparison to conventional schemes.