With the advantages of high velocity measurement accuracy and fast dynamic response, the laser Doppler velocimeter (LDV) is expected to replace the odometer to be combined with a strapdown inertial navigation system (SINS) to form a higher precision integrated navigation system. However, LDV scale factor error and misalignment angles between LDV and inertial measurement unit will affect the accuracy of navigation. Considering that not all global navigation satellite system (GNSS) receivers can directly provide velocity information and current mainstream calibration methods are sensitive to the measurement noise and outliers of velocity and position information, a robust calibration method aided by GNSS is proposed in this paper, which is based on position observation. Different from current popular calibration methods, the attitude information of the GNSS/SINS integrated navigation system obtained by an adaptive Kalman filter is used to construct the observation vector together with LDV velocity outputs and GNSS position outputs in this method. The LDV scale factor error and the misalignment angle are determined by the ratio of two observation vector modulus and the Davenport's q-method method, respectively. The accuracy and robustness of the calibration method are verified by one vehicle test with normal GNSS signals and one vehicle test with GNSS signals with outliers. And the horizontal position error of dead reckoning of the calibrated LDV/SINS integrated system are less than 0.0314% and 0.1033% of the mileage, respectively.