This paper presents a greedy optimization algorithm for sampling design to calibrate WDS hydraulic model. The proposed approach starts from the existing sensors and sequentially adds one new sensor at each optimization simulation step. In each step, the algorithm tries to minimize the calibration prediction uncertainty. The new sensor is installed in the location where the uncertainty is greatest but also sensitive to other nodes. The robustness of the proposed approach is tested under different spatial and temporal demand distribution. We found that both the number of sensors and the perturbation ratio affect the calibration accuracy as defined by the average nodal pressure deviation itself and its variability. The plot of the calibration accuracy versus the number of sensors can reasonably guide the trade-off between model calibration accuracy and number of sensors placed or the cost. This proposed approach is superior in calibration accuracy and modeling efficiency when compared to the standard genetic algorithm (SGA) and Monte Carlo Sampling algorithm (MCS).