The accuracy of time-delay estimation (TDE) in ultrasound elastography is usually measured by calculating the value of normalized cross correlation (NCC) at the estimated displacement. NCC value, however, could be very high at a displacement estimate with large error, a well-known problem in TDE referred to as peak-hopping. Furthermore, NCC value could suffer from jitter error, which is due to electric noise and signal decorrelation. Herein, we propose a novel method to assess the accuracy of TDE by investigating the NCC profile around the estimated time-delay. We extract several features from the NCC profile, and utilize support vector machine to classify peak-hopping and jitter error. The results on simulation, phantom, and in vivo data show the significant improvement of the proposed algorithm compared to the state of the art techniques.