The concept of natural time turned out to be useful in revealing dynamical features behind complex time series including electrocardiograms, ionic current fluctuations of membrane channels, seismic electric signals, and seismic event correlation. However, the origin of this empirical usefulness is yet to be clarified. Here, it is shown that this time domain is in fact optimal for enhancing the signals in time-frequency space by employing the Wigner function and measuring its localization property.