The definition of a nonequilibrium temperature through generalized fluctuation-dissipation relations relies on the independence of the fluctuation-dissipation temperature from the observable considered. We argue that this observable independence is deeply related to the uniformity of the phase-space probability distribution on the hypersurfaces of constant energy. This property is shown explicitly on three different stochastic models, where observable dependence of the fluctuation-dissipation temperature arises only when the uniformity of the phase-space distribution is broken. The first model is an energy transport model on a ring, with biased local transfer rules. In the second model, defined on a fully connected geometry, energy is exchanged with two heat baths at different temperatures, breaking the uniformity of the phase-space distribution. Finally, in the last model, the system is connected to a zero temperature reservoir, and preserves the uniformity of the phase-space distribution in the relaxation regime, leading to an observable-independent temperature.