A thermal diode or rectifier is a system that transmits heat or energy in one direction better than in the opposite direction. We investigate the influence of the distribution of energy among wave numbers on the diode effect for the junction of two dissimilar harmonic chains. An analytical expression for the diode coefficient, characterizing the difference between heat fluxes through the junction in two directions, is derived. It is shown that the diode coefficient depends on the distribution of energy among wave numbers. For an equilibrium energy distribution, the diode effect is absent, while for non-equilibrium energy distributions the diode effect is observed even though the system is harmonic. We show that the diode effect can be maximized by varying the energy distribution and relative position of spectra of the two harmonic chains. Conditions are formulated under which the system acts as an ideal thermal rectifier, i.e., transmits heat only in one direction. The results obtained are important for understanding the heat transfer in heterogeneous low-dimensional nanomaterials.