The model proposed in this paper is based on the fact that the reflection might have a significant contribution to the attenuation of the acoustic waves propagating through the cancellous bone. The numerical implementation of the mentioned effect is realized by the development of a new representative volume element that includes an infinitesimally thin 'transient' layer on the contact surface of the bone and the marrow. This layer serves to model the amplitude transformation of the incident wave by the transition through media with different acoustic impedances and to take into account the energy loss due to the reflection. The proposed representative volume element together with the multiscale finite element is used to simulate the wave propagation and to evaluate the attenuation coefficient for samples with different effective densities in the dependence of the applied excitation frequency. The obtained numerical values show a very good agreement with the experimental results. Moreover, the model enables the determination of the upper and the lower bound for the attenuation coefficient.