To improve the model-based inversion performance of microwave induced thermoacoustic tomography for breast tumor imaging, an active adjoint modeling (AAM) method is proposed. It aims to provide a more realistic breast acoustic model used for tumor inversion as the background by actively measuring and reconstructing the structural heterogeneity of human breast environment. It utilizes the reciprocity of acoustic sensors, and adapts the adjoint tomography method from seismic exploration. With the reconstructed acoustic model of breast environment, the performance of model-based inversion method such as time reversal mirror is improved significantly both in contrast and accuracy. To prove the advantage of AAM, a checkerboard pattern model and anatomical realistic breast models have been used in full wave numerical simulations.