Evolution of microstructure in magneto-active elastomers (MAEs) which can be caused by an applied magnetic field is a fascinating phenomenon with a significant impact on the mechanical behavior of the composite. To gain insight into the underlying mechanisms of this phenomenon, it is essential to create a model that can appropriately describe the field induced change in the particle distribution and its mechanical implications. The magneto-mechanical coupling is driven by magnetic interactions between the particles in the applied field. These magnetic interactions can result in macroscopic deformation of the sample and also in rearrangement of the microstructure, i.e. the local positions of the particles. In the case of initially isotropic MAEs made with a sufficiently soft matrix, this leads to the formation of chains of magnetized particles, creating a significant increase in the mechanical moduli along the field direction. In this paper, we implement a transversely isotropic Neo-Hookean material model to account for such anisotropic elastic behavior. A dipolar mean field approach is used to describe magnetic interactions between the particles. A penalty term is introduced to compensate for the micro-mechanical elastic energy required to move the particles inside the cross-linked elastomer. The resulting model can predict the huge magneto-rheological effects observed in experiments, and improves our understanding of how microstructure evolution affects magnetically induced deformation and stiffness of MAEs.