Retroviruses are enveloped viruses that assemble on the inner leaflet of cellular membranes. Improving biophysical techniques has recently unveiled many molecular aspects of the interaction between the retroviral structural protein Gag and the cellular membrane lipids. This interaction is driven by the N-terminal matrix domain of the protein, which probably undergoes important structural modifications during this process, and could induce membrane lipid distribution changes as well. This review aims at describing the molecular events occurring during MA-membrane interaction, and pointing out their consequences in terms of viral assembly. The striking conservation of the matrix membrane binding mode among retroviruses indicates that this particular step is most probably a relevant target for antiviral research.