In response to the presence of compatible rhizobium bacteria, legumes form symbiotic organs called nodules on their roots. These nodules house nitrogen-fixing bacteroids that are a differentiated form of the rhizobium bacteria. In some legumes, the bacteroid differentiation comprises a dramatic cell enlargement, polyploidization, and other morphological changes. Here, we demonstrate that a peptidoglycan-modifying enzyme in Bradyrhizobium strains, a DD-carboxypeptidase that contains a peptidoglycan-binding SPOR domain, is essential for normal bacteroid differentiation in Aeschynomene species. The corresponding mutants formed bacteroids that are malformed and hypertrophied. However, in soybean, a plant that does not induce morphological differentiation of its symbiont, the mutation does not affect the bacteroids. Remarkably, the mutation also leads to necrosis in a large fraction of the Aeschynomene nodules, indicating that a normally formed peptidoglycan layer is essential for avoiding the induction of plant immune responses by the invading bacteria. In addition to exopolysaccharides, capsular polysaccharides, and lipopolysaccharides, whose role during symbiosis is well defined, our work demonstrates an essential role in symbiosis for yet another rhizobial envelope component, the peptidoglycan layer.