We report a detailed structural investigation of supramolecular polymers formed by hydrogen bonded self-assembly of bis-urea monomers. The careful exploration of the energy landscape by molecular mechanics/molecular dynamics (MM/MD) simulations has allowed us to identify three distinct self-assembled structures of similar stabilities. These structures have been compared to X-ray crystal data. We observe that a slight change in the molecular structure can favor a particular structure over the others. Detailed analysis shows that hydrogen bonds stabilize all three structures to a similar extent. Therefore, it is the interactions among the lateral substituents, and with the filament environment, that are the decisive factors in the competition between the possible self-assembled structures. This study constitutes a clear reminder that the conformation of a supramolecular polymer is a sensitive function of the molecular structure and may significantly differ from the solid-state conformation of a model compound.