The interplay between polymer-polymer and polymer-solvent interactions as well as interactions that impose secondary structures determines the conformation of polymer chains in dilute solution. Polypeptoids-poly(N-substituted glycines) have been shown to form helical secondary structures primarily driven by steric interactions from chiral, bulky side chains, while polypeptoids with a racemic mixture of the same side chains lead to unstructured coil chains with a shorter Kuhn length. Small-angle neutron scattering (SANS) of the polypeptoids in dilute solution reveals that the helical polypeptoids are only locally stiffer than the coil chains formed from the racemic analogue, but exhibit overall flexibility. We show that chain conformations of both helical and coil polypeptoids (in terms of radius of gyration, Rg) are insensitive to solvent quality (parametrized by the second virial coefficient, A2). Potential effects from the bulky, chiral/racemic side chains dominating chain conformations are excluded by comparison with an achiral polypeptoid lacking side chain chirality. The specific interactions between polypeptoid segments are likely dominating the chain conformations in this type of polypeptoids as opposed to polymer-solvent interactions or energetic contributions from the helical secondary structure.