Phoneme-sized phonetic segments are often defined as the most basic unit of language organization. Two common inferences made from this description are that there are clear correlates to phonetic segments in the acoustic speech stream, and that humans have access to these segments from birth. In fact, well-replicated studies have shown that the acoustic signal of speech lacks invariant physical correlates to phonetic segments, and that the ability to recognize segmental structure is not present from the start of language learning. Instead, the young child must learn how to process the complex, generally continuous acoustic speech signal so that phonetic structure can be derived. This paper describes and reviews experiments that have revealed developmental changes in speech perception that accompany improvements in access to phonetic structure. In addition, this paper explains how these perceptual changes appear to be related to other aspects of language development, such as syntactic abilities and reading. Finally, evidence is provided that these critical developmental changes result from adequate language experience in naturalistic contexts, and accordingly suggests that intervention strategies for children with language learning problems should focus on enhancing language experience in natural contexts.