Under the framework of computer-aided diagnosis, optical coherence tomography (OCT) has become an established ocular imaging technique that can be used in glaucoma diagnosis by measuring the retinal nerve fiber layer thickness. This letter presents an automated retinal layer segmentation technique for OCT images. In the proposed technique, an OCT image is first cut into multiple vessel and nonvessel sections by the retinal blood vessels that are detected through an iterative polynomial smoothing procedure. The nonvessel sections are then filtered by a bilateral filter and a median filter that suppress the local image noise but keep the global image variation across the retinal layer boundary. Finally, the layer boundaries of the filtered nonvessel sections are detected, which are further classified to different retinal layers to determine the complete retinal layer boundaries. Experiments over OCT for four subjects show that the proposed technique segments an OCT image into five layers accurately.