This study proposes a portrait image relighting system based on a simplified photometric stereo method. The system, comprising a controllable digital single lens reflex camera and five polarized flashlights, can obtain a color shade-less image and synthesize a normal map from shaded images. When calibrating the photometric stereo, the normal map is taken as a linear combination of shaded images and clamped with respect to specific normal directions on a white-coated sphere. The relit images were generated through inverse rendering in a predefined virtual environment. To evaluate personal preference, 24 adult subjects were recruited to conduct subjective assessments comparing the deep portrait relighting method results. From experiments regarding different scenarios, we concluded that the proposed system based on a simplified photometric stereo performs acceptably for relighting portrait images.