A binocular full-color holographic three-dimensional near eye display system using a single spatial light modulator (SLM) is proposed. In the display system, the frequency spectrum shifting operation and color spectrum shifting operation are adopted to realize the frequency division multiplexing (FDM) and frequency superposition multiplexing (FSM) by manipulating the frequency spectrums of each color- and view-channel sub-holograms. The FDM combined with polarization multiplexing will be used to implement binocular display using a single SLM, and the FSM working with a bandpass filter for each view-channel will be used to achieve full-color display from single frame hologram. The optical analysis and experiments with 3D color objects confirm the feasibility of the proposed system in the practical application.