A novel directional-view image scaling method that corrects chromatic dispersion and enhances the quality of three-dimensional (3D) images reconstructed by a full-color holographic display system is proposed. When the 3D information of the real scene is acquired through the integral imaging pickup method, the orthographic projection image is reconstructed. Then, each directional-view image is separated and synthesized onto the computer-generated hologram. To correct the chromatic dispersion of the full-color holographic 3D display, each directional-view image is scaled depending on the relation between the different wavelengths of single-channel holograms and resolutions of the sub-holograms. According to the optical experimental results, it can be concluded that the proposed method is an effective way of producing full-color holographic images from an orthographic projection image through a simple process.