An optical single-channel color image encryption scheme based on chaotic fingerprint phase mask and diffractive imaging is proposed. In this proposed encryption scheme, the fingerprint used to generate the random phase masks is served as a secret key directly. Additionally, the random phase masks generated by the fingerprint, chaotic Lozi map, and secure hash algorithm (SHA-256) are used only as interim variables. With the help of the chaotic fingerprint phase masks placed at different diffraction distances, the color image that is encoded into a grayscale pattern by the phase-truncation technique is encrypted into a noise-like diffraction pattern. For decryption, the color image can be retrieved from the noise-like diffraction pattern by using an iterative phase retrieval algorithm, fingerprint, and phase keys generated from the encryption process. Since the fingerprint key shared by the sender and authorized receiver is strongly linked with the user and does not need to be transmitted over the open network, the security of this proposed encryption scheme can be greatly improved. Additionally, the parameters of the chaotic Lozi map and Fresnel diffraction distances can also provide additional security to the proposed encryption scheme. Furthermore, compared with the encryption schemes based on digital holography, the implementation of this proposed encryption scheme is relatively simple. The numerical simulations and analysis verify the feasibility, security, and robustness of this proposed encryption scheme.