Optical image encryption based on interference has attracted a lot of attention recently. The technique employs two pure phase masks derived from the complex field of the image in the Fresnel diffraction domain. The image decryption procedure can be carried out by inverse Fresnel transformation of the summation of two pure phase masks. However, the silhouette of the original image, which is recovered by either of the two phase-only masks, impedes the application of this technique. In this paper, a very simple method for binary image encryption based on interference of two phase-only masks is proposed without any silhouette problem. The binary image in combination with a random phase mask is separated into two phase-only masks directly, and the decryption by summation of the two masks can be performed digitally or optically. In this paper, the encryption and decryption processes are analyzed, after which both the optical simulation and the experimental results based on single-beam holography are given to demonstrate the feasibility of the encryption method. As information nowadays is mainly digitized into binary codes, the proposed encryption method may find applications in the information processing field.