Face recognition plays an essential role for the biometric authentication. Conventional lens-based imagery keeps the spatial fidelity with respect to the object, thus, leading to the privacy concerns. Based on the point spread function engineering, we employed a coded mask as the encryption scheme, which allows a readily noninterpretable representation on the sensor. A deep neural network computation was used to extract the features and further conduct the identification. The advantage of this data-driven approach lies in that it is neither necessary to correct the lens aberration nor revealing any facial conformity amid the image formation chain. To validate the proposed framework, we generated a dataset with practical photographing and data augmentation by a set of experimental parameters. The system has the capability to adapt a wide depth of field (DoF) (60-cm hyperfocal distance) and pose variation (0 to 45 deg). The 100% recognition accuracy on real-time measurement was achieved without the necessity of any physics priors, such as the encryption scheme.