A hybrid method using the support vector machine (SVM) correlation filter and the phase-shift interferometry (PSI) holography is proposed to recognize 3D object, which can improve the correct decision rate and resist the distortion of object rotation and noise. The different images of two types of both in-plane and out-of-plane rotated object recorded by digital holography are reconstructed. The reconstructed images of two types are selected to synthesize the SVM correlation filter, respectively. To compare the correct decision rates of the SVM correlation filter with other three ones, it is found that the experimental result is better in rotation resistance and noise tolerance.