The non-negative matrix factorization (NMF) algorithm represents the original image as a linear combination of a set of basis images. This image representation method is in line with the idea of "parts constitute a whole" in human thinking. The existing deep NMF performs deep factorization on the coefficient matrix. In these methods, the basis images used to represent the original image is essentially obtained by factorizing the original images once. To extract features reflecting the deep localization characteristics of images, a novel deep NMF architecture based on underlying basis images learning is proposed for the first time. The architecture learns the underlying basis images by deep factorization on the basis images matrix. The deep factorization architecture proposed in this paper has strong interpretability. To implement this architecture, this paper proposes a deep non-negative basis matrix factorization algorithm to obtain the underlying basis images. Then, the objective function is established with an added regularization term, which directly constrains the basis images matrix to obtain the basis images with good local characteristics, and a regularized deep non-negative basis matrix factorization algorithm is proposed. The regularized deep nonlinear non-negative basis matrix factorization algorithm is also proposed to handle pattern recognition tasks with complex data. This paper also theoretically proves the convergence of the algorithm. Finally, the experimental results show that the deep NMF architecture based on the underlying basis images learning proposed in this paper can obtain better recognition performance than the other state-of-the-art methods.