Graphs are essential to improve the performance of graph-based machine learning methods, such as spectral clustering. Various well-designed methods have been proposed to learn graphs that depict specific properties of real-world data. Joint learning of knowledge in different graphs is an effective means to uncover the intrinsic structure of samples. However, the existing methods fail to simultaneously mine the global and local information related to sample structure and distribution when multiple graphs are available, and further research is needed. Hence, we propose a novel intrinsic graph learning (IGL) with discrete constrained diffusion-fusion to solve the above problem in this article. In detail, given a set of the predefined graphs, IGL first obtains the graph encoding the global high-order manifold structure via the diffusion-fusion mechanism based on the tensor product graph. Then, two discrete operators are integrated to fine-prune the obtained graph. One of them limits the maximum number of neighbors connected to each sample, thereby removing redundant and erroneous edges. The other one forces the rank of the Laplacian matrix of the obtained graph to be equal to the number of sample clusters, which guarantees that samples from the same subgraph belong to the same cluster and vice versa. Moreover, a new strategy of weight learning is designed to accurately quantify the contribution of pairwise predefined graphs in the optimization process. Extensive experiments on six single-view and two multiview datasets have demonstrated that our proposed method outperforms the previous state-of-the-art methods on the clustering task.