We aim to automatically learn a latent graph structure in a low-dimensional space from high-dimensional, unsupervised data based on a unified density estimation framework for both feature extraction and feature selection, where the latent structure is considered as a compact and informative representation of the high-dimensional data. Based on this framework, two novel methods are proposed with very different but intuitive learning criteria from existing methods. The proposed feature extraction method can learn a set of embedded points in a low-dimensional space by naturally integrating the discriminative information of the input data with structure learning so that multiple disconnected embedding structures of data can be uncovered. The proposed feature selection method preserves the pairwise distances only on the optimal set of features and selects these features simultaneously. It not only obtains the optimal set of features but also learns both the structure and embeddings for visualization. Extensive experiments demonstrate that our proposed methods can achieve competitive quantitative (often better) results in terms of discriminant evaluation performance and are able to obtain the embeddings of smooth skeleton structures and select optimal features to unveil the correct graph structures of high-dimensional data sets.