There is a growing interest in exploring the connectivity patterns of the human brain. Specifically, the utility of noninvasive neuroimaging data and graph theoretical analysis have provided important insights into the anatomical connections and topological pattern of human brain structural networks in vivo. This review focuses on recent methodological and application studies, utilizing graph theoretical approaches, on brain structural networks with structural magnetic resonance imaging (MRI) and diffusion MRI. These studies showed many nonrandom properties of structural brain networks, such as small-worldness, modularity, and highly connected hubs. Importantly, topological organization of the networks shows changes during normal development, aging, and neuropsychiatric diseases. Network structures have also been found to correlate with behavioral or cognitive functions, which imply their associations with functional dynamics. These advances not only help us to understand how the healthy human brain is structurally organized, but also provide a novel insight into the biological mechanisms of brain disorders. Future studies will involve the combination of structural/diffusion MRI and functional MRI, to realize how the structural connectivity patterns of the brain underlie its functional states, and will explore whether graph theoretical analysis of structural brain networks could serve as potential imaging biomarkers for disease diagnosis and treatment.