The G matrix measures the components of phenotypic variation that are genetically heritable. The structure of G, that is, its principal components and their associated variances, determines, in part, the direction and speed of multivariate trait evolution. In this article we present a framework and results that give the structure of G under the assumption of neutrality. We suggest that a neutral expectation of the structure of G is important because it gives a null expectation for the structure of G from which the unique consequences of selection can be determined. We demonstrate how the processes of mutation, recombination, and drift shape the structure of G. Furthermore, we demonstrate how shared common ancestry between segregating alleles shapes the structure of G. Our results show that shared common ancestry, which manifests itself in the form of a gene genealogy, causes the structure of G to be nonuniform in that the variances associated with the principal components of G decline at an approximately exponential rate. Furthermore we show that the extent of the nonuniformity in the structure of G is enhanced with declines in mutation rates, recombination rates, and numbers of loci and is dependent on the pattern and modality of mutation.