Water molecules in the brain diffuse preferentially along the fiber tracts within white matter that form the anatomical connections across spatially distant brain regions. A diffusion tensor (DT) is a probabilistic ellipsoid composed of three orthogonal vectors, each having a direction and an associated scalar magnitude, that represent the probability of water molecules diffusing in each of those directions. The 3D morphologies of DTs can be compared across groups of subjects to reveal disruptions in structural organization and neuroanatomical connectivity of the brains of persons with various neuropsychiatric illnesses. Comparisons of tensor morphology across groups have typically been performed on scalar measures of diffusivity, such as Fractional Anisotropy (FA) rather than directly on the complex 3D morphologies of DTs. Scalar measures, however, are related in nonlinear ways to the eigenvalues and eigenvectors that create the 3D morphologies of DTs. We present a mathematical framework that permits the direct comparison across groups of mean eigenvalues and eigenvectors of individual DTs. We show that group-mean eigenvalues and eigenvectors are multivariate Gaussian distributed, and we use the Delta method to compute their approximate covariance matrices. Our results show that the theoretically computed mean tensor (MT) eigenvectors and eigenvalues match well with their respective true values. Furthermore, a comparison of synthetically generated groups of DTs highlights the limitations of using FA to detect group differences. Finally, analyses of in vivo DT data using our method reveal significant between-group differences in diffusivity along fiber tracts within white matter, whereas analyses based on FA values failed to detect some of these differences.