Despite substantial progress in measuring the anatomical and functional variability of the human brain, little is known about the genetic and environmental causes of these variations. Here we developed an automated system to visualize genetic and environmental effects on brain structure in large brain MRI databases. We applied our multi-template segmentation approach termed "Multi-Atlas Fluid Image Alignment" to fluidly propagate hand-labeled parameterized surface meshes, labeling the lateral ventricles, in 3D volumetric MRI scans of 76 identical (monozygotic, MZ) twins (38 pairs; mean age=24.6 (SD=1.7)); and 56 same-sex fraternal (dizygotic, DZ) twins (28 pairs; mean age=23.0 (SD=1.8)), scanned as part of a 5-year research study that will eventually study over 1000 subjects. Mesh surfaces were averaged within subjects to minimize segmentation error. We fitted quantitative genetic models at each of 30,000 surface points to measure the proportion of shape variance attributable to (1) genetic differences among subjects, (2) environmental influences unique to each individual, and (3) shared environmental effects. Surface-based statistical maps, derived from path analysis, revealed patterns of heritability, and their significance, in 3D. Path coefficients for the 'ACE' model that best fitted the data indicated significant contributions from genetic factors (A=7.3%), common environment (C=38.9%) and unique environment (E=53.8%) to lateral ventricular volume. Earlier-maturing occipital horn regions may also be more genetically influenced than later-maturing frontal regions. Maps visualized spatially-varying profiles of environmental versus genetic influences. The approach shows promise for automatically measuring gene-environment effects in large image databases.