Ventral frontal cortex is commonly involved in traumatic brain injury (TBI). The smell identification test (SIT), object alternation (OA), and the Iowa gambling task (IGT) are associated with this brain region in experimental and neuropsychological research. We examined the relationship of performance on these tests to residual structural brain integrity quantified from MRI in 58 TBI patients, including 18 patients with focal cortical contusions and 40 patients with diffuse injury only. Image analysis yielded regional volumetric measures of gray matter, white matter and cerebrospinal fluid. Multivariate analyses identified distributed patterns of regional volume loss associated with test performance across all three behavioral measures. The tasks were sensitive to effects of TBI. In multivariate analyses, performance in all three tasks was related to gray matter loss including ventral frontal cortex, but the SIT was most sensitive to ventral frontal cortex damage, even in patients without focal lesions. The SIT was further related to temporal lobe and posterior cingulate/retrosplenial volumes. OA and the IGT were associated with superior medial frontal volumes. Complex tasks, such as OA and the IGT, do not consistently localize to a single cortical region. The SIT is associated with the integrity of ventral frontal regions, but it is also affected by distributed damage, although the contribution of undetected olfactory tract or bulb damage could not be ruled out. This study illustrates the scope and limitations of functional localization in human ventral frontal cortex.