Cultivated pepper (Capsicum annuum) is a phenotypically diverse species grown throughout the world. Wild and landrace peppers are typically small-fruited and pungent, but contain many important traits such as insect and disease resistance. Cultivated peppers vary dramatically in size, shape, pungency, and color, and often lack resistance traits. Fruit characteristics (e.g. shape and pericarp thickness) are major determinants for cultivar selection, and their association with disease susceptibility can reduce breeding efficacy. This study evaluated a diverse collection of peppers for mature fruit phenotypic traits, correlation among fruit traits and Phytophthora fruit rot resistance, genetic diversity, population structure, and trait broad sense heritability. Significant differences within all fruit phenotype categories were detected among pepper lines. Fruit from Europe had the thickest pericarp, and fruit from Ecuador had the thinnest. For fruit shape index, fruit from Africa had the highest index, while fruit from Europe had the lowest. Five genetic clusters were detected in the pepper population and were significantly associated with fruit thickness, end shape, and fruit shape index. The genetic differentiation between clusters ranged from little to very great differentiation when grouped by the predefined categories. Broad sense heritability for fruit traits ranged from 0.56 (shoulder height) to 0.98 (pericarp thickness). When correlations among fruit phenotypes and fruit disease were evaluated, fruit shape index was negatively correlated with pericarp thickness, and positively correlated with fruit perimeter. Pepper fruit pericarp, perimeter, and width had a slight positive correlation with Phytophthora fruit rot, whereas fruit shape index had a slight negative correlation.