The degree of admixture in Brazil between historically isolated populations is complex and geographically variable. Studies differ as to what the genetic and phenotypic consequences of this mixing have been. In Northeastern Brazil, we enrolled 522 residents of Salvador and 620 of Fortaleza whose distributions of self-declared color were comparable to those in the national census. Using the program Structure and principal components analysis there was a clear correlation between biogeographic ancestry and categories of skin color. This correlation with African ancestry was stronger in Salvador (r=0.585; P<0.001) than in Fortaleza (r=0.236; P<0.001). In Fortaleza, although self-declared blacks had a greater proportion of European ancestry, they had more African ancestry than the other categories. When the populations were analyzed without pseudoancestors, as in some studies, the relationship of 'race' to genetic ancestry tended to diffuse or disappear. The inclusion of different African populations also influenced ancestry estimates. The percentage of unlinked ancestry informative markers in linkage disequilibrium, a measure of population structure, was 3-5 times higher in both Brazilian populations than expected by chance. We propose that certain methods, ascertainment bias and population history of the specific populations surveyed can result in failure to demonstrate a correlation between skin color and genetic ancestry. Population structure in Brazil has important implications for genetic studies, but genetic ancestry is irrelevant for how individuals are treated in society, their health, their income or their inclusion. These track more closely with perceived skin color than genetic ancestry.