For calculating color differences, the CIEDE2000 and CIE94 equations are widely used and recommended. These equations were derived more than a decade ago, based for a large part on the RIT-Dupont set of visual data. This data was collected from a series of psychophysical tests that use the method of constant stimuli. In this method, observers need to compare the color difference within a sample pair to that between a reference pair. In the current investigation, we show that the color difference equation significantly changes if reference pairs are chosen in the underlying visual experiments that differ from what was used when creating the RIT-Dupont dataset. The investigation is done using metallic paint samples representing two color centers, red and yellow-green. We show that the reproducibility differs for three different reference pairs, and that for modeling the visual data for the yellow-green color center, extra model terms are required as compared to the CIEDE2000 equation. Our results suggest that observers differ in their ability to mentally convert a color difference recognized in a sample pair into an equivalent color difference along the color difference direction represented by the reference pair. We also find that in these tests the tolerance to lightness differences is widened by a factor of 1.3 to 1.6, and that for the red color center the tolerance ellipsoid is rotated by 30° as compared to the CIEDE2000 equation. The latter observations are possibly due to the metallic texture in the samples used for the current experiment.