There is a large literature focused on the color perception of matte surface. However, recent research showed that the component of surface specular reflection, such as glossiness, also affects categorical color perception. For instance, the color term "gold" was used to name high specular stimuli within a specific range of chromaticity, which overlaps with those of yellow and orange for low specular stimuli. In the present study, we investigated whether the component of surface specular reflectance affects the color perception of 5- to 8-month-old infants by using the preferential looking technique. In the first experiment, we conducted a simple test to determine whether infants perceive yellow and gold as the same color by comparing their preference for these colors over green. If the infants perceive yellow and gold as the same color, they would show similar preference scores over green. On the other hand, if infants show different preference scores over green, it indicates that infants do not perceive yellow and gold as the same color. Only the 7-8 month-old infants showed different preference scores for gold and yellow over green. This result indicates that the 7-8 month-old infants perceive gold and yellow as different colors. In Experiment 2, we eliminated the component of specular reflectance on the gold surface and presented it against green to infants. A similar preference score of yellow over green was obtained. This result suggests that the difference between the preference scores for gold and yellow over green in Experiment 1 was based on representations of glossiness.