Both observer age and size of stimulus as characterized using the field of view (FOV) are two important parameters to affect the color matching functions (CMFs) of human observers. They are also included in the cone fundamental and CMFs models that were recently proposed by the International Commission on Illumination (CIE) in 2006. In contrast to the great number of studies investigating the performance of CMFs in characterizing color matches and mismatches using different primary sets, few study investigated the effect of CMFs in characterizing these two factors. In this study, we carefully designed a series of test stimuli in five colors, which had different magnitudes of calculated color difference to the corresponding reference stimulus using the CIE 2006 CMFs model. The stimuli were presented in two FOVs (i.e., 8.6° and 2.9°). A group of young and senior observers were asked to judge which of the two test stimuli appeared to be similar to the reference stimulus, which was a forced choice. The color differences calculated using the CIE 1931 2° and 1964 10° CMFs were found to have higher correlations to the visual color differences judged by the senior and young observers respectively, regardless of the actual FOVs used in the experiment. In addition, though the CIE 2006 CMFs with the different parameter settings always had better performance in predicting the perceived color differences, they failed to characterize the effect of observer age and stimulus size. Also, the experiment results did not support the CIE's recommendation of using the 10° CMFs for stimuli with an FOV greater than 4° and the 2° CMFs for those smaller than 4°.