Whiteness is an important colorimetric characteristic for surface colors. Currently, the CIE whiteness formula is widely used to characterize the whiteness of a surface in industry, whose limitations have been well documented. In this study, the whiteness appearance of 55 textile samples [43 non-fluorescent whitening agents (FWAs) and 12 FWA samples] were evaluated under six lighting conditions, comprising three levels of correlated color temperature (CCT) (i.e., 3000, 4000, and 6500 K) and two levels of UV radiation (i.e., zero and high), by 12 observers with normal color vision. The results indicated that the CIE whiteness formula with the CAT02 chromatic adaptation transform and the optimized CIE whiteness formula had the best performance in predicting the whiteness of surface colors, regardless of whether FWAs are contained, among the six formulas.