We propose and experimentally demonstrate a noninvasive credit card verifier with a low false rejection rate (FRR). Our key idea is based on the use of three broadband light sources in our data-nonintrusive photonics-based credit card verifier structure, where spectral components of the embossed hologram images are registered as red, green, and blue. In this case, nine distinguishable variables are generated for a feed-forward neural network (FFNN). In addition, we investigate the center of mass of the image histogram projected onto the x axis (I(color)), making our system more tolerant of the intensity fluctuation of the light source. We also reduce the unwanted signals on each hologram image by simply dividing the hologram image into three zones and then calculating their corresponding I(color) values for red, green, and blue bands. With our proposed concepts, we implement our field test prototype in which three broadband white light light-emitting diodes (LEDs), a two-dimensional digital color camera, and a four-layer FFNN are used. Based on 249 genuine credit cards and 258 counterfeit credit cards, we find that the average of differences in I(color) values between genuine and counterfeit credit cards is improved by 1.5 times and up to 13.7 times. In this case, we can effectively verify credit cards with a very low FRR of 0.79%.