Smartphones have shown promise as an enabling technology for portable and distributed point-of-care diagnostic tests. The CMOS camera sensor can be used for detecting optical signals, including fluorescence for applications such as isothermal nucleic acid amplification tests. However, such analysis is typically limited mostly to end point detection of single targets. Here we present a smartphone-based image analysis pipeline that utilizes the CIE xyY (chromaticity-luminance) color space to measure the luminance (in lieu of RGB intensities) of fluorescent signals arising from nucleic acid amplification targets, with a discrimination sensitivity (ratio between the positive to negative signals), which is an order of magnitude more than traditional RGB intensity based analysis. Furthermore, the chromaticity part of the analysis enables reliable multiplexed detection of different targets labeled with spectrally separated fluorophores. We apply this chromaticity-luminance formulation to simultaneously detect Zika and chikungunya viral RNA via end point RT-LAMP (Reverse transcription Loop-Mediated isothermal amplification). We also show real time LAMP detection of Neisseria gonorrhoeae samples down to a copy number of 3.5 copies per 10 μL of reaction volume in our smartphone-operated portable LAMP box. Our chromaticity-luminance analysis is readily adaptable to other types of multiplexed fluorescence measurements using a smartphone camera.