Using organic photodetectors for multispectral sensing is attractive due to their unique capabilities to tune spectral response, transmittance, and polarization sensitivity. Existing methods lack tandem multicolor detection and exhibit high spectral cross talk. We exploit the polarization sensitivity of organic photodetectors, together with birefringent optical filters to design single-pixel multispectral detectors that achieve high spectral selectivity and good radiometric performance. Two different architectures are explored and optimized, including the Solc-based and multitwist-retarder-based organic photodetectors. Although the former demonstrated a higher spectral resolution, the latter enables a more compact sensor as well as greater flexibility in device fabrication.