Color has an unresolved role in natural scene recognition. Whereas rapid serial visual presentation paradigms typically find no advantage for colored over grayscale scenes, color seems to play a decisive role for recognition memory. The distinction between detection and memorization has not been addressed directly in one paradigm. Here we asked ten observers to detect animals in 2-s 20 Hz sequences. Each sequence consisted of two 1-s segments, one of grayscale images and one of colored; each segment contained one or no target, totaling zero, one, or two targets per sequence. In one-target sequences, hit rates were virtually the same for targets appearing in the first or second segment, as well as for grayscale and colored targets, though observers were more confident about detecting colored targets. In two-target sequences, observers preferentially reported the second of two identical targets, in comparison to categorically related (same-species animals) or unrelated (different-species animals) targets. Observers also showed a strong preference for reporting colored targets, though only when targets were of different species. Our findings suggest that color has little effect on detection, but is used in later stages of processing. We may speculate that color ensures preferential access to or retrieval from memory when distinct items must be rapidly remembered.