Image segmentation is an essential step in many image analysis pipelines and many algorithms have been proposed to solve this problem. However, they are often evaluated subjectively or based on a small number of examples. To fill this gap, we hand-segmented a set of 97 fluorescence microscopy images (a total of 4009 cells) and objectively evaluated some previously proposed segmentation algorithms.We focus on algorithms appropriate for high-throughput settings, where only minimal user intervention is feasible.The hand-labeled dataset (and all software used to compare methods) is publicly available to enable others to use it as a benchmark for newly proposed algorithms.