Natural visual images are typically composed of multiple objects, which need to be segregated from each other and from the background. The visual system has evolved to capture a great variety of cues that allow a meaningful segmentation of the visual input. One of these cues is connectedness. Connected image regions are likely to belong to a single visual object, whereas disconnected image regions typically belong to different objects. The visual system should therefore be rather proficient in recovering connected image regions. In the present article we will review evidence in favour of an important role of connectedness detection for figure-ground segmentation, and speculate on the physiological mechanisms that allow the visual system to perform this non-trivial task. We argue that biologically plausible feedforward networks are maladapted for the detection of connectedness. It is proposed that neurons that respond to connected image regions are linked by a network of recurrent connections that we call the interaction skeleton. Neurons spread a tag through the interaction skeleton, which labels cells that respond to the same perceptual object. Tag-spreading costs time and is therefore inconsistent with extremely rapid object recognition. We will discuss the pros and cons of two such tags: synchrony and rate modulation.