Any complete theory of human stereopsis must model not only how the correspondences between locations in the two views are determined and the depths are recovered from their disparity, but also how the ambiguity arising from such factors as noise, periodicity, and large regions of constant intensity are resolved and missing data are interpolated. In investigating this process of recovering surface structure from sparse disparity information, using stereo pairs with sparse identifiable features, we made an observation that contradicts all extant models. It suggests the inadequacy of retinotopic representation in modeling surface perception in this stage. We also suggest a possible alternative theory, which is a minimization of the modulus of Gaussian curvature.