We present an approach to detect anatomical structures by configurations of interest points, from a single example image. The representation of the configuration is based on Markov Random Fields, and the detection is performed in a single iteration by the MAX-SUM algorithm. Instead of sequentially matching pairs of interest points, the method takes the entire set of points, their local descriptors and the spatial configuration into account to find an optimal mapping of modeled object to target image. The image information is captured by symmetry-based interest points and local descriptors derived from Gradient Vector Flow. Experimental results are reported for two data-sets showing the applicability to complex medical data.