Memory colors generated continuous interest in the color community. Previous studies focused on reflecting color chips and color samples in real scenes or on monitors. The cognitive effect of culture was rarely considered. In this paper, we performed a comprehensive investigation of the long-term memory colors of 26 familiar objects using the asymmetric color matching method among Chinese and German observers on a display. Three experiments were conducted to evaluate the variations introduced by culture, context-based gray image, and initial matching color. Memory colors of important objects were collected and representative memory colors were quantified in terms of CIELAB L*, a*, and b* values. The intra- and inter-observer variations were analyzed by mean-color-difference-from-mean values and chromatic ellipses. The effects of different cultural groups and experimental settings were also shown.