Memory plays a critical role in time estimation, yet detailed mechanisms underlying temporal memory have not been fully understood. The current functional magnetic resonance imaging (fMRI) study investigated memory phenomena in absolute identification of time durations and line lengths. In both time and length identification, participants responded faster to end-of-range stimuli (e.g., the shortest or longest items of the stimulus set) than to middle stimuli. Participants performed worse in the incongruent condition (mismatch between time and length in the stimulus position) than in the congruent condition, indicating cross-dimensional interference between time and length. Both phenomena reflect increased difficulty of retrieving information relevant to the current context in the presence of context-irrelevant information. A region in the lateral inferior prefrontal cortex showed a greater response to the middle stimuli and in the incongruent condition suggesting greater demands for controlled memory retrieval. A cognitive model based on the ACT-R (Adaptive Control of Thought - Rational) declarative memory mechanisms accounted for the major behavioral and imaging results. The results suggest that contextual effects in temporal memory can be understood in terms of domain-general memory principles established outside the time estimation domain.