Mnemonic deficits in patients with medial temporal lobe (MTL) damage arising from temporal lobe epilepsy (TLE) are traditionally constrained to long-term episodic memory, sparing short-term and working memory (WM). This view of WM as being independent of MTL structures has recently been challenged by a small number of patient and neuroimaging studies, which have focused primarily on visual and visuospatial WM. In the present study we investigated material-specific lateralization of WM in 96 patients with unilateral damage to MTL stemming from TLE (56 left) and 30 control subjects using a pair of matched verbal and visuospatial supraspan tasks. Patients with unilateral TLE were impaired on both verbal and visuospatial WM tasks irrespective of the affected hemisphere. Patients with unilateral right TLE showed an additional deficit for visuospatial WM capacity when contrasted with patients with left TLE, whereas patients with unilateral left TLE showed increased intrusion errors on the verbal task when compared to patients with right TLE. These findings suggest a material-specific lateralization of WM in the MTL.