This paper presents a depth-based defocus map estimation method from a single camera with multiple off-axis apertures. The proposed estimation algorithm consists of two steps: (i) object distance estimation using off-axis apertures and (ii) defocus map estimation based on the object distance. The proposed method can accurately estimate the defocus map using object distances that are well-characterized in a color shift model-based computational camera. Experimental results show that the proposed method outperforms the state-of-the-art defocus estimation methods in the sense of both accuracy and the estimation range. The proposed defocus map estimation method is suitable for multifocusing, refocusing, and extended depth of field (EDoF) systems.