In non-contacting depth extraction there are several issues, such as the accuracy and the measurement speed. In the issue of the measurement speed, the computation cost for image processing is significant. We present an all-optical depth extraction method by coloring objects according to their depth. Our system is operated fully optically and both encoding and decoding processes are optically performed. Therefore, all-optical depth coloring has a distinct advantage to extract the depth information in real time without any computation cost. We invent a directional gating method to extract the points from the object which are positioned at the same distance. Based on this method, the objects look painted by different colors according to the distance when the objects are observed through our system. In this paper, we demonstrate the all-optical depth coloring system and verify the feasibility of our method.