In this Letter, we demonstrate the application of light field imaging to endoscopy. By introducing a microlens array into the image plane of a conventional endoscope, the 4D light field can be captured in one snapshot. This information can be used to obtain perspective images and to digitally refocus to different planes. These features allow for the recovery of 3D information in minimally invasive surgery. Important optical setup and performance parameters are derived to enable task specific engineering of the light field imaging system.