We demonstrate a compact lensless microscope which can capture video-rate phase contrast images of moving objects and allows numerical scanning of the focal distance after recording. Using only an RGB-detector and illumination from a single mode fiber, diffraction patterns at three wavelengths are recorded simultaneously, enabling high-speed data collection and reconstruction of phase and amplitude. The technique is used for imaging of a moving test target, beads in a flow cell, and imaging of Caenorhabditis elegans moving in a droplet of liquid.