Poor Man's Virtual Camera: Real-Time Simultaneous Matting and Camera Pose Estimation

IEEE Comput Graph Appl. 2019 Nov-Dec;39(6):108-119. doi: 10.1109/MCG.2016.39. Epub 2016 Mar 18.

Abstract

Today's film and advertisement production heavily uses computer graphics combined with living actors by chroma keying. The matchmoving process typically takes a considerable manual effort. Semiautomatic matchmoving tools exist as well, but they still work offline and require manual check-up and correction. In this paper, we propose an instant matchmoving solution for green screen. It uses a recent technique of planar uniform marker fields. Our technique can be used in indie and professional filmmaking as a cheap and ultramobile virtual camera, and for shot prototyping and storyboard creation. The matchmoving technique based on marker fields of shades of green is very computationally efficient: we developed and present in this paper a mobile application running at 33 FPS. Our technique is thus available to anyone with a smartphone at low cost and with an easy setup, opening space for new levels of filmmakers' creative expression.

Publication types

  • Research Support, Non-U.S. Gov't