Getting the Most of Casual Visual Capture - Mohamed Sayed (University College London)
posted on 24 January, 2023


Abstract: When capturing images and video with a camera, there are many ways the capture could be ruined. The camera may be out of focus or be in the wrong place, the images may be blurry, or the subjects of interest could be out of frame. Not only do these errors result in footage of low aesthetic quality, but downstream vision tasks may suffer from reduced accuracy when trying to understand the world through subpar glasses. While the camera operator is usually to blame for these errors, they are not always voluntary. The user may be inexperienced, unable to set the correct settings and accurately position and orient their camera for optimal capture. In other cases, the user is simply preoccupied and unable to focus on deliberate and attention-consuming capture without compromising on another priority - their safety or enjoyment. In this work, we aim to make the most of subpar capture. We first tackle the problem of recovering object detection accuracy under ego-induced motion blur. We then take on the challenge of actively orienting the camera to frame actors for cinematographic filmmaking. Accurate 3D reconstruction requires diverse views, but these views are not always available, so for our third challenge we make the most of views from casual video for accurate depth estimation and mesh reconstruction.