Motion Capture from Body-Mounted Cameras

Takaaki Shiratori1          Hyun Soo Park2          Leonid Sigal1
Yaser Sheikh2          Jessica K. Hodgins1,2
1Disney Research, Pittsburgh     2Carnegie Mellon University

Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large areas. In this paper, we present the theory and practice of using body-mounted cameras to reconstruct the motion of a subject. Outward-looking cameras are attached to the limbs of the subject, and the joint angles and root pose are estimated through non-linear optimization. The optimization objective function incorporates terms for image matching error and temporal continuity of motion. Structure-from-motion is used to estimate the skeleton structure and to provide initialization for the non-linear optimization procedure. Global motion is estimated and drift is controlled by matching the captured set of videos to reference imagery. We show results in settings where capture would be difficult or impossible with traditional motion capture systems, including walking outside and swinging on monkey bars. The quality of the motion reconstruction is evaluated by comparing our results against motion capture data produced by a commercially available optical system.

Snapshot for paper Takaaki Shiratori, Hyun Soo Park, Leonid Sigal, Yaser Sheikh, Jessica K. Hodgins
"Motion Capture from Body-Mounted Cameras"
ACM Transactions on Graphics, Vol. 30, No. 4 (also Proc. ACM SIGGRAPH 2011), July 2011
[Paper (pdf, 17MB)] [BiBTeX]
[Movie (QuickTime, 85MB)]

March 27, 2012: Featured by Inside Science Television.

Media Coverage

Copyright notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.