Fusion4D: Real-time Performance Capture of Challenging Scenes
- Mingsong Dou ,
- Sameh Khamis ,
- Yury Degtyarev ,
- Philip Davidson ,
- Sean Ryan Fanello ,
- Adarsh Kowdle ,
- Sergio Orts Escolano ,
- Christoph Rhemann ,
- David Kim ,
- Jonathan Taylor ,
- Pushmeet Kohli ,
- Vladimir Tankovich ,
- Shahram Izadi
ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH 2016 | , Vol 35
We contribute a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time. Our algorithm supports both incremental reconstruction, improving the surface estimation over time, as well as parameterizing the nonrigid scene motion. Our approach is highly robust to both large frame-to-frame motion and topology changes, allowing us to reconstruct extremely challenging scenes. We demonstrate advantages over related real-time techniques that either deform an online generated template or continually fuse depth data nonrigidly into a single reference model. Finally, we show geometric reconstruction results on par with offline methods which require orders of magnitude more processing time and many more RGBD cameras.
Fusion4D: Real-time Performance Capture of Challenging Scenes
We contribute a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time. Our algorithm supports both incremental reconstruction, improving the surface estimation over time, as well as parameterizing the nonrigid scene motion. Our approach is highly robust to both large frame-to-frame motion and topology changes, allowing us to reconstruct extremely challenging scenes. We demonstrate advantages over related real-time techniques that either deform an online generated template or continually fuse depth data non-rigidly into a single reference model. Finally, we show geometric reconstruction results on par with offline methods which require orders of magnitude more processing time and many more RGBD cameras.