Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Viewport: A Fully Distributed Immersive Teleconferencing System with Infrared Dot Pattern

C. Zhang, Q. Cai, P. Chou, Z. Zhang, and R. Martin-Brualla

Abstract

In this paper, we present Viewport, a fully distributed system for immersive teleconferencing. The participant at each site is captured by a camera rig, which contains multiple color and infrared cameras and multiple infrared projectors. The projected infrared dot pattern is used to help reconstruct a 3D representation in real-time, which will be rendered in a shared virtual environment for immersive telecommunication. We conduct a multimodal fusion scheme for multiview stereo and 3D reconstruction, which combines the knowledge of the infrared dot patterns from the projectors and the observations of IR and RGB cameras. One novel aspect of our approach is the use of sparse point cloud instead of dense multiview stereo for geometry reconstruction, which leads to significant speed up in 3D reconstruction and rendering. In addition, we introduce the scheme of virtual seating, where the point clouds are positioned to maintain the same seating geometry as face-to-face meetings after careful calibration, such that the mutual gaze between participants are faithfully maintained.

Details

Publication typeArticle
Published inIEEE MultiMedia
URLhttp://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6461359
Pages17-27
Volume20
Number1
PublisherIEEE Computer Society

Previous versions

Cha Zhang, Qin Cai, Philip Chou, Zhengyou Zhang, and Ricardo Martin-Brualla. Viewport: A Fully Distributed Immersive Teleconferencing System with Infrared Dot Pattern, 1 April 2012.

> Publications > Viewport: A Fully Distributed Immersive Teleconferencing System with Infrared Dot Pattern