Visual Echo Cancellation for Seamless Integration of Remote Sites
In a typical remote collaboration setup, two or more projector-camera pairs are “cross-wired?to form a full-duplex system for two-way communication. A whiteboard can be used as the projector screen, and in that case, the whiteboard serves as an output device as well as an input device. Users can write on the whiteboard to comment on what is projected or to add new thoughts in the discussion. That is, images from the projector are mixed with real objects (such as papers, writings, and hands) to create a shared space.
In such a setup, the captured video contains writings or user gesture (foreground) along with projected contents. If we simply send the image for display on the other end, there could be a feedback loop that will distort the projected image. After a few frames, some part of the image becomes saturated and some part of the real writing has ghosting effect. Analogous to audio echo in telephone communication, we call this effect visual echo, i.e., the appearance of the projected contents viewed by the camera. Our approach to visual echo cancellation includes an off-line calibration procedure that records the geometric and photometric transfer between the projector and the camera in a look-up table. During run-time, projected contents in the captured video are identified using the calibration information and suppressed, therefore achieving the goal of canceling visual echo.
A video filming a live demo is available here (19.4MB).
"Computer Vision Technologies for Remote Collaboration
Using Physical Whiteboards, Projectors and Cameras", in Workshop on Computer
Vision for Interactive and Intelligent Environments (CV4IIE 2005), pp. 109-122,
IEEE Press, May 2006. PDF file.
"Robust and Accurate Visual Echo Cancellation in a Full-duplex Projector-camera System", in IEEE International Workshop on Projector-Camera Systems (ProCams), New York, NY, USA, June 17, 2006. PDF file.
"Visual echo cancellation in a projector-camera-whiteboard system", in Proc. International Conference on Image Processing (ICIP), Vol. 5, pp. 2885-2888, Oct.24-27, 2004. PDF file.