A. Criminisi, A. Blake, G. Smyth, C. Rother

 

Project description

    At Microsoft Research in Cambridge we are developing new machine vision algorithms to enhance desktop-based visual communication (e.g. web-cam based live chat). The i2i technology delivers: smart-framing for automatic camera management and better bandwidth usage; eye-gaze correction for improved speakers interaction; three-dimensional emoticon insertion; background processing and substitution for attention focusing and privacy protection; finally, high-quality image synthesis is achieved by our real-time SPS algorithm.
Demos        (click below to play videos)

overview of i2i (5.5Mb .wmv video clip)
wrong gaze corrected gaze wrong gaze

Eye gaze correction and image interpolation
original image smart framed

Smart Framing (automatic camera management)
original background substituted background

High-quality background substitution

Smart icons (3D object insertion)

Smart Focus (depth-driven image processing)

 

Stereo Software Development Kit
At MSRC we are developing a stereo vision software development kit. The SDK allows researchers to easily capture image frames from a synchronized, USB 2 stereo web cam and run various stereo algorithms on those. A number of example applications such as stereo-based head tracking, anaglyph generation and background substitution are also provided.

Go to the Stereo SDK page

 

Hardware

conventional single-camera configuration

i2i stereo web-cam configuration

a prototype of our i2i stereo camera

an i2i-powered messenger session
The key idea is using a stereo webcam rather than a conventional one. The stereo web-cam, coupled with our real-time stereo algorithm provides great enhancements in one-to-one visual communication.

 

Science The i2i team has discovered a way to compute accurate scene geometry from two cameras in real time. The powerful stereo algorithm described in our scientific publications  (below) is at the basis of the i2i technology.
Data Here is a number of rectified test stereo image pairs and sequences with ground-truth segmentation for other researchers to use.
Acknowledgements We are grateful to Cypress for providing a pair of camera development boards.

 

Scientific publications
  1. V. Kolmogorov, A. Criminisi, A. Blake, G. Cross, C. Rother. Bi-layer segmentation of binocular stereo video 2005 San Diego, CA, US Proc. IEEE Computer Vision and Pattern Recognition (CVPR).
  2. A. Criminisi, A. Blake. The SPS Algorithm: Patching Figural Continuity and Transparency by Split-Patch Search 2004 Washington DC, US Proc. IEEE Computer Vision and Pattern Recognition (CVPR).
  3. P. H.S. Torr, A. Criminisi. Dense Stereo Using Pivoted Dynamic Programming 2004 Image and Vision Computing (IVC).
  4. A. Criminisi, J. Shotton, A. Blake, C. Rother, P. H.S. Torr. Efficient Dense Stereo and Novel-view Synthesis for Gaze Manipulation in One-to-one Teleconferencing 2003 Cambridge, UK Microsoft Research.
  5. A. Criminisi, J. Shotton, A. Blake, P. H.S. Torr. Gaze Manipulation for One-to-one Teleconferencing Oct. 2003 Nice Proc. IEEE International Conference on Computer Vision (ICCV).
  6. A. Blake, P. H.S. Torr, I. Cox, A. Criminisi. Estimating uncertainty in dense stereo disparity maps Mar. 2003 Cambridge, UK Microsoft Research
back to Computer Vision @ MSRC

Web site designed and maintained by A. Criminisi