Researchers in the Visual Computing Group are turning ideas into reality in areas that include computer vision, pattern recognition, machine learning, image/video editing, and retrieval. The projects push the state-of-the-art forward in a variety of areas such as face detection and tracking, face modeling and recognition, desktop photo management, image collage, video montage, video repairing, 2D and 3D object tracking, and Digital effects for MSN online video chat.

Digital Effects for MSN Messenger Demo Download

EasyAlbum Demo Download

Face Recognition Technologies

The goals of the Face Recognition Technologies project center on the development of algorithms and technologies for real-time, automated, and highly accurate face detection, tracking, alignment, and recognition. The team is designing systems that can adapt to different lighting conditions and variations in poses and expressions. Applications include human computer interface, graphics and animation, security and surveillance, and desktop photo management.

Pattern Recognition and Machine Learning

The goal of the Pattern Recognition and Machine Learning project is to understand fundamental problems in visual and audio information processing. The team is developing new techniques and algorithms for the analysis and classification of real world images, videos, and audio.

Digital Effects for Video

The increasing power and falling cost of computers, video cameras, and Internet connections is making it far easier for users to produce and post video clips on portals such as YouTube and Soapbox. Research is indispensable to developing products that allow simple-to-use video-effect tools to provide entertainment and fun to videos. Existing technologies such as blurring, color correction, and enhancement are simple to use, but do not add as much fun to viewing videos as we believe is possible. We have been working on a set of new video-editing tools to make it easy for users to add special digital effects to video clips to make viewing even more entertaining and enjoyable. The tools include:

• Video-object insertion: embedding synthetic 3-D objects or hyperlinks.
• Video-object segmentation: separating foreground and background.
• Video repairs: restoring corrupted video portions.
• Video montages: quick browsing of lengthier video clips.
• Online video effects for MSN video chat (a working demo for MSN can be downloaded at: http://research.microsoft.com/vc/DigitalEffectDownload.aspx).

Human Motion Analysis and Synthesis

The Visual Computing Group is developing algorithms and technologies to track, recognize, and synthesize complex human motions. Based on a statistical model, the key features are:

1. Analyzing motion captured data and effectively modeling low-dimensional linear subspace of motion data embedded in high-dimensional nonlinear space;
2. Modeling the dynamics and kinematics of human motion based on automatic control theory;
3. Modeling statistical distribution of complex human motions;
4. Synthesizing realistic human motions under various constraints.

Primary Contact: Xiaoou Tang
Liang, Lin	Lin, Zhouchen	Matsushita, Yasuyuki	Sun, Jian
Tang, Xiaoou	Wang, Qiang	Wei, Yichen	Wen, Fang
Wilburn, Bennett	Xiao, Rong	Xu, Ying-Qing	Yang, Qiong
Zhang, Weiwei
Affiliate Members
Kang, Sing Bing	Lin, Steve	Ofek, Eyal	Shum, Harry
Szeliski, Rick	Tong, Xin	Wei, Hao	Zhang, Lei
Zhang, Zhengyou

Xiaobo Lu Tsinghua University

Lijun Sun Beijing Film Academy

Stan Z. Li Chinese Academy of Sciences

Ce Liu Massachusetts Institute of Technology

Hua Zhong Carnegie Mellon University

Yan Li Carnegie Mellon University

Yi Zhou Carnegie Mellon University

CHAI, Jinxiang, Carnegie Mellon University
CHEN, Hong, University of California, Los Angeles
CHENG, Chiyi, Intel China (Shanghai)
FENG, Wei, City University of Hong Kong
GONG, Dian, Tsinghua University
HU, Xing, Beijing University of Aeronautics and Astronautics
HE, Junfeng, Hong Kong University of Science and Technology
LI, Yin, Microsoft Research, Asia
LI, Yong, Tsinghua University
LI, Yuanzhen, Massachusetts Institute of Technology (MIT)
LIN, Dahua, Chinese University of Hong Kong
LIN, Zhouchen, Microsoft Research Asia
LIU, Bo, Tsinghua University
LIU, Feng, University of Wisconsin, Madison
LIU, Hongze, University of Utah
LIU, Zhiqiang, University of California, Los Angeles
LU, Yue, University of Illinois at Urbana-Champaign
SHEN, Ji, Florida State University
STUMPFEL, Jessi California Institute of Technology
SU, Sara, Massachusetts Institute of Technology
SU, Yuanyuan, Beijing Film Academy
SUN, Jian, Microsoft Research, Asia
TA, Ping, Hong Kong University of Science and Technology
WAN, Liang, Chinese University of Hong Kong
WANG, Jue, University of Washington
WANG, Rongrong, Oracle China
WANG, Tianshu, IBM Research Center, China
WU, Chenyu, Carnegie Mellon University (CMU)
WU, Qing, University of Illinois at Urbana-Champaign
XU, Songhua, Yale University
YANG, Yanning, Beijing University Health Science Center
YE, Ding, Rochester Institute of Technology
YIN, Kangkang, University of British Columbia
YU, Feng, Intel China (Shanghai)
YUAN, Lu, Hong Kong University of Science and Technology
ZHANG, Chaolin, State University of New York at Stony Brook
ZHANG, Jinyun, Motorola China

1. Image Vectorization using Optimized Gradient Meshes
   Jian Sun, Lin Liang, Fang Wen, Heung-Yeung Shum.Image Vectorization using Optimized Gradient Meshes.SIGGRAPH 2007
2. Offline Signature Verification Using Online Handwriting Registration
   Yu Qiao, Jianzhuang Liu,Xiaoou Tang. Offline Signature Verification Using Online Handwriting Registration, CVPR 2007
3. EasyAlbum: An Interactive Photo Annotation System Based on Face Clustering and Re-ranking
   Jingyu Cui, Fang Wen, Rong Xiao and Xiaoou Tang. EasyAlbum: An Interactive Photo Annotation System Based on Face Clustering and Re-ranking. SIGCHI 2007.
4. 拖放图像粘贴
   Jiaya Jia (Chinese University of Hong Kong), Jian Sun, Chi-Keung Tang (Hong Kong University of Science and Technology), Heung-Yeung Shum. Drag-and-Drop Pasting. SIGGRAPH 2006
5. 色彩谐调之研究
   Daniel Cohen-Or (Tel Aviv University), Olga Sorkine (Tel Aviv University), Ran Gal (Tel Aviv University), Tommer Leyvand (Tel Aviv University), Ying-Qing Xu. Color Harmonization. SIGGRAPH 2006
6. 基于闪光的高精度前景物体分割
   Jian Sun, Yin Li, Sing Bing Kang, Heung-Yeung Shum. Flash Matting. SIGGRAPH 2006
7. 基于网格编织的几何纹理合成
   Kun Zhou, Xin Huang, Xi Wang, Yiying Tong, Mathieu Desbrun, Baining Guo, Heung-Yeung Shum. Mesh Quilting For Geometric Texture Synthesis. SIGGRAPH 2006
8. 基于表观流型的时变材质表观建模方法
   Jiaping Wang, Xin Tong, Stephen Lin, Minghao Pan, Chao Wang, Hujun Bao, Baining Guo, Heung-Yeung Shum. Appearance Manifolds for Modeling Time-Variant Appearance of Materials. SIGGRAPH 2006.
9. 动态场景中基于球面调和指数的实时软影绘制
   Zhong Ren, Rui Wang, John Snyder, Kun Zhou, Xinguo Liu, Bo Sun, Peter-Pike Sloan, Hujun Bao, Qunsheng Peng, Baining Guo. Real-time Soft Shadows in Dynamic Scenes using Spherical Harmonic Exponentiation. SIGGRAPH 2006.
10. 子空间梯度域的网格形变
    Jin Huang, Xiaohan Shi, Xinguo Liu, Kun Zhou, Li-Yi Wei, Shang-Hua Teng, Hujun Bao, Baining Guo, Heung-Yeung Shum. Subspace Gradient Domain Mesh Deformation. SIGGRAPH 2006.
11. Color Sketch Generation
    Fang Wen, Qing Luan, Lin Liang, Ying-Qing Xu and Heung-Yeung Shum. Color Sketch Generation. NPAR 2006.
12. Accurate Face Alignment using Shape Constrained Markov Network
    Lin Liang, Fang Wen, Ying-Qing Xu, Xiaoou Tang and Heung-Yeung Shum, Accurate Face Alignment using Shape Constrained Markov Network, Proc. CVPR 2006
13. An Integrated Model for Accurate Shape Alignment
    Liang Liang, Fang Wen, Xiaoou Tang, Yingqing Xu, An Integrated Model for Accurate Shape Alignment, Proc. ECCV 2006
14. Background Cut
    Jian Sun, Weiwei Zhang, Xiaoou Tang, Heung-Yeung, Background Cut, Proc. ECCV 2006
15. Joint Boosting Feature Selection for Robust Face Recognition
    Rong Xiao, Wujun Li, Yuandong Tian, Xiaoou Tang, Joint Boosting Feature Selection for Robust Face Recognition, Proc. CVPR 2006
16. An Intensity Similarity Measure in Low-Light Conditions
    F. Alter, Yasuyuki Matsushita, Xiaoou Tang, An Intensity Similarity Measure in Low-Light Conditions, Proc. ECCV 2006
17. The Design of High-Level Features for Photo Quality Assessment
    Yan Ke, Xiaoou Tang, Feng Jing, The Design of High-Level Features for Photo Quality Assessment, Proc. CVPR 2006
18. Picture Collage
    Jingdong Wang, Jian Sun, Long Quan, Xiaoou Tang, Heung-Yeung Shum, Picture Collage, Proc. CVPR 2006
19. Video Completion by Motion Field Transfer
    Takaaki Shiratori, Yasuyuki Matsushita, Sing Bing Kang, Xiaoou Tang, Video Completion by Motion Field Transfer, Proc. CVPR 2006
20. Space-Time Video Montage
    Hong-Wen Kang, Yasuyuki Matsushita, Xiaoou Tang, Xue-Quan Chen, Space-Time Video Montage, Proc. CVPR 2006
21. Automatic 3D Face Modeling from Video
    Le Xin, Qiang Wang, Jianhua Tao, Xiaoou Tang, Tieniu Tan, and Harry Shum, “Automatic 3D Face Modeling fromVideo,” Proc. IEEE ICCV, Volume 2, Page(s):1193 - 1199, 2005.
22. Patch Based Blind Image Super Resolution
    Qiang Wang, Xiaoou Tang, Harry Shum, “Patch Based Blind Image Super Resolution,” Proc. IEEE ICCV, Volume 1, Page(s):709 - 716, 2005
23. Modeling Hair from Multiple Views
    Y. Wei, Eyal Ofek, L. Quan, Heung-Yeung Shum. Modeling Hair from Multiple Views. SIGGRAPH 2005.
24. Image Completion with Structure Propagation
    Jian Sun, Lu Yuan, Jiaya Jia, Heung-Yeung Shum. Image Completion with Structure Propagation. SIGGRAPH 2005.
25. Video Object Cut and Paste
    Yin Li, Jian Sun, Heung-Yeung Shum. Video Object Cut and Paste. SIGGRAPH 2005.
26. A Bayesian Mixture Model for Multi-view Face Alignment
    Yi Zhou, Wei Zhang, Xiaoou Tang, Harry Shum, A Bayesian Mixture Model for Multi-view Face Alignment, Proc. CVPR 2005
27. Bi-directional Tracking using Trajectory Segment Analysis
    Jian Sun, Weiwei Zhang, Xiaoou Tang, Heung-Yeung Shum, Bi-directional Tracking Using Trajectory Segment Analysis, Proc. ICCV 2005
28. Detecting Doctored Images Using Camera Response Normality and Consistency
    Zhouchen Lin, Rongrong Wang, Xiaoou Tang, Heung-Yeung Shum, Detecting Doctored Images Using Camera Response Normality and Consistency, Proc. CVPR 2005
29. Detecting Doctored JPEG Images Via DCT Coefficient Analysis
    Zhouchen Lin, Rongrong Wang, Xiaoou Tang, Heung-Yeung Shum, Detecting Doctored Images Using Camera Response Normality and Consistency Analysis, Proc. CVPR 2005
30. A Theory of Inverse Light Transport
    Steven M. Seitz, Yasuyuki Matsushita, Kiriakos N. Kutulakos. A Theory of Inverse Light Transport, Proc. ICCV 2005 University of Toronto
31. Interactive Shape from Shading
    Gang Zeng, Yasuyuki Matsushita, Long Quan, Heung-Yeung Shum, Interactive Shape from Shading, Proc. CVPR 2005
32. A Symmetric Patch-Based Correspondence Model for Occlusion Handling
    Yi Deng, Qiong Yang, Xueyin Lin, Xiaoou Tang, A Symmetric Patch-Based Correspondence Model for Occlusion Handling, Proc. ICCV 2005
33. Symmetric Stereo Matching for Occlusion Handling
    Yi Deng, Qiong Yang, Xueyin Lin, Xiaoou Tang, A Symmetric Patch-Based Correspondence Model for Occlusion Handling, Proc. ICCV 2005
34. Full-frame Video Stabilization
    Yasuyuki Matsushita, Eyal Ofek, Xiaoou Tang, Heung-Yeung Shum, Full-frame Video Stabilization, Proc. CVPR 2005
35. Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance
    Yasuyuki Matsushita, Ko Nishino, Katsushi Ikeuchi, and Masao Sakauchi, Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 26, NO. 10, OCTOBER 2004
36. Video Tooning
    Jue Wang, Yingqing Xu, Heung-Yeung Shum, Michael F. Cohen,"Video Tooning", ACM Trans. on Graphics (Proc. of SIGGAPH2004).
37. A Geometric Analysis of Light Field Rendering
    Zhouchen Lin and Heung-Yeung Shum, A Geometric Analysis of Light Field Rendering, International J. of Computer Vision, Vol. 58, No. 2, pp. 121-138, July 2004.
38. Example-Based Composite Sketching of Human Portraits
    Hong Chen,Ziqiang Liu, Chunk Rose, Yingqing Xu, Heung-Yeung Shum, David Salesin,"Example-Based Composite Sketching of Human Portraits", NPAR, 2004
39. Bayesian Correction of Image Intensity with Spatial Consideration.
    Jiaya Jia, Jian Sun, Chi-Keung Tang, and Heung-Yeung Shum. Bayesian Correction of Image Intensity with Spatial Consideration. ECCV 2004.
40. Synthesizing Dynamic Texture with Closed-Loop Linear Dynamic System
    Lu Yuan, Fang Wen, Ce Liu, and Heung-Yeung Shum,Synthesizing Dynamic Texture with Closed-Loop Linear Dynamic System, ECCV 2004
41. Estimating Intrinsic Images from Image Sequences with Biased Illumination
    Yasuyuki Matsushita, Stephen Lin, Sing Bing Kang, and Heung-Yeung Shum, Estimating Intrinsic Images from Image Sequences with Biased Illumination, ECCV2004
42. Image and Video Segmentation by Anisotropic Kernel Mean Shift
    Jue Wang, Bo Thiesson, Yingqing Xu, and Michael F. Cohen (2004). "Image and video segmentation by anisotropic mean shift". In Proceedings of European Conference on Computer Vision, pages 238-249. Springer-Verlag, May 2004.
43. Lazy Snapping
    Yin Li, Jian Sun, Chi-Keung Tang and Heung-Yeung Shum. Lazy Snapping. SIGGRAPH 2004. (ACM Transaction on Graphics, Vol 23, No. 3, April 2004.)
44. Pop-Up LightField
    Heung-Yeung Shum, Jian Sun, Shuntaro Yamazaki, Yin Li and Chi-Keung Tang. Pop-Up LightField. ACM Transaction on Graphics, Vol 23, No. 2, April 2004.
45. Poisson Matting
    Jian Sun, Jiaya Jia, Chi-Keung Tang and Heung-Yeung Shum. Poisson Matting. SIGGRAPH 2004. (ACM Transaction on Graphics, Vol 23, No. 3, April 2004.)
46. A Hybrid Approach to Rendering Handwritten Characters
    Sara L. Su, Chenyu Wu, Ying-Qing Xu, and Heung-Yeung Shum, A Hybrid Approach to Rendering Handwritten Characters, In Proceedings of WSCG, Plzen, Czech Republic, February 2004
47. Robust Multi-Pose Face Detection in Images
    R. Xiao, M.J. Li, H.J. Zhang, "Robust Multi-Pose Face Detection in Images", IEEE Transactions on Circuits and Systems for Video Technology, Vol:14, Issue:1 , Pages:31 – 41, Jan. 2004
48. Feature Selection on Combinations for Efficient Learning from Images
    R.Xiao, L. Zhang, H.J. Zhang, “Feature Selection on Combinations for Efficient Learning from Images”, In Proceedings of the Asian Conference on Computer Vision. vol. 2, pp. 1074–1079.Korea. January 2004.
49. Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation
    Zhouchen Lin and Heung-Yeung Shum, Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 26, No. 1, pp. 83-97, 2004.
50. Off-line Signature Verification Incorporating the Prior Model
    Liang Wan, Zhouchen Lin, Rong-Chun Zhao, Off-line Signature Verification Incorporating the Prior Model, International Conference on Machine Learning and Cybernetics (ICMLC2003), Xi'an, China, pp. 1602 - 1606, 2-5 November, 2003.
51. Boosting Chain Learning for Object Detection
    R. Xiao, L. Zhu, and H.J. Zhang. "Boosting Chain Learning for Object Detection", ICCV03, Nice, France.
52. Ranking Prior Likelihood Distributions for Bayesian Shape Localization Framework
    Shuicheng Yan, Mingjing Li, Hongjiang Zhang, Qiansheng Cheng, Ranking Prior Likelihood Distributions for Bayesian Shape Localization Framework, ICCV 2003
53. Stereo Matching using Belief Propagation.
    Jian Sun, Heung-Yeung Shum and Nan-Ning Zheng. Stereo Matching using Belief Propagation. PAMI, Vol.25, No. 7, July 2003
54. Bayesian Tangent Shape Model: Estimating Shape and Pose Parameters via Bayesian Inference
    Yi Zhou, Lie Gu, Hong-Jiang Zhang, Bayesian Tangent Shape Model: Estimating Shape and Pose Parameters via Bayesian Inference, Computer Vision and Pattern Recognition, 2003
55. Generic Image Hallucination with Primal Sketch Prior.
    Jian Sun, Nan-Ning Zheng, Hai Tao and Heung-Yeung Shum. Generic Image Hallucination with Primal Sketch Prior. CVPR 2003.
56. Correction and Rectification of Light Fields
    Ke Deng, Lifeng Wang, Zhouchen Lin, Tao Feng, and Zhidong Deng. Correction and Rectification of Light Fields, Computers & Graphics, Vol. 27, No. 2, pp. 169-177, 2003.
57. Face alignment using texture-constrained active shape models
    Shuicheng Yan, Ce Liu, Stan Z. Li, Hongjiang Zhang, Heung-Yeung Shum, Qiansheng Cheng, Face alignment using texture-constrained active shape models, Image and Vision Computing 21 (2003) 69–75

Position Title: Researcher/Associate Researcher/Postdoc
Working location: Beijing, P. R. China

The Visual Computing Group of Microsoft Research Asia, Beijing, China, has several immediate openings for Researcher/Associate Researcher/Postdoc in the areas of pattern recognition, computer vision, content based image retrieval, image processing, computer graphics, and applied machine learning.

Candidates are expected to hold a Ph.D. or MS in CS, EE or related fields with a strong publication record in major journals and conferences. Candidates need to demonstrate strong interpersonal skills, be able to work independently on problems, be an effective team player and be willing to accept challenges and responsibility.

Please send resume to Dr. Xiao Rong at rxiao@microsoft.com.