Overview

Devices such as PCs, Pocket PCs, Smartphones, and embedded devices are taking on a greater and greater role in everyday computing and the lives of individuals. At the same time, the growth in bandwidth of wired and wireless networks and local area and wide area networks is facilitating greater communication across platforms. At Microsoft Research Asia, the Media Communication Group is developing applications and services to advance communication, collaboration, and entertainment among various platforms and under different network conditions.

Researchers in the Media Communication Group leverage their expertise in audio/video compression, image processing, networking, user interface, and database management to develop practical applications and services for both consumers and information workers. The group’s research is used across the realm of communications applications, including video broadcast, instant messaging, video conferencing, multi-view video, and peer-to-peer video streaming. The group’s innovations have made an impact on a number of Microsoft products and technologies.

Projects

Peer-to-peer Video Broadcast System
MSR Asia Peer-to-peer (P2P) Video Broadcast System is a complete software package for broadcasting live video streams on the Internet. By leveraging the power of P2P network, the system can broadcast live video to a large number of users using only a small portion of server bandwidth. The main components of the system include:

• A high performance P2P real-time communication library with unified programming interface. It also consists of a distributed NAT traversal moduel and a congestion control module.
• A P2P broadcast service composed of video publish server, user registration server and watching client module. It is designed to be seamlessly integrated with Windows Media Player.
• A performance monitor system for the deployed P2P streaming service that can report online statistics as well as performing some advanced performance analysis for the P2P overlay.
• A web-based management system for managing a number of servers and video channels.

Lecture Broadcast System
Lecture Broadcast System (LBS) is a live lecture and video broadcast system based on P2P streaming technology. It includes:

• Channel-based program scheduling and management by seamless cooperation with Outlook;
• Real-time video/audio capture and sharing;
• Stream transmission based on P2P technology;
• Real-time PowerPoint slides capture and uploading;
• Browser-based synchronous playback of video/audio and PowerPoint slides at the client side.

The software kit of Lecture Broadcast System, including lecture program manage software, video/audio capturing and uploading software, PowerPoint slides capturing software, and IE plug-in for stream media playback at client side, can also serve as a general solution for constructing lecture live broadcasting or distance education systems.

A Performance Tool For Distributed Systems
Distributed systems are becoming key engines of IT companies for data centric computing. However, it is very hard to diagnose and improve the performance of a distributed system. Through analyzing the performance of a distributed system step by step, we demonstrate a series of technologies, including inspecting a running distributed system, non-intrusively aligning machine time, creating system snapshot, calculating running cost, digging process relationship and application topology, and detecting bottleneck.

Real-time Interactive Multi-view Video System
Pre-captured movies and broadcast television (TV), live digital television, video-on-demand (VoD), streaming video over the Internet, and improvements in video technology have had a significant impact over the years by consistently delivering better quality pictures, more programming, and greater flexibility. Despite progress in these fields, traditional video formats are limited to one video stream for an event at any time and one viewing direction, as selected by program editors. Today’s technology places the viewer in a passive mode, having to accept the angle or views presented by a program/event’s editor.

The rapid development of electronic and computing technologies has allowed the Media Communication Group to expand the realm of possibilities with the development of a Real-time Interactive Multi-view Video System and a completely new video format. This new system and video format allow for more proactive participation by viewers, giving them the freedom to choose a viewing angle and enjoy special visual effects. It also delivers a new viewing experience and a high degree of user interactivity. The system is widely applicable to movies, advertisements, educational videos, sports games, and event broadcasts.

Microsoft Portrait
Microsoft Portrait is a research prototype for mobile video communication. It supports .NET Messenger Service, Session Initiation Protocol and Internet Locator Service on PCs, Pocket PCs, Handheld PCs, and Smartphones. It runs on local area networks, dialup networks, and even wireless networks with bandwidths as low as 9.6 kilobits/second. Microsoft Portrait delivers portrait-like video if users are in low bandwidths, and displays full-color video if users are in broadband. In low bandwidths, portrait video possesses clearer shapes, smoother motion, shorter latency, and much cheaper computational costs than conventional video technologies. Research in Microsoft Portrait is also pursuing presence notification, chat/voice/video functions at any time and anywhere on any device.

Peer-to-peer Communication
Current instant messaging applications are all based on servers. If servers crash, all connections are broken. Our system connects users through peer-to-peer routing as well as existing instant messaging applications. Using our system, user can attend multiple peer-to-peer text/voice/video meetings in the same time with each meeting allowing multiple attendees.

Instant Voice Communication
Instant voice communication is receiving renewed interests in the cellular world. Known as push-to-talk (PTT), this half-duplex service provides a lightweight complement to the telephone communication. PTT over cellular (PoC) was first introduced by US operator Nextel. Now, it is provided by almost all the dominant mobile operators, and there are undergoing standardization efforts in Open Mobile Alliance (OMA).

PTT can coexist with the full-duplex telephone service not only because of its cost effectiveness, but it offers increased privacy and requires reduced interactional commitment in comparison to telephone. Similar reasons have boosted the success of the instant messaging (IM) service. While the dominant IM services are based on text, we see the potential of voice IM for the communication between both end consumers and information workers.

In this project, we evaluate the possibility of providing PTT service over IP network, and seek to find a unified solution that could bridge the PTT service between cellular and IP networks. We have designed and implemented an asynchronous record-and-forward voice messaging tool as well as an on-line multiparty PTT system. The prototype of the former one was transferred MSN Messenger team. They released it as Voice Clip feature in version 7.5.

Publications

1. A Multiparty Videoconferencing System Over an Application-Level Multicast Protocol
   Chong Luo, Wei Wang, Jian Tang, Jun Sun, and Jiang Li, A Multiparty Videoconferencing System Over an Application-Level Multicast Protocol, IEEE Transactions on Multimedia, Volume 9, Issue 8, Dec. 2007.
2. Distributed Density Estimation Using Non-parametric Statistics
   Yusuo Hu, Hua Chen, Jian-Guang Lou, and Jiang Li, Distributed Density Estimation Using Non-parametric Statistics, The 27th International Conference on Distributed Computing Systems, 25-27, June 2007.
3. An Epipolar Geometry-Based Fast Disparity Estimation Algorithm for Multiview Image and Video Coding
   Jiangbo Lu, Hua Cai, Jian-Guang Lou, and Jiang Li, An Epipolar Geometry-Based Fast Disparity Estimation Algorithm for Multiview Image and Video Coding, IEEE Transactions on Circuits and Systems for Video Technology, Volume 17, Issue 6, Pages 737-750, June 2007.
4. Color Image Coding by using Inter-Color Correlation
   Xing San, Hua Cai, and Jiang Li, Color Image Coding by Inter-color Correlation, IEEE International Conference on Image Processing, Atlanta, GA, USA, 8-11 October, 2006.
5. An Effective Epipolar Geometry Assisted Motion Estimation Technique for Multi-View Image and Video Coding
   Jiangbo Lu, Hua Cai, Jian-Guang Lou, and Jiang Li, An Effective Epipolar Geometry Assisted Motion Estimation Technique for Multi-View Image and Video Coding, 2006 IEEE International Conference on Image Processing, Atlanta, GA, USA, Pages 1089-1092, 8-11 October 2006.
6. Multicast Of Real-time Multi-view Video
   Li Zuo, Jian-Guang Lou, Hua Cai, and Jiang Li, Multicast of Real-time Multi-view Video, 2006 IEEE International Conference on Multimedia & Expo, Toronto, Ontario, Canada, 9-12 July, 2006
7. Estimating Available Bandwidth Using Multiple Overloading Streams
   Minjian Zhang, Chong Luo, and Jiang Li, Estimating Available Bandwidth Using Multiple Overloading Streams, IEEE International Conference on Communications, Istanbul, Turkey, 11-15 June 2006.
8. A Novel Model-Based Rate-Control Method for Portrait Video Coding
   Keman Yu, Jiang Li, Cuizhu Shi, and Shipeng Li, A Novel Model-Based Rate-Control Method for Portrait Video Coding, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 15, No. 12, December 2005, pages: 1545-1556.
9. A RealTime Interactive Multi-View Video System
   Jian-Guang Lou, Hua Cai, Jiang Li, A Real-Time Interactive Multi-View Video System, The 13th ACM International Conference on Multimedia (ACMMM 2005), Singapore, 6-11 November 2005, pages: 161-170.
10. Embedded Image Coding With Context Partitioning and Quantization
    Hua Cai, Xing San, and Jiang Li, Embedded Image Coding With Context Partitioning and Quantization, 2005 IEEE International Conference on Image Processing (ICIP 2005), Genova, Italy, 11-14 September 2005.
11. Lossless Image Compression With Tree Coding of Magnitude Levels
    Hua Cai and Jiang Li, Lossless Image Compression With Tree Coding of Magnitude Levels, 2005 IEEE International Conference on Multimedia & Expo (ICME 2005), Amsterdam, The Netherlands, 6-8 July 2005.
12. A Framework for Fine-granular Computational-complexity Scalable Motion Estimation
    Zhi Yang, Hua Cai, and Jiang Li, A Framework for Fine-granular Computational-complexity Scalable Motion Estimation, The 2005 IEEE International Symposium on Circuits and Systems (ISCAS 2005), Kobe, Japan, 23-26 May, 2005.
13. An Effective Variable Block-Size Early Termination Algorithm for H.264 Video Coding
    Libo Yang, Keman Yu, Jiang Li, and Shipeng Li. An Effective Variable Block-Size Early Termination Algorithm for H.264 Video Coding, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 15, No. 6, June 2005. Pages: 784-788.
14. A Low Complexity Motion Compensated Frame Interpolation Method
    Jiefu Zhai, Keman Yu, Jiang Li, and Shipeng Li, A Low Complexity Motion Compensated Frame Interpolation Method, The 2005 IEEE International Symposium on Circuits and Systems (ISCAS 2005), Kobe, Japan, 23-26 May 2005.
15. Prediction-based Directional Fractional Pixel Motion Estimation for H.264 Video Coding
    Libo Yang, Keman Yu, Jiang Li, and Shipeng Li, Prediction-based Directional Fractional Pixel Motion Estimation for H.264 Video Coding, 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2005), Philadelphia, PA, USA, 18-23 March, 2005.
16. DigiMetro - An Application-Level Multicast System for Multi-party Video Conferencing
    Chong Luo, Jiang Li, and Shipeng Li, DigiMetro - An Application-Level Multicast System for Multi-party Video Conferencing, 2004 IEEE Global Telecommunication Conference (Globecom 2004), Dallas, Texas, 29 November - 3 December 2004.
17. DigiParty- A Decentralized Multi-party Video Conferencing System
    Ling Chen, Chong Luo, Jiang Li, and Shipeng Li, DigiParty - A Decentralized Multi-party Video Conferencing System, 2004 IEEE International Conference on Multimedia & Expo (ICME 2004), Taipei, Taiwan, 27 - 30 June 2004.

Overview

• People
• Projects
• Publications
• Asia Lab