The Media Computing group focuses its research efforts on data-centric media computing platforms that collect, find, aggregate, and understand media and related data to provide a new user media experience.
It also handles new media data types, such as informatics data for digital healthcare. The goal is to build an intelligent media computing platform that provides people with services and empowers them with knowledge.
Areas of Focus:
Multimedia Search and Mining
The explosion of multimedia data brings us new challenges with regards to multimedia content understanding, organization, presentation, and search - especially in terms of scalability. However, both computational costs and performance are still far from satisfactory. This increasing amount of data, along with the massive amount of grassroots users, also provides us with new opportunities to attack these challenges, as well as conventional problems in multimedia analysis and computer vision. This research direction will use all this information, as well as technologies in multimedia content analysis, machine learning, data mining and human-computer interface, to facilitate multimedia search on and across desktops, the Internet, and mobile devices. Research topics include: large-scale video/image annotation/tagging, ranking and re-ranking, new multimedia query interfaces, multimedia search result organization and presentation, as well as accessible multimedia search.
The internet has become the biggest and most powerful computing and service platform for multimedia content. This content has become the primary source for more effective online advertising. Conventional advertising systems treat multimedia content the same as general text, without considering automatically monetizing the rich content of images and videos. This research direction will leverage content analysis and understanding to enable more effective and efficient advertising on multimedia content.
Mobile Multimedia Computing
Mobile devices are becoming the most frequently used terminal to access information through the Internet and social networks. More and more multimedia content is spreading over the network of mobile devices. This area is concerned with intelligent multimedia techniques to facilitate effort-free multimedia experiences on mobile devices, including media acquisition, editing, sharing, browsing, management, search, advertising, and related user interface.
Internet-scale Multimedia Retrieval and Mining
Recent years have witnessed an explosive growth of online multimedia data. Stimulated by recent progress in scalable machine learning, feature indexing and multi-modal analysis techniques, we are particularly interested in exploring challenges and new opportunities for developing internet-scale approaches for multimedia retrieval and mining. Our topics of interest are: (i) Indexing and retrieval for large multimedia collections; (ii) web-scale social- and content-network analysis; (iii) near-duplicate detection over large data sets; (iv) automatic machine tagging, semantic annotation and object recognition on massive multimedia collections; (v) scalable and distributed machine learning and data mining methods for multimedia; (vi) construction of standard large-scale multimedia collections; (vi) interfaces for exploring, browsing and visualizing large multimedia collections; and (vii) video event and temporal analysis over diverse sources.
Our eHealth project aims to build a user-centric & data-driven personalized eHealth software and services platform for improving quality of life and efficiency of treatment. eHealth will research how personal health data can be ubiquitously collected through body sensor networks into heterogeneous devices and into the cloud; how the data can be smartly processed, visualized, analyzed, summarized, personalized, and searched; and how eHealth computing can be optimized between devices and the cloud to provide a seamless user experience and personalized services across different devices.
Media Cloud Computing
Media content distribution, searching, editing, sharing, publishing, and delivery across different devices poses a great challenge, due to multimedia’s quality of service (QoS) requirement and the heterogeneity of devices. This project envisions seamless and ubiquitous media services across different devices with good QoS. It will explore how media data and programming will be optimized among clients and the cloud, and how media content will be distributed, searched, and delivered ubiquitously across heterogeneous devices.
Human Interactive Proofs and Web Security
We are innovating technologies by exploiting the capability difference between humans and computers to: 1) Prevent bots from pretending to be humans to interact with applications designated for human users; and 2) integrate human intelligence in a computing process to solve AI (Artificial Intelligence) problems. We are also advancing technologies to fortify Web and cloud computing security.
Program Defect Detection and Performance Improvement
We are building up novel technologies to: 1) Detect and locate program defects and vulnerabilities to ensure that programs are reliable and trustworthy; and 2) diagnose performance anomalies and improve program performance.
- tao mei, yong rui, shipeng li, and qi tian, Multimedia Search Reranking: A Literature Survey, in ACM Computing Surveys, 2014.
- Wenyuan Yin, Tao Mei, and Chang Wen Chen, Automatic Generation of Social Media Snippets for Mobile Browsing, ACM Multimedia, October 2013.
- Wu Liu, Tao Mei, Yongdong Zhang, Jintao Li, and Shipeng Li, Listen, Look, and Gotcha: Instant Video Search with Mobile Phones by Layered Audio-Video Indexing, ACM Multimedia, October 2013.
- Ting Yao, Tao Mei, Chong-Wah Ngo, and Shipeng Li, Annotation for Free: Video Tagging by Mining User Search Behavior, ACM Multimedia, October 2013.
- Ting Yao, Yuan Liu, Chong-Wah Ngo, and Tao Mei, Unified Entity Search in Social Media Community, in International World-Wide Web Conference (WWW), May 2013.
- Tao Mei, Lin-Xie Tang, Jinhui Tang, and Xian-Sheng Hua, Near-Lossless Semantic Video Summarization and Its Applications to Video Analysis, in ACM Trans. Multimedia Computing Communications and Applications, ACM, 2013.
- Jitao Sang, Tao Mei, Jiaotao Sun, Changsheng Xu, and Shipeng Li, Probabilistic Sequential POIs Recommendation via Check-In Data, ACM SIGSPATIAL, 6 November 2012.
- Suman D. Roy, Tao Mei, Wenjun Zeng, and Shipeng Li, SocialTransfer: Cross-Domain Transfer Learning from Social Streams for Media Applications, ACM Conference on Multimedia, 2 November 2012.
- Bisheng Chen, Jingdong Wang, Qinghua Huang, and Tao Mei, Personalized Video Recommendation Through Tripartite Graph Propagation, in Proceedings of ACM Multimedia, November 2012.
- Tao Mei, Jiebo Luo, Houqiang Li, Shipeng Li Heng Liu, Tao Mei, and Shipeng Li, Finding Perfect Rendezvous On the Go: Accurate Mobile Visual Localization and Its Applications to Routing, in ACM Multimedia, ACM Multimedia, November 2012.