Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Representative work



Internet Multimedia Advertising



Survey on Multimedia Ads 


Contextual Internet Multimedia Advertising


Tao Mei, Xian-Sheng Hua

Proceedings of the IEEE, 98(80): 1416-1433, Aug. 2010.


The advent of media-sharing sites has led to the unprecedented Internet delivery of community-contributed media like images and videos. Those visual contents have become the primary sources for online advertising. Conventional advertising treats multimedia advertising as general text advertising, without considering the potential advantages which could be brought by media contents. We summarize the trend of Internet multimedia advertising and conduct a broad survey on the methodologies for advertising which are driven by the rich contents of images and videos.





Contextual In-Image Advertising

Tao Mei, Xian-Sheng Hua, Shipeng Li
ACM Multimedia, pp. 439-448, 2008.  


The community-contributed media contents over the Internet have become one of the primary sources for online advertising. In this work, we propose an innovative contextual advertising system driven by images, which automatically associates relevant ads with an image rather than the entire text in a Web page and seamlessly inserts the ads in the nonintrusive areas within each individual image.






VideoSense - Towards Effective Online Video Advertising

Tao Mei, Xian-Sheng Hua, Linjun Yang, Shipeng Li
ACM Multimedia, pp. 1075-1084, 2007.


Online video advertising is becoming increasingly pervasive. We present a novel advertising system for online video service called VideoSense, which automatically associates the most relevant video ads with online videos and seamlessly inserts the ads at the most appropriate positions within each individual video. Unlike most current video-oriented sites that only display a video ad at the beginning or the end of a video, VideoSense aims to embed more contextually relevant ads at less intrusive positions within the video stream.





When Multimedia Advertising Meets the New Internet Era

Xian-Sheng Hua, Tao Mei, Shipeng Li
IEEE Workshop on Multimedia Signal Processing, 2008.
Best Poster Paper Award from MMSP 2008


The advent of media-sharing sites, especially along with the so-called Web 2.0 wave, has led to the unprecedented Internet delivery of community-contributed media contents, which have become the primary sources for online advertising. However, conventional ad-networks treat image and video advertising as general text advertising by displaying the ads either relevant to the queries or the Web page content, without considering automatically monetizing the rich contents of individual images and videos. In this paper, we summarize the trends of online advertising and propose an innovative advertising model driven by the compelling contents of images and videos.



Mobile Media Computing (Search and Location-based Services



JIGSAW - Interactive Mobile Visual Search with Multimodal Queries



JIGSAW: Interactive Mobile Visual Search with Multimodal Queries


Yang Wang *, Tao Mei, Jingdong Wang, Houqiang Li, Shipeng Li

ACM Multimedia, 2011. 


The traditional text-based visual search has not been sufficiently improved over the years to accommodate the new emerging demand of mobile users. While on the go, searching on one’s phone is becoming pervasive. This paper presents an innovative application for mobile phone users to facilitate their visual search experience.


Contextual and Personalized Recommendation for Mobile Users



When Recommendation Meets Mobile: Contextual and Personalized Recommendation On The Go


Jinfeng Zhuang *, Tao Mei, Steven C. H. Hoi, et al.

ACM Ubicomp, 2011.


Mobile devices are becoming ubiquitous. People use their phones as a personal concierge discovering and making decisions anywhere and anytime. Understanding user intent on the go therefore becomes important for task completion on the phone. This paper presents an approach to context-aware and personalized entity recommendation for mobile users.



Multimedia Search (Reranking and Annotation) 



Visual Query Suggestion 



Visual Query Suggestion

Zheng-Jun Zha *, Linjun Yang, Tao Mei, et al.
ACM Multimedia, pp. 15-24, 2009.
Best Paper Award from SIGMM 2009 


Query suggestion is an effective approach to improve the usability of image search. Most existing search engines are able to automatically suggest a list of textual query terms based on users’ current query input, which can be called Textual Query Suggestion. This paper proposes a new query suggestion scheme named Visual Query Suggestion which is dedicated to image search. It provides a more effective query interface to formulate an intent-specific query by joint text and image suggestions.






CrowdReranking: Exploring Multiple Search Engines for Visual Search Reranking

Yuan Liu *, Tao Mei, Xian-Sheng Hua
ACM SIGIR, pp. 500-507, Boston, USA, 2009.


Most existing approaches to visual search reranking predominantly focus on mining information within the initial search results. However, the initial ranked list cannot provide enough cues for reranking by itself due to the typically unsatisfying visual search performance. This paper presents a new method for visual search reranking called CrowdReranking, which is characterized by mining relevant visual patterns from image search results of multiple search engines which are available on the Internet.



Video Annotation via Search and Mining 



Video Annotation through Search and Graph Reinforcement Mining


Emily Moxley *, Tao Mei, B. S. Manjunath

IEEE Trans. on Multimedia, 12(3): 184-193, April 2010. 


Unlimited vocabulary annotation of multimedia documents remains elusive despite progress solving the problem in the case of a small, fixed lexicon. Taking advantage of the repetitive nature of modern information and online media databases with independent annotation instances, we present an approach to automatically annotate multimedia documents that uses mining techniques to discover new annotations from similar documents and to filter existing incorrect annotations.



Image Annotation



Coherent Image Annotation by Learning Semantic Distance

Tao Mei, Yong Wang *, Xian-Sheng Hua, et al.
IEEE CVPR, 2008.


We propose a novel approach to image annotation which simultaneously learns a semantic distance by capturing the prior annotation knowledge and propagates the annotation of an image as a whole entity.



Video Annotation



Multi-Layer Multi-Instance Learning for Video Concept Detection

Zhiwei Gu *, Tao Mei, Xian-Sheng Hua, et al.
IEEE Trans. on Multimedia, 10(8): 1605-1616, 2008. 


This paper presents a novel learning-based method, called “multi-layer multi-instance (MLMI) learning,” for video concept detection. Video is essentially a kind of media with ML structure. For example, a video can be represented by a hierarchical structure including, from large to small, shot, frame, and region, where each pair of contiguous layers fits the typical MI setting. We systematically study both ML structure and MI relations embedded in video content by formulating video concept detection as a MLMI learning problem.



Multimedia Presentation and Visualization





Video Collage: Presenting a Video Sequence Using a Single Image

Tao Mei, Bo Yang *, Shi-Qiang Yang, Xian-Sheng Hua
The Visual Computer, 25(1): 39-51, 2009.
Best Demo Award from SIGMM 2007 


The explosive growth of video data demands the video presentation technique which supports fast browsing of video content. we present an automatic procedure for constructing a compact synthesized collage from a video sequence. The synthesized image, called “Video Collage”, is a kind of static video summary.



Video Recommendation



Video Recommendation 



Contextual Video Recommendation by Multimodal Relevance and User Feedback


Tao Mei, Bo Yang *, Xian-Sheng Hua, Shipeng Li

ACM Trans. on Information Systems, Vol. 29, No. 2, April 2011. 


With Internet delivery of video content surging to an unprecedented level, video recommendation, which suggests relevant videos to targeted users according to their historical and current viewings or preferences, has become one of most pervasive online video services. This paper presents a novel contextual video recommendation system, called VideoReach, based on multimodal content relevance and user feedback.