Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Multimedia Search and Mining

Multimedia Search and Mining (MSM) group focuses on pattern analysis and extraction for multimedia understanding, search, and data mining. We are working on research problems in search-based image annotation, large scale visual (image and video) indexing and search, sketch-based image search, object recognition with 3D structures, social multimedia analytics, etc.

  • Mobile Video Search
    Mobile video is quickly becoming a mass consumer phenomenon. More and more people are using their smartphones to search and browse video contents while on the move. This project is to develop an innovative instant mobile video search system through which users can discover videos by simply pointing their phones at a screen to capture a very few seconds of what they are watching.
  • 3D Object Reconstruction and Recognition
    We study the problem of 3D object reconstruction and recognition. For reconstruction, we aim at developing algorithms and systems to lower down the barrier of 3D reconstruction for common users. In this way, we can collect a world-class 3D object repository via leveraging crowdsourcing. For recognition, we aim at dealing with a large-scale task (e.g. identifying thousands of objects), and providing real-time performance.
  • Picto: A large scale visual indexing and recognition system
    In this project, we focus on developing algorithms for large-scale image indexing and recognition. Our research covers low-level image features, middle level image representations, and indexing and ranking algorithms.
  • MindFinder: Finding Images by Sketching
    Sketch-based image search is a well-known and difficult problem, in which little progress has been made in the past decade in developing a large-scale and practical sketch-based search engine. We have revisited this problem and developed a scalable solution to sketch-based image search. The MindFinder system has been built by indexing more than two million web images to enable efficient sketch-based image retrieval, and many creative applications can be expected to advance the state of the art.
  • Mobile Multimedia Computing
    Mobile devices are becoming the most frequently used terminal to access the information through the Internet and social networks. More and more multimedia content is spreading over the network of mobile devices. This area is concerned with intelligent multimedia techniques to facilitate effort-free multimedia experiences on mobile devices, including media acquisition, editing, sharing, browsing, management, search, advertising, and related user interface.
  • Multimedia Advertising
    The ever increasing multimedia content on the Internet has become the primary source for more effective online advertising. Conventional advertising systems treat multimedia content as the same as general text, without considering automatically monetizing the rich content of the images and videos. This research direction will leverage content analysis and understanding to enable more effective and efficient advertising on multimedia content, whether on the Internet and mobile devices.
  • Video Collage
    Video Collage is a kind of synthesized image that enable users to quickly browse the video content. Given a video, Video Collage is able to select the most representative images from the video, extract salient regions of interest from these images, and seamlessly arrange ROI on a given canvas. Video Collage can be used for Windows Vista Explorer, Live Search Video, as well as MSN Soapbox.