Sketch-based image search is a well-known and difficult problem, in which little progress has been made in the past decade in developing a large-scale and practical sketch-based search engine. We have revisited this problem and developed a scalable solution to sketch-based image search. Based on this solution, a system called MindFinder has been built by indexing more than two million web images to enable efficient sketch-based image retrieval, and many creative applications can be expected to advance the state of the art.
We built the Sketch2Tag system for hand-drawn sketch recognition. Due to large variations presented in hand-drawn sketches, most of existing work was limited to a particular domain or limited predefined classes. Different from existing work, Sketch2Tag is a general sketch recognition
system, towards recognizing any semantically meaningful object that a child can recognize. This system enables a user to draw a sketch on the query panel, and then provides real-time recognition results.
We built the Sketch2Cartoon system, which is an automatic cartoon making system. It enables users to sketch major curves of characters and props in their mind, and real-time search results from millions of clipart images could be selected to compose the cartoon images. The selected components are vectorized and thus could be further edited. By enabling sketch-based input, even a child who is too young to read or write can draw whatever he/she imagines and get interesting cartoon images.
As a common scenario, a tourist usually asks the following questions when he/she is planning his/her trip in an unfamiliar place: 1) Are there any travel route suggestions for a one-day or three-day trip in Beijing? 2) What is the most popular travel path within the Forbidden City? To facilitate a tourist’s trip planning, we propose to leverage existing travel clues recovered from 20 million geo-tagged photos to suggest customized travel routes for 100+ countries and territories.
Arista (lARge-scale Image Search To Annotation) is a data-driven and model-less image auto-tagging system, which annotates an image based on large-scale image search. It is based on the assumption that close similar images share similar semantics. This project was started at 2006. In 2009, Arista is able to perform online tagging based on 2 billion web images for popular images which have near-duplicates in the 2B dataset.
In this project, we focus on developing algorithms for large-scale image indexing and recognition. Our research covers low-level image features, middle level image representations, and indexing and ranking algorithms. Currently, we focus on three research directions.