With Show Similar Image, you can select an image from a results set, originally based on a text query, and then request, with a single click, that the resulting images are re-ranked according to their visual similarity to the selected image. By comparing the visual features, such as face, directionality, energy, edge, color, spatial distribution, Show Similar Image categorizes the query image that you selected so that visual similarity can be measured by using different feature combinations.
Show Similar Image enables you to search visually with images.
How Show Similar Image Works
Existing major search engines use only text to input a query, meaning that results from these searches can often be unsatisfactory, due to ambiguity (words can have multiple meanings, for example, the search term “apple” may return a picture of a fruit or a company logo) and noise (images irrelevant to the query, such as a picture called apple.jpg not being of an apple or an image crawled from a web page containing the text “apple”). Images are ranked largely by how well the query matches the Web page containing the image; the image content itself is not used. In most cases, it’s almost completely impossible to adequately describe the image being looked for, and the phrase “a picture is worth a thousand words” becomes an understatement. But what if there were a way of fine tuning the results set, maybe by re-ranking the returned images according to a different, non-text based query?
Show Similar Image allows a user to do just that; select an image from a results set, originally based on a text query, and then request, with a single click, that the resulting images are re-ranked according to their visual similarity to the selected image.
The challenges involved with performing this type of query are twofold:
- How is effective visual similarity defined?
- How are visual features efficiently extracted for use on a web-scale image search engine?
By “looking” at the visual features (such as face, directionality, energy, edge, color, and spatial distribution), the query image, selected by the user, can be categorized by the system into several categories (such as general objects, objects with a simple background, scenery images, portraits, and people), so that visual similarity can be adaptively measured using different feature combinations. The key here is to attempt to understand what the user is looking for, or the intent of the user. For example, if the system can understand that the user intends to search for a face, which would be categorized as a portrait, then a facial recognition algorithm would be more effective than a general texture classification algorithm.
The benefits of this type of search to the user are huge. They can simply and quickly further refine and filter their original search, using an image as a query instead of text. This gives them a more flexible and accurate image search, providing a results set that is more likely to show the images that they are actually looking for.
Transferred Technology
The Show Similarity technology was created by Jian Sun, Fang Wen, and their team members at Microsoft Research Asia. The technology was transferred to the Microsoft Live Search product on 1 December 2008 through a successful partnership with developers on the Live Search team: Minghui Xia, Xiaodong Fan, Richard Qian, and Gang Hua. It is the first image search engine of this type in the world.
