Automatic Text Pop-Up for Web Images

Interactive Visual Media Group - Microsoft Research Redmond

Krishnan Ramnath, Juliet Fiss (U. of Washington), Dahua Lin (Massachusetts Inst. of Tecnology),
Anitha Kannan, Qifa Ke, Xin-Jing Wang, Lei Zhang, Matt Uyttendaele,
Lucy Vanderwende, Simon Baker


The Bing home page provides teaser captions for an interesting image in the form of Bing tiles. The images are chosen carefully and the captions written by a person to make it interesting. The Automatic “Text Pop-Up” for Web Images application automatically generates similar text descriptions for a large fraction of the most popular images on the web. At the core of the system is an offline text-extraction process, in which the application mines the web for meaningful captions that relate to a given image. During this, the application checks sentence semantics for relevancy, diversity, and optimal structure, and performs content filtering. The results are indexed in a database. The front end of the application is integrated into the Bing Toolbar in IE. Whenever a user navigates to a webpage, the application queries the database and overlays text descriptions for the images on the webpage in the form of Bing tiles: the text pop-up. 


      Download the demo video here 





To provide interesting information to the user

To provide a passive browsing experience; user does nto need to initiate a query

To add virtual hyperlinks on top of the web, from an image to related content



Contact : Krishnan Ramnath (kramnath at or Simon Baker (sbaker at


Contact Us Terms of Use Trademarks Privacy Statement © Microsoft Corporation. All rights reserved.Microsoft