This project explores the use of abstract scenes created
from clip art to:
This project explores the use of abstract scenes created from clip art to:
Study the semantic understanding of scenes from visual features.
Learn common sense knowledge.
Discover measures of semantic similarity between scenes.
Relating visual information to its linguistic semantic meaning remains an open and challenging area of research. The semantic meaning of images depends on the presence of objects, their attributes and their relations to other objects. But precisely characterizing this dependence requires extracting complex visual information from an image, which is in general a difficult and yet unsolved problem. We propose studying semantic information in abstract images created from collections of clip art. Abstract scenes allow for the direct study of how to infer high-level semantic information, since they remove the reliance on noisy low-level object, attribute and relation detectors, or the tedious hand-labeling of images. Similarly, common sense knowledge may be directly learned by observing these abstract scenes and analyzing the relations of the objects, both spatially and temporally.
Larry Zitnick (Microsoft Research)
Devi Parikh (Virginia
Devi Parikh (Virginia Tech)
Vanderwende (Microsoft Research)
Lucy Vanderwende (Microsoft Research)
Version 1.0 - Released June 2013
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013 (Oral)
IEEE International Conference on Computer Vision (ICCV), 2013
[Supplementary material] [Dataset coming soon!]