Learning to Relate Literal and Sentimental Descriptions of Visual Properties

Mark Yatsgar, Svitlana Volkova, Asli Celikyilmaz, Bill Dolan, and Luke Zettlemoyer

Abstract

Language can describe our visual world at many levels, including not only what is literally there but also the sentiment that it invokes. In this paper, we study visual language, both literal and sentimental, that describes the overall appearance and style of virtual characters. Sentimental properties, including labels such as “youthful” or “country western,” must be inferred from descriptions of the more literal properties, such as facial features and clothing selection. We present a new dataset, collected to describe Xbox avatars, as well as models for learning the relationships between these avatars and their literal and sentimental descriptions. In a series of experiments, we demonstrate that such learned models can be used for a range of tasks, including predicting sentimental words and using them to rank and build avatars. ogether, these results

demonstrate that sentimental language

provides a concise (though noisy) means of

specifying low-level visual properties.

Details

Publication typeProceedings
PublisherNorth American Chapter of the Association for Computational Linguistics
> Publications > Learning to Relate Literal and Sentimental Descriptions of Visual Properties