Joint design of data analysis algorithms and user interface for video applications

Nebojsa Jojic, Microsoft Research

Sumit Basu, Microsoft Research

Nemanja Petrovic, University of Illinois

Brendan Frey, University of Toronto

Thomas Huang, University of Illinois


The graphical modeling paradigm provides a way of representing data through hidden causes of variability which can be estimated from the data in an unsupervised manner. Recently, a lot of research has been dedicated to finding efficient inference and learning engines for graphical models in general, as well as to finding various ways of using graphical models to perform recognition, classification, segmentation, and tracking tasks in video applications. Little research, however, has focused on another advantage of a graphical model - by discovering the structural elements in the data, it renders the data much easier to browse, manipulate, or interact with. In this paper, we present several ideas on how the user interface and the data analysis tools can be designed jointly starting from an appropriate data representation scheme and a generative model based on it. We base our approach on three basic principles:

Compatibility of the graphical model’s structure with our own perception of the world

Simplicity in representation, leading to more efficient inference

Providing intuitive interactivity on the level of hidden causes of variability


Longer abstract as PDF file


Return to Machine Learning and User Interface workshop page.