Introduction and Goals of the Workshop
traditional role of the human operator in machine learning
problems is that of a batch labeler, whose work is done
before the learning even begins. However, there is an
important class of problems in which the human is
interacting directly with the learning algorithm as it
learns. Canonical problem scenarios which fall into this
space include active learning, interactive clustering, query
by selection, learning to rank, and others. Such problems
are characterized by three main factors:
algorithm requires input from the human during training,
in the form of labels, feedback, parameter guidance,
user cannot express an explicit loss function to
optimize, either because it is impractical to label a
large training set or because they can only express
stopping criterion is performance that's "good enough"
in the eyes of the user.
of this workshop is to focus on the machine learning
techniques that apply to these problems, both in terms of
surveying the major paradigms and sharing information about
new work in this area. Through a combination of invited
talks, discussions, and posters, we hope to gain a better
understanding of the available algorithms and best practices
for this space, as well as their inherent limitations.
Call for Abstracts
We invite all researchers interested in
presenting at the workshop to submit a one-page abstract of
their work. The presentation format will be a
spotlight summary talk along with a poster session later in
the afternoon. We encourage presentations on
new, developing ideas, as well as previously published work
the authors would like to discuss in this forum. Feel
free to email us if you are concerned about whether your
work is appropriate for the workshop. Note that
there will not be formal proceedings for the workshop, so
authors need not be concerned about publishing work they
present here at a later venue.
[The deadline for
submissions (Oct. 31) has now passed]
Speakers and Attendees
We will have four invited talks (titles
Jerry Zhu (University of Wisconsin, Madison)
Carlos Guestrin (CMU)
Rich Caruana (Microsoft Research)
Pushmeet Kohli (MSR Cambridge, Cambridge University)
7:30-8:00 - Introduction;
Developing a Syllabus for Interactive Machine Learning*
8:00-8:30 - Invited Talk: Rich Caruana,The Need for User
Interaction and Feedback in Clustering
Invited Talk: Carlos Guestrin, Using Personalization to
Tame Information Overload
9:00-9:30 - Coffee Break
9:30-10:30 - Poster
3:30-4:00 - Invited Talk: Pushmeet Kohli, Learning and
Evaluating Interactive Segmentation Systems
4:00-5:00 - Poster Session
(1.5 hrs, including coffee break)
5:00-5:30 - Coffee
5:30-6:00 - Invited Talk: Jerry Zhu,
Human Machine Co-Learning
6:00-6:30 - Open Problems, Challenges, Opportunities
*Note that the
syllabus-in-progress will be left on the board so that
participants may continue to contribute to it during the
poster session, final discussion, etc.
Saleema Amershi, James Fogarty, Ashish
Kapoor, and Desney Tan. Designing for End-User
Interactive Concept Learning in CueFlik.
Pranjal Awasthi, Maria-Florina Balcan,
and Avrim Blum. Clustering with Interactive
Vineeth Balasubramanian, Shayok
Chakraborty, and Sethuraman Panchanathan.
Online Active Learning Using Conformal Predictions.
Scott Blunsden and Cristina Versino.
Interactively Reviewing Large Image Sets.
Krzysztof Gajos. Beyond
Feature Relevance: Incorporating Rich User Feedback Into
Interactive Machine Learning Applications.
Ajay J. Joshi, Fatih Porikli, and
Nikolaos Papanikolopoulos. Multi-Class Active
Learning with Binary User Feedback.
Goo Jun, Alexander Liu, and Joydeep
Ghosh. Interactive Learning on Multiple Binary
Karrie Karahalios and Tony Bergstrom.
Leveraging People and Computers for NLP.
W. Bradley Knox and Peter Stone.
Interactively Shaping Agents via Human Feedback:
Joshua M. Lewis. Software,
Psychophysics, and Selection: Towards Anthropocentric
Lihong Li, Wei Chu, John Langford, and
Robert E. Schapire. A Contextual-Bandit Approach to
Personalized News Article Recommendation.
Xu Miao, Rajesh P. N. Rao, and
Shin'ichi Satoh. Interactive Structural Learning for
Image and Video Analysis.
Stephane Ross and J. Andrew Bagnell.
Achieving Small Regret Using an Interactive Learning
Approach to Imitation Learning.
Kevin Small and Dan Roth.
Interactive Feature Space Construction.
Yisong Yue. Online Gradient
Descent Using Interactive User Feedback.
Alice X. Zheng, John Dunagan, and
Ashish Kapoor. Actively Cutting Graphs: Think
Globally, Cut Locally.
Basu is a researcher in the Knowledge Tools group of
Microsoft Research, where he investigates interactive
machine learning problems in a variety of applications
domains. His focus is on techniques which help the user
achieve their goals in scenarios where they may not be able
to express an explicit cost function, but can show examples
or express preferences. He is on the Senior PC of the
Intelligent User Interfaces conference, and also works in
the area of systems and machine learning: he was
co-organizer of the MLSys Workshop at NIPS 2007, as well as
co-chair of the SysML Workshop at OSDI 2008.
Kapoor is a Researcher in the Adaptive Systems and
Interaction Group at Microsoft Research where his research
interests are centered around interactive machine learning.
His recent work focused on systems that often involve humans
in the loop and have the ability to adapt and learn over
long periods of time. Ashish also co-organized related
workshops at AAAI 2009 spring symposium on Human Behavior
Modeling and IJCAI 2009 workshop on Intelligence and
Interaction. These workshops focused mostly on the
interaction facet of IML and the proposed NIPS workshop
would complement the research efforts in the area with
discussion on formal machine learning methods.
©2010 Microsoft Corporation. All rights reserved.