Inter-Active Learning with Queries on Instances and Features
In this talk, I will discuss a few projects aimed at “closing the loop” for interactive natural language annotation and learning. In particular, I describe two systems that combine active and semi-supervised learning by asking humans to label both instance queries (e.g., passages of text) and feature queries (e.g., advice about words or capitalization patterns, and the class labels they imply). Empirical results from real user studies show that these systems are better than state-of-the-art “passive” learning and even instance-only “active” learning, in terms of accuracy given a fixed budget of annotation time. The results are quite replicable and also provide insight into human annotator behavior, suggesting how human factors can and should be taken into account for interactive machine learning.
Speaker Details
Burr Settles is a staff research scientist and software engineer at Duolingo, an award-winning language education platform, where he has worked on numerous projects applying machine learning to language learning and testing. He also runs FAWM.ORG, an annual online collaborative songwriting experiment. Previously, he was a postdoc in machine learning at Carnegie Mellon University, and earned a PhD in computer sciences from the University of Wisconsin-Madison. His book Active Learning — an introduction to learning algorithms that are adaptive, curious, or exploratory (if you will) — was published by Morgan & Claypool in 2012. He gets around by bike and, among other things, plays guitar in the pop band Delicious Pastries.
- Series:
- Microsoft Research Talks
- Date:
- Speakers:
- Burr Settles
- Affiliation:
- Duolingo
-
-
Jeff Running
-
Series: Microsoft Research Talks
-
-
-
-
Galea: The Bridge Between Mixed Reality and Neurotechnology
Speakers:- Eva Esteban,
- Conor Russomanno
-
Current and Future Application of BCIs
Speakers:- Christoph Guger
-
Challenges in Evolving a Successful Database Product (SQL Server) to a Cloud Service (SQL Azure)
Speakers:- Hanuma Kodavalla,
- Phil Bernstein
-
Improving text prediction accuracy using neurophysiology
Speakers:- Sophia Mehdizadeh
-
-
DIABLo: a Deep Individual-Agnostic Binaural Localizer
Speakers:- Shoken Kaneko
-
-
Recent Efforts Towards Efficient And Scalable Neural Waveform Coding
Speakers:- Kai Zhen
-
-
Audio-based Toxic Language Detection
Speakers:- Midia Yousefi
-
-
From SqueezeNet to SqueezeBERT: Developing Efficient Deep Neural Networks
Speakers:- Sujeeth Bharadwaj
-
Hope Speech and Help Speech: Surfacing Positivity Amidst Hate
Speakers:- Monojit Choudhury
-
-
-
-
-
'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project
Speakers:- Peter Clark
-
Checkpointing the Un-checkpointable: the Split-Process Approach for MPI and Formal Verification
Speakers:- Gene Cooperman
-
Learning Structured Models for Safe Robot Control
Speakers:- Ashish Kapoor
-
-