Saleema Amershi

Contact

Saleema Amershi, Ph.D.
Researcher, Microsoft Research
samershi[at]microsoft[dot]com

One Microsoft Way
Redmond, WA 98052
USA

[CV-Sept.2016] [Research Statement]

About Me

I'm a Researcher in the Machine Teaching Group at Microsoft Research. "Machine teaching" is machine learning with a focus on increasing user, or "teacher", productivity and effectiveness.

My research lies at the intersection of human-computer interaction and machine learning. In particular, I create tools to support both practitioner and end-user interaction with machine learning systems. Examples include general purpose tools to support data scientists and machine learning experts building reusable predictive models for production use and application specific tools to support the average person interacting with machine learning in their everyday lives (e.g., automation technologies and recommender systems). Throughout my work, I distill guiding principles applicable in a broader context to help provide a foundation for future human-driven machine learning systems.

In 2012 I received my PhD in Computer Science from the University of Washington's Computer Science & Engineering department, where I was advised by James Fogarty. My dissertation entitled "Designing for Effective End-User Interaction with Machine Learning" won the University of Washington's 2013 Distinguished Dissertation Award. During my time in grad school, I also had the opportunity to work with some amazing people at Google Research, Microsoft Research (VIBE, ASI and TEM) and IBM Research.

Prior to UW, I completed a MSc in Computer Science at the University of British Columbia where I worked at The Laboratory for Computational Intelligence. I also have a BSc in Computer Science and Mathematics from the University of British Columbia.

Refereed Articles

[P.27] Huang, T. K., Li, L., Vartanian, A., Amershi, S., and Zhu, J. (2016) Active Learning with Oracle Epiphany. To Appear in the Proceedings of Neural Information Processing Systems (NIPS 2016).
[P.26] Ren, D., Amershi, S., Lee, B., Suh, J., and Williams, J. D. (2016) Squares: Supporting Interactive Performance Analysis for Multiclass Classifiers. In IEEE Transactions on Visualization and Computer Graphics (TVCG), Visual Analytics Science and Technology (VAST 2016). [pdf] [mp4]
[P.25] Toutanova, K., Brockett, C., Tran, K. M., and Amershi, S. (2016) A Dataset and Evaluation Metrics for Abstractive Sentence and Paragraph Compression. To Appear in the Proceedings of the International Conference on Empirical Methods in Natural Language Processing (EMNLP 2016). [pdf]
[P.24] Suh, J., Zhu, J., and Amershi, S. (2016) The Label Complexity of Mixed-Initiative Classifier Training. In Proceedings of the International Conference on Machine Learning (ICML 2016). [pdf]
[P.23] Brooks, M., Amershi, S., Lee, B., Drucker, S., Kapoor, A., and Simard, P. (2015) FeatureInsight: Visual Support for Error-Driven Feature Ideation in Text Classification. In Proceedings of the IEEE Conference on Visual Analytics Science and Technology (VAST 2015), pp. 105-112. [pdf]
[P.22] Amershi, S., Chickering, M., Drucker, S., Lee, B., Simard, P., and Suh, J. (2015) ModelTracker: Redesigning Performance Analysis Tools for Machine Learning. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2015), 337-346. [pdf] [short (mp4)] [long (mp4)] [long (avi)]
[P.21] Amershi, S., Cakmak, M., Knox, W.B., and Kulesza, T. (Winter 2014) Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35 (4): pp. 105-120. [pdf]
[P.20] Kulesza, T., Amershi, S., Caruana, R., Fisher, D., and Charles, D. (2014) Structured Labeling to Facilitate Concept Evolution in Machine Learning. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2014), pp. 3075-3084. [pdf]
Best Paper Award.
[P.19] Amershi, S., Mahmud, J., Nichols, J., Lau, T., and Ruiz, G.A. (2013) LiveAction: Automating Web Task Model Generation. ACM Transactions on Interactive and Intelligent Systems (TiiS) 3 (3): pp. 14:1-14:23. [pdf]
[P.18] Amershi, S., Fogarty, J. and Weld, D.S. (2012) ReGroup: Interactive Machine Learning for On-Demand Group Creation in Social Networks. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2012), pp. 21-30. [pdf]
[P.17] Amershi, S., Lee, B., Kapoor, A., Mahajan, R. and Christian, B. (2011) Human-Guided Machine Learning for Fast and Accurate Network Alarm Triage. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2011), Best Papers from Sister Conferences Track, pp. 2564-2569. [pdf]
Invited Paper.
[P.16] Amershi, S., Fogarty, J., Kapoor, A., and Tan, D.(2011) Effective End-User Interaction with Machine Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2011), Nectar Track, pp. 1529-1532. [pdf]
[P.15] Amershi, S., Lee, B., Kapoor, A., Mahajan, R. and Christian, B. (2011) CueT: Human-Guided Fast and Accurate Network Alarm Triage. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2011), pp. 157-166. [pdf]
Best Paper Honorable Mention, Invited to IJCAI 2011 Best Papers Track.
[P.14] Chen, J., Amershi, S., Dhananjay, A., and Lakshmi, S.(2010) Comparing Web Interaction Models in Developing Regions. In Proceedings of the ACM Symposium on Computing for Development (DEV 2010).
[P.13] Amershi, S., Fogarty, J., Kapoor, A., and Tan, D.(2010) Examining Multiple Potential Models in End-User Interactive Concept Learning. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2010), pp. 1357-1360. [pdf]
[P.12] Amershi, S. and Conati, C. (2010) Automatic Recognition of Learner Types in Exploratory Learning Environments. Handbook of Educational Data Minint, Chapter 15. Data Mining and Knowledge Discovery Series (eds. R. Cohen and V. Kumar), Chapman & Hall/CRC Press.
[P.11] Amershi, S., Morris, M. R., Moraveji, N., Balakrishnan, R., and Toyama, K. (2010) Multiple Mouse Text Entry for Single-Display Groupware. In Proceeding of the ACM Conference on Computer Supported Cooperative Work (CSCW 2010), pp. 169-178. [pdf] [mov]
Best Paper Honorable Mention.
[P.10] Amershi, S., Fogarty, J., Kapoor, A. and Tan, D. (2009) Overview-Based Examples Selection in Mixed-Initiative Interactive Concept Learning. In Proceeding of the ACM Symposium on User Interface Software and Technology (UIST 2009), pp. 247-256.[pdf]
[P.9] Amershi, S. and Conati, C. (2009) Combining Unsupervised and Supervised Machine Learning to Build User Models for Exploratory Learning Environments. In The Journal of Educational Data Mining 1, 1 (JEDM 2009). [pdf]
[P.8] Hoffman, R., Amershi, S., Patel, K., Wu, F., Fogarty, J., Weld, D.S.(2009) Amplifying Community Content Creation with Mixed-Initiative Information Extraction. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2009), pp. 1849-1858. [pdf]
Best Paper Honorable Mention.
[P.7] Weld, D.S., Wu, F., Adar, E., Amershi, S., Fogarty, J., Hoffmann, R., Patel, K., and Skinner, M.(2008) Intelligence in Wikipedia. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 08) Senior Papers Track, pp. 1609-1614. [pdf]
[P.6] Amershi, S. and Morris, M.R. (2008) CoSearch: A System for Co-located Collaborative Web Search. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2008), pp. 1647-1656. [pdf] [mov]
[P.5] Amershi, S., Carenini, G., Conati, C., Mackworth, A., and Poole, D. (2008) Pedagogy and Usability in Interactive Algorithm Visualizations - Designing and Evaluating CIspace. Interacting with Computers - The Interdisciplinary Journal of Human-Computer Interaction 20 (1): pp. 64-96. [pdf]
[P.4] Conati, C., Merten, C., Amershi, S., and Muldner, K. (2007) Using Eye-tracking Data for High-Level User Modeling in Adaptive Interfaces. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 07) Nectar Track, pp. 1614-1617. [pdf]
[P.3] Amershi, S. and Conati, C. (2007) Unsupervised and Supervised Machine Learning in User Modeling for Intelligent Learning Environments. In Proceedings of the ACM/SIGCHI Conference on Intelligent User Interfaces (IUI 2007), pp. 72-81.[pdf]
[P.2] Amershi, S. and Conati, C. (2006) Automatic Recognition of Learner Groups in Exploratory Learning Environments. In Proceedings of Intelligent Tutoring Systems (ITS 2006), pp. 463-472.[pdf]
[P.1] Amershi, S., Arksey, N., Carenini, G., Conati, C., Mackworth, A., Maclaren, H., and Poole, D. (2005) Designing CIspace: Pedagogy and Usability in a Learning Environment for AI. In Proceedings of the ACM/SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2005), pp. 178-182.[pdf]

Technical Reports, Refereed Workshop Papers, Demos & Posters

[W.9] Simard, P., Chickering, M., Lakshmiratan, A., Garcia Jurado Suarez, C., Amershi, S., Verwey, J., and Suh, J. (2014) ICE: Interactive Classification and Entity Extraction. Neural Information Processing Systems Demonstrations (NIPS 2014).
[W.8] Simard, P., Chickering, M., Lakshmiratan, A., Charles, D., Bottou, L., Garcia Jurado Suarez, C., Grangier, D., Amershi, S., Verwey, J., and Suh, J. (2014) ICE: Enabling Non-Experts to Build Models Interactively for Large-Scale Lopsided Problems. arXiv:1409.4814 [cs.AI], Microsoft Research 2014.
[W.7] Amershi, S., Fogarty, J., Kapoor, A., and Tan, D. (2009) Designing for End-User Interactive Concept Learning in CueFlik. Workshop on Analysis and Design of Algorithms for Interactive Machine Learning at NIPS 2009 (ADA-IML at NIPS 2009).
[W.6] Amershi, S. and Morris, M.R. (2009) Co-located Collaborative Web Search: Understanding Status Quo Practices. The ACM Conference on Human Factors in Computing Systems - Extended Abstracts (CHI 2009).
[W.5] Hoffman, R., Amershi, S., Patel, K., Wu, F., Fogarty, J., and Weld, D.S. (2008) Amplifying Community Content Creation with Mixed-Initiative Information Extractions. The ACM Symposium on User Interface and Software Technology (UIST 2008).
[W.4] Amershi, S. and Morris, M.R. (2008) CoSearch: Leveraging Multiple Devices to Enhance Collaboration in Resource-Constrained Environments. The ACM Conference on Human Factors in Computing Systems Workshop on HCI for Community and International Development (CHI 2008).[pdf]
[W.3] Morris, M.R. and Amershi, S. (2008) Shared Sensemaking: Enhancing the Value of Collaborative Web Search Tools. The ACM Conference on Human Factors in Computing Systems Workshop on Sensemaking (CHI 2008). [pdf]
[W.2] Amershi, S. and Morris, M.R. (2008) CoSearch: A System for Co-located Collaborative Web Search. Microsoft Research TechFest (TechFest 2008).
[W.1] Amershi, S., Conati, C. and Maclaren, H. (2006) Using Feature Selection and Unsupervised Clustering to Identify Affective Expressions in Educational Games. In Proceedings of The Intelligent Tutoring Systems Workshop on Motivational and Affective Issues in ITS (ITS 2006), pp. 21-28.[pdf]

Theses

[T.2] Amershi, S. (2012) Designing for Effective End-User Interaction with Machine Learning. Doctoral Dissertation, University of Washington (UW).[pdf]
2013 University of Washington Distinguished Dissertation Award.
[T.1] Amershi, S. (2007) Combining Unsupervised and Supervised Machine Learning to Build User Models for Intelligent Learning Environments. Masters Thesis, University of British Columbia (UBC).[pdf]