ModelTracker: Redesigning Performance Analysis Tools for Machine Learning
- Saleema Amershi ,
- Max Chickering ,
- Steven Drucker ,
- Bongshin Lee ,
- Patrice Simard ,
- Jina Suh
Proceedings of the Conference on Human Factors in Computing Systems (CHI 2015) |
Published by ACM - Association for Computing Machinery
Model building in machine learning is an iterative process. The performance analysis and debugging step typically involves a disruptive cognitive switch from model building to error analysis, discouraging an informed approach to model building. We present ModelTracker, an interactive visualization that subsumes information contained in numerous traditional summary statistics and graphs while displaying example-level performance and enabling direct error examination and debugging. Usage analysis from machine learning practitioners building real models with ModelTracker over six months shows ModelTracker is used often and throughout model building. A controlled experiment focusing on ModelTracker’s debugging capabilities shows participants prefer ModelTracker over traditional tools without a loss in model performance.
© ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version can be found at http://dl.acm.org.