Empirical Studies of Software Engineering

Several groups in MSR are studying software engineering process and products using empirical methods. The methods range from statistical analysis of software repositories to observations of programmers at work.

Publications

2015

Abram Hindle, Christian Bird, Thomas Zimmermann, and Nachiappan Nagappan, Do Topics Make Sense to Managers and Developers?, in Empirical Software Engineering, Springer, December 2015

Emerson Murphy-Hill, Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan, The Design Space of Bug Fixes and How Developers Navigate It, in IEEE Transactions on Software Engineering, IEEE – Institute of Electrical and Electronics Engineers, December 2015

2014

Kim Herzig, Using Pre-Release Test Failures to Build Early Post-Release Defect Prediction Models, in Proceedings of the 25th International Symposium on Software Reliability Engineering, IEEE – Institute of Electrical and Electronics Engineers, November 2014

André N. Meyer, Thomas Fritz, Gail C. Murphy, and Thomas Zimmermann, Software Developers’ Perceptions of Productivity, in FSE '14: Proceedings of the 22nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering, ACM – Association for Computing Machinery, November 2014

Kim Herzig and Nachiappan Nagappan, The Impact of Test Ownership and Team Structure on the Reliability and Effectiveness of Quality Test Runs, in Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurement, ACM, September 2014

Ashish Gupta, Thomas Zimmermann, Christian Bird, Nachiappan Nagappan, Thirumalesh Bhat, and Syed Emran, Mining Energy Traces to Aid in Software Development: An Empirical Case Study, in ESEM '14: Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurements, ACM – Association for Computing Machinery, September 2014

Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan, An Empirical Study of Refactoring Challenges and Benefits at Microsoft, in IEEE Transactions on Software Engineering, vol. 40, no. 7, IEEE – Institute of Electrical and Electronics Engineers, July 2014

Emerson Murphy-Hill, Thomas Zimmermann, and Nachiappan Nagappan, Cowboys, Ankle Sprains, and Keepers of Quality: How Is Video Game Development Different from Software Development?, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014

Christian Bird, Venkatesh-Prasad Ranganath, Thomas Zimmermann, Nachiappan Nagappan, and Andreas Zeller, Extrinsic Influence Factors in Software Reliability: A Study of 200,000 Windows Machines, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014

Andrew Begel and Thomas Zimmermann, Analyze This! 145 Questions for Data Scientists in Software Engineering, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014

Shaun Phillips, Thomas Zimmermann, and Christian Bird, Understanding and Improving Software Build Teams, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014

Emerson Murphy-Hill, Thomas Zimmermann, and Nachiappan Nagappan, Motion Sickness Susceptibility in Software Developers, no. MSR-TR-2014-24, 4 March 2014

Brendan Murphy, JAcek Czerwonka, and Laurie Williams, Branching Taxonomy, no. MSR-TR-2014-23, 26 February 2014

2013

Kim Herzig, Sascha Just, Andreas Rau, and Andreas Zeller, Predicting Defects Using Change Genealogies, in Proceedings of the 2013 IEEE 24nd International Symposium on Software Reliability Engineering, IEEE, November 2013

Andrew Begel and Thomas Zimmermann, Analyze This! 145 Questions for Data Scientists in Software Engineering, no. MSR-TR-2013-111, 28 October 2013

Brendan Murphy, Christian Bird, Thomas Zimmermann, Laurie Williams, Nachiappan Nagappan, and Andrew Begel, Have Agile Techniques been the Silver Bullet for Software Development at Microsoft?, ACM ESEM , 11 October 2013

Andrew Begel and Thomas Zimmermann, Appendix to Analyze This! 145 Questions for Data Scientists in Software Engineering, no. MSR-TR-2013-84, 14 September 2013

Meiyappan Nagappan, Thomas Zimmermann, and Christian Bird, Diversity in Software Engineering Research, in Proceedings of the 9th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2013), ACM, August 2013

Ekrem Kocaguneli, Thomas Zimmermann, Christian Bird, Nachiappan Nagappan, and Tim Menzies, Distributed Development Considered Harmful?, in Proceedings of the International Conference on Software Engineering (Software Engineering in Practice track), IEEE, May 2013

Emerson Murphy-Hill, Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan, The Design of Bug Fixes, in Proceedings of the 35th International Conference on Software Engineering (ICSE 2013), IEEE, May 2013

Emerson Murphy-Hill, Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan, Appendix to The Design of Bug Fixes, no. MSR-TR-2013-22, 15 April 2013

Kim Herzig and Andreas Zeller, The Impact of Tangled Code Changes, in Proceedings of the 10th Working Conference on Mining Software Repositories, IEEE, 2013

Kim Herzig, Sascha Just, and Andreas Zeller, It's not a Bug, It's a Feature: How Misclassification Impacts Bug Prediction, in Proceedings of the 2013 International Conference on Software Engineering, IEEE, 2013

2012

Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan, A Field Study of Refactoring Challenges and Benefits, in Proceedings of the 20th International Symposium on Foundations of Software Engineering (FSE 2012), Association for Computing Machinery, Inc., November 2012

Christian Bird and Thomas Zimmermann, Assessing the Value of Branches with What-if Analysis, in Proceedings of the 20th International Symposium on Foundations of Software Engineering (FSE 2012), Association for Computing Machinery, Inc., November 2012

Abram Hindle, Christian Bird, Thomas Zimmermann, and Nachiappan Nagappan, Relating Requirements to Implementation via Topic Analysis, in Proceedings of the 2012 International Conference on Software Maintenance (ICSM 2012), IEEE, 25 September 2012

Meiyappan Nagappan, Thomas Zimmermann, and Christian Bird, Representativeness in Software Engineering Research, no. MSR-TR-2012-93, 5 September 2012

Raymond P.L. Buse and Thomas Zimmermann, Information Needs for Software Development Analytics, in Proceedings of the 34th International Conference on Software Engineering (ICSE 2012 SEIP Track), IEEE, June 2012

Thomas Zimmermann, Nachiappan Nagappan, Philip J. Guo, and Brendan Murphy, Characterizing and Predicting Which Bugs Get Reopened, in Proceedings of the 34th International Conference on Software Engineering (ICSE 2012 SEIP Track), IEEE, June 2012

Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan, Appendix to A Field Study of Refactoring Rationale, Benefits, and Challenges at Microsoft, no. MSR-TR-2012-4, 11 January 2012

2011

Ashish Gupta, Thomas Zimmermann, Christian Bird, Nachiappan Nagappan, Thirumalesh Bhat, and Syed Emran, Detecting Energy Patterns in Software Development, no. MSR-TR-2011-106, 16 November 2011

Tim Menzies, Andrew Butcher, Andrian Marcus, and Thomas Zimmermann, Local vs Global Models for Effort Estimation and Defect Prediction, in Proceedings of the 26st IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), IEEE, November 2011

Alexander Tarvo, Thomas Zimmermann, and Jacek Czerwonka, An Integration Resolution Algorithm for Mining Multiple Branches in Version Control Systems (Industry Track), in Proceedings of the 27th IEEE International Conference on Software Maintenance (ICSM 2011), IEEE, September 2011

Ayse Tosun Misirli, Brendan Murphy, Thomas Zimmermann, and Ayse Basar, An Explanatory Analysis on Eclipse Beta-Release Bugs Through In-Process Metrics, in Proceedings of the 8th International Workshop on Software Quality (WoSQ 2011), ACM, September 2011

Sunghun Kim, Thomas Zimmermann, and Nachiappan Nagappan, Crash Graphs: An Aggregated View of Multiple Crashes to Improve Crash Triage (Practical Experience Report), in Proceedings of the 2011 IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2011), IEEE, July 2011

Paul Luo Li, Ryan Kivett, Zhiyuan Zhan, Sung-eok Jeon, Nachiappan Nagappan, Brendan Murphy, and Andrew J. Ko, Characterizing the Difference Between Pre- and Post- Release Versions of Software, International Conference on Software Engineering, 25 May 2011

Andrew Begel and Libby Hemphill, Not Seen and Not Heard, no. MSR-TR-2011-136, 25 April 2011

Christian Bird, Brendan Murphy, Nachi Nagappan, and Thomas Zimmermann, Empirical Software Engineering at Microsoft Research, in Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), Association for Computing Machinery, Inc., 19 March 2011

Philip J. Guo, Thomas Zimmermann, Nachiappan Nagappan, and Brendan Murphy, "Not My Bug!" and Other Reasons for Software Bug Report Reassignments, in Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), Association for Computing Machinery, Inc., March 2011

Thomas Zimmermann, Nachiappan Nagappan, Kim Herzig, Rahul Premraj, and Laurie Williams, An Empirical Study of the Factors Relating Field Failures and Dependencies, in Proceedings of the 4th International Conference on Software Testing, Verification and Validation (ICST), IEEE, March 2011

Raymond P.L. Buse and Thomas Zimmermann, Information Needs for Software Development Analytics, no. MSR-TR-2011-8, 30 January 2011

2010

Philip J. Guo, Thomas Zimmermann, Nachiappan Nagappan, and Brendan Murphy, Characterizing and Predicting Which Bugs Get Fixed: An Empirical Study of Microsoft Windows, in Proceedings of the 32th International Conference on Software Engineering (ICSE), Association for Computing Machinery, Inc., 2 May 2010

2009

Laurie Williams, Gunnar Kudrjavets, and Nachiappan Nagappan, On the Effectiveness of Unit Test Automation at Microsoft, in Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE), IEEE, November 2009

Audris Mockus, Nachiappan Nagappan, and Trung T. Dinh-Trong, Test Coverage and Post-Verification Defects: A Multiple Case Study, in Proceedings of the ACM-IEEE Empirical Software Engineering and Measurement Conference (ESEM), IEEE Computer Society, October 2009

Andrew Begel and Nachiappan Nagappan, Coordination in Large-Scale Software Development: Helpful and Unhelpful Behaviors, no. MSR-TR-2009-135, 28 September 2009

Christian Bird, Nachiappan Nagappan, Premkumar Devanbu, Harald Gall, and Brendan Murphy, Does distributed development affect software quality? An empirical case study of Windows Vista, in Communications of the ACM, vol. 52, no. 8, pp. 85–93, Association for Computing Machinery, Inc., New York, NY, USA, August 2009

Stephan Neuhaus and Thomas Zimmermann, The Beauty and the Beast: Vulnerabilities in Red Hat's Packages, in Proceedings of the 2009 USENIX Annual Technical Conference (USENIX ATC), USENIX, June 2009

Andrew Sutherland and Gina Venolia, Can Peer Code Reviews be Exploited for Later Information Needs?, in Proc. ICSE 2009, IEEE, May 2009

Jorge Aranda and Gina Venolia, The Secret Life of Bugs: Going Past the Errors and Omissions in Software Repositories, in Proc. ICSE 2009, IEEE, May 2009

2008

Andrew Begel and Nachiappan Nagappan, Pair programming: what's in it for me?, in ESEM '08: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ACM, New York, NY, USA, October 2008

Andrew Begel and Beth Simon, Novice Software Developers, All Over Again, in ICER '08: Proceedings of the Fourth International Workshop on Computing Education Research, ACM, New York, NY, USA, September 2008

Andrew Begel and Nachiappan Nagappan, Global Software Development: Who Does It?, in International Conference on Global Software Engineering, IEEE Computer Society, August 2008

Lucas Layman, Nachiappan Nagappan, Sam Guckenheimer, Jeff Beehler, and Andrew Begel, Mining Software Effort Data: Preliminary Analysis of Visual Studio Team System Data, in Proceedings of the 2008 International Working Conference on Mining Software Repositories, ACM, New York, NY, USA, May 2008

Andrew Begel and Beth Simon, Struggles of new college graduates in their first software development job, in Proceedings of the 39th Technical Symposium on Computer Science Education, ACM, New York, NY, USA, March 2008

Andrew Begel, Effecting Change: Coordination in Large-scale Software Development, in Proceedings of the 2008 International Workshop on Cooperative and Human Aspects of Software Engineering, ACM, New York, NY, USA, 2008

2007

Andrew Begel and Nachiappan Nagappan, Usage and Perceptions of Agile Software Development in an Industrial Context: An Exploratory Study, in First International Symposium on Empirical Software Engineering and Metrics, IEEE Computer Society, September 2007

Andrew J. Ko, Robert DeLine, and Gina Venolia, Information Needs in Collocated Software Development Teams, in ICSE '07: Proceedings of the 29th international conference on Software Engineering, IEEE Computer Society, Washington, DC, USA, May 2007

Mauro Cherubini, Gina Venolia, Rob DeLine, and Andrew J. Ko, Let's go to the whiteboard: how and why software developers use drawings, in CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems, Association for Computing Machinery, Inc., New York, NY, USA, May 2007

2006

Li-Te Cheng, Anthony Cox, Rob DeLine, Cleidson de Souza, Kevin Schneider, Janice Singer, Margaret-Anne Storey, and Gina Venolia, Proceedings of Supporting the Social Side of Large-scale Software Development, a CSCW Workshop, Microsoft Research, November 2006

Thomas D. LaToza, Gina Venolia, and Robert DeLine, Maintaining mental models: a study of developer work habits, in ICSE '06: Proceedings of the 28th international conference on Software engineering, ACM, New York, NY, USA, May 2006

2005

Gina D. Venolia, Robert DeLine, and Thomas LaToza, Software Development at Microsoft Observed, no. MSR-TR-2005-140, October 2005