Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Empirical Studies of Software Engineering

Several groups in MSR are studying software engineering process and products using empirical methods. The methods range from statistical analysis of software repositories to observations of programmers at work.

Publications

2015

Brittany Johnson, Thomas Zimmermann, and Christian Bird, Appendix to The Effect of Work Environments on Productivity, no. MSR-TR-2015-66, 28 August 2015.

Christopher Theisen, Kim Herzig, Patrick Morrison, Brendan Murphy, and Laurie Williams, Approximating Attack Surfaces with Stack Traces, in Companion Proceedings of the 37th International Conference on Software Engineering, IEEE – Institute of Electrical and Electronics Engineers, May 2015.

Kim Herzig, Michaela Greiler, Jacek Czerwonka, and Brendan Murphy, The Art of Testing Less without Sacrificing Quality, in Proceedings of the 2015 International Conference on Software Engineering, IEEE – Institute of Electrical and Electronics Engineers, May 2015.

Kim Herzig and Nachiappan Nagappan, Empirically Detecting False Test Alarms Using Association Rules, in Companion Proceedings of the 37th International Conference on Software Engineering, IEEE – Institute of Electrical and Electronics Engineers, May 2015.

Miryung Kim, Thomas Zimmermann, Robert DeLine, and Andrew Begel, The Emerging Role of Data Scientists on Software Development Teams, no. MSR-TR-2015-30, 12 April 2015.

Patrick Morrison, Kim Herzig, Brendan Murphy, and Laurie Williams, Challenges with Applying Vulnerability Prediction Models, in Proceedings of the 2015 Symposium and Bootcamp on the Science of Security, ACM – Association for Computing Machinery, April 2015.

Abram Hindle, Christian Bird, Thomas Zimmermann, and Nachiappan Nagappan, Do Topics Make Sense to Managers and Developers?, in Empirical Software Engineering, vol. 20, no. 2, pp. 479-515, Springer, April 2015.

Emerson Murphy-Hill, Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan, The Design Space of Bug Fixes and How Developers Navigate It, in IEEE Transactions on Software Engineering, vol. 41, no. 1, pp. 65-81, IEEE – Institute of Electrical and Electronics Engineers, January 2015.

2014

Baishakhi Ray, Meiyappan Nagappan, Christian Bird, Nachiappan Nagappan, and Thomas Zimmermann, The Uniqueness of Changes: Characteristics and Applications, no. MSR-TR-2014-149, 25 November 2014.

André N. Meyer, Thomas Fritz, Gail C. Murphy, and Thomas Zimmermann, Software Developers’ Perceptions of Productivity, in FSE '14: Proceedings of the 22nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering, ACM – Association for Computing Machinery, November 2014.

Kim Herzig, Using Pre-Release Test Failures to Build Early Post-Release Defect Prediction Models, in Proceedings of the 25th International Symposium on Software Reliability Engineering, IEEE – Institute of Electrical and Electronics Engineers, November 2014.

Ashish Gupta, Thomas Zimmermann, Christian Bird, Nachiappan Nagappan, Thirumalesh Bhat, and Syed Emran, Mining Energy Traces to Aid in Software Development: An Empirical Case Study, in ESEM '14: Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurements, ACM – Association for Computing Machinery, September 2014.

Kim Herzig and Nachiappan Nagappan, The Impact of Test Ownership and Team Structure on the Reliability and Effectiveness of Quality Test Runs, in Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurement, ACM, September 2014.

Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan, An Empirical Study of Refactoring Challenges and Benefits at Microsoft, in IEEE Transactions on Software Engineering, vol. 40, no. 7, IEEE – Institute of Electrical and Electronics Engineers, July 2014.

Christian Bird, Venkatesh-Prasad Ranganath, Thomas Zimmermann, Nachiappan Nagappan, and Andreas Zeller, Extrinsic Influence Factors in Software Reliability: A Study of 200,000 Windows Machines, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014.

Emerson Murphy-Hill, Thomas Zimmermann, and Nachiappan Nagappan, Cowboys, Ankle Sprains, and Keepers of Quality: How Is Video Game Development Different from Software Development?, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014.

Shaun Phillips, Thomas Zimmermann, and Christian Bird, Understanding and Improving Software Build Teams, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014.

Andrew Begel and Thomas Zimmermann, Analyze This! 145 Questions for Data Scientists in Software Engineering, in Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), ACM, June 2014.

Emerson Murphy-Hill, Thomas Zimmermann, and Nachiappan Nagappan, Motion Sickness Susceptibility in Software Developers, no. MSR-TR-2014-24, 4 March 2014.

Brendan Murphy, JAcek Czerwonka, and Laurie Williams, Branching Taxonomy, no. MSR-TR-2014-23, 26 February 2014.

2013

Kim Herzig, Sascha Just, Andreas Rau, and Andreas Zeller, Predicting Defects Using Change Genealogies, in Proceedings of the 2013 IEEE 24nd International Symposium on Software Reliability Engineering, IEEE, November 2013.

Andrew Begel and Thomas Zimmermann, Analyze This! 145 Questions for Data Scientists in Software Engineering, no. MSR-TR-2013-111, 28 October 2013.

Brendan Murphy, Christian Bird, Thomas Zimmermann, Laurie Williams, Nachiappan Nagappan, and Andrew Begel, Have Agile Techniques been the Silver Bullet for Software Development at Microsoft?, ACM ESEM , 11 October 2013.

Andrew Begel and Thomas Zimmermann, Appendix to Analyze This! 145 Questions for Data Scientists in Software Engineering, no. MSR-TR-2013-84, 14 September 2013.

Meiyappan Nagappan, Thomas Zimmermann, and Christian Bird, Diversity in Software Engineering Research, in Proceedings of the 9th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2013), ACM, August 2013.

Emerson Murphy-Hill, Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan, The Design of Bug Fixes, in Proceedings of the 35th International Conference on Software Engineering (ICSE 2013), IEEE, May 2013.

Kim Herzig, Sascha Just, and Andreas Zeller, It's not a Bug, It's a Feature: How Misclassification Impacts Bug Prediction, in Proceedings of the 2013 International Conference on Software Engineering, IEEE, May 2013.

Ekrem Kocaguneli, Thomas Zimmermann, Christian Bird, Nachiappan Nagappan, and Tim Menzies, Distributed Development Considered Harmful?, in Proceedings of the International Conference on Software Engineering (Software Engineering in Practice track), IEEE, May 2013.

Emerson Murphy-Hill, Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan, Appendix to The Design of Bug Fixes, no. MSR-TR-2013-22, 15 April 2013.

Kim Herzig and Andreas Zeller, The Impact of Tangled Code Changes, in Proceedings of the 10th Working Conference on Mining Software Repositories, IEEE, 2013.

2012

Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan, A Field Study of Refactoring Challenges and Benefits, in Proceedings of the 20th International Symposium on Foundations of Software Engineering (FSE 2012), Association for Computing Machinery, Inc., November 2012.

Christian Bird and Thomas Zimmermann, Assessing the Value of Branches with What-if Analysis, in Proceedings of the 20th International Symposium on Foundations of Software Engineering (FSE 2012), Association for Computing Machinery, Inc., November 2012.

Abram Hindle, Christian Bird, Thomas Zimmermann, and Nachiappan Nagappan, Relating Requirements to Implementation via Topic Analysis, in Proceedings of the 2012 International Conference on Software Maintenance (ICSM 2012), IEEE, 25 September 2012.

Meiyappan Nagappan, Thomas Zimmermann, and Christian Bird, Representativeness in Software Engineering Research, no. MSR-TR-2012-93, 5 September 2012.

Raymond P.L. Buse and Thomas Zimmermann, Information Needs for Software Development Analytics, in Proceedings of the 34th International Conference on Software Engineering (ICSE 2012 SEIP Track), IEEE, June 2012.

Thomas Zimmermann, Nachiappan Nagappan, Philip J. Guo, and Brendan Murphy, Characterizing and Predicting Which Bugs Get Reopened, in Proceedings of the 34th International Conference on Software Engineering (ICSE 2012 SEIP Track), IEEE, June 2012.

Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan, Appendix to A Field Study of Refactoring Rationale, Benefits, and Challenges at Microsoft, no. MSR-TR-2012-4, 11 January 2012.

2011

Ashish Gupta, Thomas Zimmermann, Christian Bird, Nachiappan Nagappan, Thirumalesh Bhat, and Syed Emran, Detecting Energy Patterns in Software Development, no. MSR-TR-2011-106, 16 November 2011.

Tim Menzies, Andrew Butcher, Andrian Marcus, and Thomas Zimmermann, Local vs Global Models for Effort Estimation and Defect Prediction, in Proceedings of the 26st IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), IEEE, November 2011.

Alexander Tarvo, Thomas Zimmermann, and Jacek Czerwonka, An Integration Resolution Algorithm for Mining Multiple Branches in Version Control Systems (Industry Track), in Proceedings of the 27th IEEE International Conference on Software Maintenance (ICSM 2011), IEEE, September 2011.

Ayse Tosun Misirli, Brendan Murphy, Thomas Zimmermann, and Ayse Basar, An Explanatory Analysis on Eclipse Beta-Release Bugs Through In-Process Metrics, in Proceedings of the 8th International Workshop on Software Quality (WoSQ 2011), ACM, September 2011.

Sunghun Kim, Thomas Zimmermann, and Nachiappan Nagappan, Crash Graphs: An Aggregated View of Multiple Crashes to Improve Crash Triage (Practical Experience Report), in Proceedings of the 2011 IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2011), IEEE, July 2011.

Paul Luo Li, Ryan Kivett, Zhiyuan Zhan, Sung-eok Jeon, Nachiappan Nagappan, Brendan Murphy, and Andrew J. Ko, Characterizing the Difference Between Pre- and Post- Release Versions of Software, International Conference on Software Engineering, 25 May 2011.

Andrew Begel and Libby Hemphill, Not Seen and Not Heard, no. MSR-TR-2011-136, 25 April 2011.

Christian Bird, Brendan Murphy, Nachi Nagappan, and Thomas Zimmermann, Empirical Software Engineering at Microsoft Research, in Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), Association for Computing Machinery, Inc., 19 March 2011.

Thomas Zimmermann, Nachiappan Nagappan, Kim Herzig, Rahul Premraj, and Laurie Williams, An Empirical Study of the Factors Relating Field Failures and Dependencies, in Proceedings of the 4th International Conference on Software Testing, Verification and Validation (ICST), IEEE, March 2011.

Philip J. Guo, Thomas Zimmermann, Nachiappan Nagappan, and Brendan Murphy, "Not My Bug!" and Other Reasons for Software Bug Report Reassignments, in Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), Association for Computing Machinery, Inc., March 2011.

Raymond P.L. Buse and Thomas Zimmermann, Information Needs for Software Development Analytics, no. MSR-TR-2011-8, 30 January 2011.

2010

Philip J. Guo, Thomas Zimmermann, Nachiappan Nagappan, and Brendan Murphy, Characterizing and Predicting Which Bugs Get Fixed: An Empirical Study of Microsoft Windows, in Proceedings of the 32th International Conference on Software Engineering (ICSE), Association for Computing Machinery, Inc., 2 May 2010.

2009

Laurie Williams, Gunnar Kudrjavets, and Nachiappan Nagappan, On the Effectiveness of Unit Test Automation at Microsoft, in Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE), IEEE, November 2009.

Audris Mockus, Nachiappan Nagappan, and Trung T. Dinh-Trong, Test Coverage and Post-Verification Defects: A Multiple Case Study, in Proceedings of the ACM-IEEE Empirical Software Engineering and Measurement Conference (ESEM), IEEE Computer Society, October 2009.

Andrew Begel and Nachiappan Nagappan, Coordination in Large-Scale Software Development: Helpful and Unhelpful Behaviors, no. MSR-TR-2009-135, 28 September 2009.

Christian Bird, Nachiappan Nagappan, Premkumar Devanbu, Harald Gall, and Brendan Murphy, Does distributed development affect software quality? An empirical case study of Windows Vista, in Communications of the ACM, vol. 52, no. 8, pp. 85–93, Association for Computing Machinery, Inc., New York, NY, USA, August 2009.

Stephan Neuhaus and Thomas Zimmermann, The Beauty and the Beast: Vulnerabilities in Red Hat's Packages, in Proceedings of the 2009 USENIX Annual Technical Conference (USENIX ATC), USENIX, June 2009.

Jorge Aranda and Gina Venolia, The Secret Life of Bugs: Going Past the Errors and Omissions in Software Repositories, in Proc. ICSE 2009, IEEE, May 2009.

Andrew Sutherland and Gina Venolia, Can Peer Code Reviews be Exploited for Later Information Needs?, in Proc. ICSE 2009, IEEE, May 2009.

2008

Andrew Begel and Nachiappan Nagappan, Pair programming: what's in it for me?, in ESEM '08: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ACM, New York, NY, USA, October 2008.

Andrew Begel and Beth Simon, Novice Software Developers, All Over Again, in ICER '08: Proceedings of the Fourth International Workshop on Computing Education Research, ACM, New York, NY, USA, September 2008.

Andrew Begel and Nachiappan Nagappan, Global Software Development: Who Does It?, in International Conference on Global Software Engineering, IEEE Computer Society, August 2008.

Lucas Layman, Nachiappan Nagappan, Sam Guckenheimer, Jeff Beehler, and Andrew Begel, Mining Software Effort Data: Preliminary Analysis of Visual Studio Team System Data, in Proceedings of the 2008 International Working Conference on Mining Software Repositories, ACM, New York, NY, USA, May 2008.

Andrew Begel and Beth Simon, Struggles of new college graduates in their first software development job, in Proceedings of the 39th Technical Symposium on Computer Science Education, ACM, New York, NY, USA, March 2008.

Andrew Begel, Effecting Change: Coordination in Large-scale Software Development, in Proceedings of the 2008 International Workshop on Cooperative and Human Aspects of Software Engineering, ACM, New York, NY, USA, 2008.

2007

Andrew Begel and Nachiappan Nagappan, Usage and Perceptions of Agile Software Development in an Industrial Context: An Exploratory Study, in First International Symposium on Empirical Software Engineering and Metrics, IEEE Computer Society, September 2007.

Mauro Cherubini, Gina Venolia, Rob DeLine, and Andrew J. Ko, Let's go to the whiteboard: how and why software developers use drawings, in CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems, Association for Computing Machinery, Inc., New York, NY, USA, May 2007.

Andrew J. Ko, Robert DeLine, and Gina Venolia, Information Needs in Collocated Software Development Teams, in ICSE '07: Proceedings of the 29th international conference on Software Engineering, IEEE Computer Society, Washington, DC, USA, May 2007.

2006

Li-Te Cheng, Anthony Cox, Rob DeLine, Cleidson de Souza, Kevin Schneider, Janice Singer, Margaret-Anne Storey, and Gina Venolia, Proceedings of Supporting the Social Side of Large-scale Software Development, a CSCW Workshop, Microsoft Research, November 2006.

Thomas D. LaToza, Gina Venolia, and Robert DeLine, Maintaining mental models: a study of developer work habits, in ICSE '06: Proceedings of the 28th international conference on Software engineering, ACM, New York, NY, USA, May 2006.

2005

Gina D. Venolia, Robert DeLine, and Thomas LaToza, Software Development at Microsoft Observed, no. MSR-TR-2005-140, October 2005.