Can you tell the JavaScript benchmark from the Real Web App?  

Abstract

JavaScript is widely used in web-based applications and is increasing popular with developers. So-called ”browser wars” in recent years have focused on JavaScript performance, specifically claiming comparative results based on benchmark suites such as SunSpider and V8. In this paper we evaluate the behavior of JavaScript web applications from commercial websites and compare this behavior with the benchmarks.

We measure three specific areas of JavaScript runtime behavior: 1) functions and code; 2) heap-allocated objects and data; 3) events and handlers. We find that the benchmarks are not representative of many real websites and that conclusions reached from measuring the benchmarks may be misleading.

Specific examples of such misleading conclusions include the following: that web applications have many loops, that non-string objects in web applications are extremely short-lived, and that web applications handle few events.

We hope our results will convince the JavaScript community to develop and adopt benchmarks that are more representative of real web applications.

Related Videos

• "JSMeter: Characterizing Real-World Behavior of JavaScript Programs" a 40 minute overview of the project and our results presented at the Velocity 2010 Web Performance and Operations Conference.
• "Project JSMeter: JavaScript Performance Analysis in the Real World" - an MSDN Channel 9 interview with Erik Meier, Ben Livshits, and Ben Zorn.
• "IE 9: First look at the new JS Engine" - an MSDN Channel 9 interview with IE9 JavaScript engine team members John Montgomery, Steve Lucco and Shanku Niyogi.  They discuss what is being done to speed up the JS engine, taking into account the needs of real-world web applications.

Other Media Coverage

• "Microsoft's JSMeter: A new way to analyze and affect JavaScript performance" by Mary Jo Foley, ZDNet All About Microsoft Blog, March 25, 2010.

People

• Ben Livshits (livshits@microsoft.com)
• Ben Zorn (zorn@microsoft.com)
• Paruj Ratanaworabhan, Kasetsart University (paruj.r@ku.ac.th)

Papers and presentations