The baseline costs of JavaScript frameworks: finding the X

We've been curious for a while about how much impact merely including a JavaScript framework has on the performance of a web page — in particular, after spending a rather frustrating afternoon trying to improve the parse and evaluation time of a JavaScript bundle on a cheap phone, only to find that most of the bundle’s size was taken up by React and its associated libraries.

Truth is, when you build your application on top of a modern JavaScript framework, you agree to pay a certain baseline performance cost that can never be optimised away. In exchange for paying this cost, you gain maintainability, developer efficiency, and (hopefully) better performance during runtime.

But you didn’t need me to tell you that.

Let’s phrase this differently: when you build your app on top of a JavaScript framework, you’re making peace with the fact that your app will never load in less than X seconds, for some value of X that varies from framework to framework.

So what exactly is this baseline performance cost, this X? Glad you asked! With a free afternoon ahead and a Redmi 6A lying around, it was time to run some numbers using React, Vue, and Angular.

Methodology

Here’s chalking out how the benchmark results you’ll see below came to be:

1. Initialized new React, Vue, and Angular projects using create-react-app, Vue CLI, and Angular CLI, and didn't make any changes to the projects generated by these tools.
2. Installed routing and data management libraries in each project, and made sure they were part of the JavaScript bundle.
3. Deployed all three projects to Netlify. Here are the pages: React, Vue, Angular.
4. Connected a Xiaomi Redmi 6A to a computer and ran a profile on all three pages using the Chrome DevTools.

The routing and data management libraries used for each project were: react-router and redux for React, vue-router and vuex for Vue, and the built-in Angular router and ngrx for Angular.

Upon having run the benchmarks, two numbers seemed interesting:

1. The time taken to download all the JavaScript required to render the page. Only the content download time was taken into account, and the network latency, the time required to setup an SSL connection, etc. ignored because of a lack of control over these factors as a bundle size.
2. Time taken to parse, compile, and evaluate the JavaScript bundle

At the time of writing, the Redmi 6A is a pretty new device, and most Indian users are still using the older Redmi 5A with a slower CPU.

The Numbers

Here are the numbers, starting with the bundle sizes for each application.

Angular’s JavaScript bundle is about twice large as the bundles for Vue and React! Our guess is that this is because Angular has a much larger API surface and ships with the entirety of RxJS.

While it takes a respectable 200 milliseconds for Chrome to parse and evaluate the React and Vue bundles, it takes over half a second to evaluate the Angular bundle. That’s a large enough interval for your users to notice, and can really cut into your performance budget.

Unsurprisingly, the React and Vue bundles are downloaded in under a second, but the Angular bundle takes over 2 seconds to download.

What’s important here is not how large or small the numbers are, or the relative performance of the three frameworks compared to each other. What’s noteworthy is the fact that your React application will never load faster than about 1.1 seconds on an average phone in India, no matter how much you optimize it. Your Angular app will always take at least 2.7 seconds to boot up. The users of your Vue app will need to wait at least 1 second before they can start using it.

This is when we haven’t even started looking at some of the other essential libraries that most projects end up using. This includes polyfills, form management libraries, date and time libraries, utility libraries such as lodash, drag and drop libraries, charting libraries, etc.

If you want your application to become interactive on your users’ devices in under 5 seconds, can you afford to spend a fifth of that time just booting up React?

Are Frameworks Evil?

We're not advocating that frameworks are evil, or that you should write all your applications in VanillaJS. That would waste far too much productive developer time, and the result will probably end up performing badly at runtime. Frameworks exist for a reason.

But it’s important to consider your audience. If you’re building for resource constrained devices — which you certainly are if your product targets a country like India — you could consider using a lighter framework such as Riot or Preact. Your users will thank you.

Or consider not using a framework at all. For websites that primarily display content, it’s more efficient and cost-effective to just send some server-rendered HTML down the wire. If there are areas of your website that require interactivity, you can always use JavaScript to build those specific parts.

In the end, it’s important to strike a balance between developer productivity and user experience. The answer will vary from project to project, so don’t take anyone’s advice as gospel. Build for your own audience, not somebody else’s. Run benchmarks on devices that represent real-world scenarios, and then decide what technologies make sense for you.