Rails without Webpacker

We recently removed webpacker from our Rails 5 application. While it was a great way to bootstrap a new Rails app with a significant amount of Javascript we encountered some serious performance problems that we felt really impacted our productivity. We build our application images after every git push and we’d like to be able to deploy them quickly.

We started out using webpacker to build our Javascript bundles and eventually switched to using webpack directly after seeing our asset build times exceed eight minutes. The end result was a 75% reduction in time spent building assets. This is a summary of our findings and the steps you can take to use vanilla webpack in your Rails application.

Fast deploys are important to us

A short feedback loop is important to us to ensure our velocity stays high and that we’re delivering software that best solves our customers’ needs. There are two main benefits to prioritizing the delivery of our software: reduced risk through smaller, more frequent deploys and higher velocity through smaller adjustments to strategy and design by incorporating feedback from customers and team members as early as possible.

Also, we’re a distributed team and the digital equivalent of a coworker rolling over to take a look at your local environment isn’t entirely frictionless; yes, we have screen-sharing and video conference tooling but the experience isn’t quite as good as someone interacting with the application directly. Being able to quickly deploy a change to staging and get immediate feedback is very valuable.

As our application and team grew, we started deploying more frequently. Deploys would queue up as we were all pushing changes to staging leading to delays over 30 minutes to see your changes take effect.

A short history of Rails asset management

The developers of webpacker are putting a _lot_ of effort into speeding it up and I think that in time its performance will reach parity with the manual webpack setup we’ve implemented. Webpacker was great for us in the beginning. We were able to easily create new bundles and only include them on pages in which they were required. We didn’t need to think about manually compiling them, creating a manifest or expiring them in the CDN. Unfortunately we don’t have the engineering resources to dedicate towards working on improving webpacker itself so I needed to find another solution.

Long-time Rails developers are familiar with the evolution of asset management within the framework. Today there are a lot of complaints about the way assets are delivered in legacy applications but in our opinion there’s nothing intrinsically wrong with the way we used to do things. It may not have been perfect but in the end it solved our problems and allowed us to focus on what really matters; solving our customers’ problems.

Asset management has progressed tremendously since the early days of Rails; we’ve gone from committing unminified application code and dependencies to declarative dependency management and automatic transpilation/compilation of Typescript into bundles containing only the JS necessary to enable a specific function of an application.

Build profiling

We strive to ensure that our engineering decisions have a basis in data; how can we claim that our changes resulted in improvement if we can’t quantify and measure the beginning or end states? “I feel like this is faster” isn’t very helpful to team members and stakeholders, especially if the end result is actually a performance regression.

Our CI provider, CircleCI, already instruments each step of our build process and those metrics are available through their API. I plotted the overall duration of our build process and while it is very variable it was not trending up over time despite the increase in Javascript dependencies and application code. The rake assets:precompile step ran sprockets in about 30 seconds while the subsequent manifest creation and pack build accounted for the majority of our eight-minute build time.

We let this go for a long time because we were so focused on building the product but eventually we reached a breaking point. We were onboarding our first engineer and decided that some time needed to be spent on the developer experience. We simultaneously ripped out webpacker in favor of vanilla webpack. The end result was a reduction of the docker build duration by 75%; from eight or more minutes to just above two when gems and JS packages aren't being updated.

If you want to see these metrics locally and are running a recent version of Docker (18.09 introduced Buildkit) set DOCKER_BUILDKIT=1 when building your application locally. You'll see timings for each step of your build:

=> [20/21] RUN yarn install                                                                62.5s
=> [21/21] RUN RAILS_ENV=${rails_env} HOST=${host} bundle exec rails assets:precompile    308.4s

Removing webpacker

Removing webpacker itself is relatively straightforward; remove it from your Gemfile and remove config/webpacker.yml. The configuration in webpacker.yml needs to be transformed into environment-specific configuration files in config/webpacker/$ENVIRONMENT.yml so keep a copy around for this process. webpack-manifest\[-plugin\]

There are two packages named webpack-manifest involved in this process. One is a JS package to generate the manifest.json files Rails needs to serve asset bundles. The other is a Ruby Gem to consume those files to be used in Rails views. Once you add it to your application using yarn add webpack-manifest-plugin or npm install webpack-manifest-plugin it needs to be added to your webpack config files.

var ManifestPlugin = require('webpack-manifest-plugin');
config.plugins.prepend = (new ManifestPlugin());

That's it, webpack will now output packs/manifest.json for consumption by Rails.

Add gem 'webpack-manifest', '~> 0.2.4' to your Gemfile and then add the following to config/initializers/webpack_manifest.rb

WebpackManifest::Rails.configuration do |c|
      c.cache = !Rails.env.development?
      c.manifest = if Rails.env.development?
'http://localhost:9000/packs/manifest.json'
      else
Rails.root.join('public', 'packs', 'manifest.json')
      end
    end

This assumes you have webpack-dev-server running on localhost:9000; adjust if you're using docker-compose and putting it in a separate container, like we are. This lets Rails fetch the manifest as necessary to update the paths as development progresses; in staging/production it’ll read the static public/packs/manifest.json file at boot time as the paths won’t change during the lifetime of the process.

webpack-dev-server

Javascript needs to be recompiled on the fly as development progresses; webpack-dev-server is available to compile and serve JS to your browser as files change. It’s especially nice because it uses websockets to push changes to React components instead of requiring a refresh.

Install webpack-dev-server if it’s not already in your package.json (`yarn add webpack-dev-server`) and configure it in `development.js`:

process.env.NODE_ENV = process.env.NODE_ENV || 'development';
const environment = require('./environment');
const config = environment.toWebpackConfig();
config.devServer = {
  host: "localhost",
  port: 9000,
}
module.exports = config;

View changes

The only view changes we needed to make were replacing javascript_pack_tag with javascript_bundle_tag and stylesheet_pack_tag with stylesheet_bundle_tag. [javascript|stylesheet]_pack_tag come from webpack-manifest and are helpers to translate javascript_bundle_tag 'Incident' into <script src="/packs/js/Incident-c60ab3c3a7573e9691c9.js"></script> where the hash comes from `manifest.json`.

The last step is to invoke webpack directly in your build process; webpacker would automatically invoke webpack during `rake assets:precompile` but now we're responsible for running it. Our applications are built in Docker so this is all we needed to add to our Dockerfile:

RUN node webpack --config config/webpack/production.js

Instrumentation

We still have some room for improvement, a plugin called SpeedMeasurePlugin instruments each step of the webpack process and outputs the size of each bundle. I don't think that it's worth spending any time on our build process to reduce the 35 second Javascript build time but everyone's results are different. Check out SMP to see how long each step of your build takes:

const SpeedMeasurePlugin = require("speed-measure-webpack-plugin");
const smp = new SpeedMeasurePlugin();
const config = environment.toWebpackConfig();
module.exports = smp.wrap(config);

SMP  ⏱
General output time took 35.91 secs
 SMP  ⏱  Plugins
CaseSensitivePathsPlugin took 16.84 secs
WebpackAssetsManifest took 2.36 secs
MiniCssExtractPlugin took 1.83 secs
EnvironmentPlugin took 0.008 secs
 SMP  ⏱  Loaders
modules with no loaders took 27.8 secs
  module count = 2182
babel-loader took 10.96 secs
  module count = 350
mini-css-extract-plugin, and
css-loader, and
postcss-loader, and
sass-loader took 7.57 secs
  module count = 25
css-loader, and
postcss-loader, and
sass-loader took 7.53 secs
  module count = 25
mini-css-extract-plugin, and
css-loader, and
postcss-loader took 3.12 secs
  module count = 4
css-loader, and
postcss-loader took 2.96 secs
  module count = 4
file-loader took 1.56 secs
  module count = 7

End Result

=> [21/23] RUN yarn install                                                              68.6s
=> [22/23] RUN bundle exec rails assets:precompile   27.5s
=> [23/23] RUN node webpack --config config/webpack/production.js                        46.2s

We've dropped our build times from an average of 8:15 to under 3:00. This reduces how much we pay our CI provider for build credits and increases our team's velocity through delivering changes more quickly. This also helped with our local development experience and led to a significant increase in battery life which is great for developers who are constantly working from new places.

If any of the above was interesting or helpful to you, please reach out. We’d love to hear your feedback and experiences.

You just got paged. Now what?

FireHydrant helps every team master incident response with straightforward processes that build trust and make communication easy.

Learn How