Speed up your bundler = personal rubygems proxy with cloudfront

Posted by Tiago Scolari on Oct 25, 2013
In Development, Ruby, Server

If you think your bundle install takes too long, or have a distributed team around the globe, this may help you.

It’s possible to have a cloudfront server pointing to rubygems, and using it as your gems source server. (If you are not aware of what cloudfront is give a look at it)

On this configuration, cloudfront will fetch requests from rubygems.org, and keep them cached on the CDN, so you should expect the first bundle install to take very close time of using rubygems.org, but the second one and forward should have some performance increase. You can also manually set the cache time there.

On a new cloudfront server, at the AWS console, add rubygems.org as origin:

I’m not a rubygems api expert, I’ve captured bundler requests and tried to cache them with “behaviours” on cloudfront. I’m also caching everything else for 10 minutes. These configurations probably can be tuned, drop a comment if you have suggestions :)

Things to have in mind:

  • origin must match viewer’s protocol
  • all the behaviours must forward query strings

On the Gemfile, you need to change the source to the new proxy server:

# Gemfile
source 'http://mycfdomain.cloudfront.net'
...

note: Theoretically https should work, but due some redirection it didn’t work for me.

Results

I’m using rvm gemset to reset scenarios between tests.

Using a rails 4 default gemfile:

  • using rubygems.org:

    bundle  6.37s user 1.59s system 21% cpu 37.840 total
    
  • using cloudfront proxy (FIRST TIME, no cached content):

    bundle  6.50s user 1.77s system 22% cpu 37.578 total
    
  • using cloudfron proxy (with cached content):

    bundle  6.35s user 1.60s system 30% cpu 25.664 total
    

Using “real-world” project Gemfile:

  • using rubygems.org:

    bundle  116.22s user 29.68s system 45% cpu 5:20.03 total
    
  • using cloudfront proxy (FIRST TIME, no cached content):

    bundle  112.99s user 29.69s system 44% cpu 5:23.24 total
    
  • using cloudfron proxy (with cached content):

    bundle  111.52s user 29.86s system 58% cpu 4:03.12 total
    

I think the difference on cached content is significative. It’s up to you to decide if it’s enough reason to pay a CF server for this.