At Salsify we run most of our Ruby on Rails based infrastructure in Heroku. We recently switched our Puma web workers over from Heroku 2X dynos to more powerful Heroku PX dynos and saw dramatic performance improvements: a 51% reduction in mean response time and a 59% reduction in 99th percentile response time. Based on this we did some experiments running our more memory/CPU intensive Delayed Jobs on PX dynos and saw a similarly encouraging 43% reduction in mean job execution time and 66% reduction in 99th percentile job execution time. This was great for a proof of concept, but only one of the eight cores on the PX dyno was being used. In order to take this to production in a cost-effective manner, we had to figure out how to utilize all of the cores on the dyno.

Most Rails developers quickly learn that minimizing the number of database queries by properly specifying eager loads is critical for good application performance. Unfortunately specifying eager loads is error prone and can cause encapsulation problems. In this post we'll explore having Rails automatically handle eager loads.

At Salsify we write lots of Capybara integration tests of our single-page app. These tests are a key part of the infrastructure that allow us to move fast and refactor without break anything. Unfortunately as we added more and more tests we started experiencing sporadic hangs and database deadlocks while tearing down tests. After some investigation we figured out the problem was our JavaScript client making AJAX requests to our Rails server during the test teardown process. This post details our journey to stamp out these problems and get our tests running reliably again.  

Previously we blogged about eager loading calculations with database views in Rails to avoid running lots of database queries. The approach we described really only works if your SQL is fairly static and you don't mind creating database views for each of your calculations. In this post we'll explore an alternative approach that avoids these issues by not using database views. First let's quickly recap the problem...

Update: The plugin discussed in this post has been packaged into the delayed_job_heartbeat_plugin gem.

Previously we've blogged about how to write Delayed Job plugins and how to aggregate jobs into job groups. In this post we'll explore how to proactively detect failed Delayed Job workers so their jobs can be retried in a timely manner. This is useful if a worker crashes, is automatically restarted by your platform provider, or is shutdown by auto-scaling infrastructure.

Recently I blogged about the Delayed Job plugin mechanism. In this post we'll explore how we can use this plugin mechanism to add job groups to Delayed Job allowing us to:

Salsify is a single-page app (Backbone on the client, Rails + Postgres + Elastic Search on the backend) that requires very low latency responses to provide an interactive experience for browsing and editing product catalogs. In order to satisfy these latency requirements we rely heavily on Delayed Job for running time consuming tasks like importing data, exporting data, and applying bulk updates in background processes. We've found useful pattern for using Delayed Jobs callbacks and hooks together that I wanted to share.