Avoiding the N+1 Query Problem in Rails

A very common query optimization case with a solution that’s also easy on the eyes

The n+1 query problem is one of the most common scalability bottlenecks. It involves fetching a list of resources from a database that includes other associated resources within them. This means that we might have to query for the associated resources separately. So if you have a list of n parent objects, another n queries will have to be executed for fetching the associated resources. Let’s try to get rid of this O(n) conundrum.

If you’re comfortable with Rails, Active Model Serializers, and already have a good idea about what our problem is going to be, then maybe you can jump straight into the code here.

A Concrete Example

Say you’re fetching an array of Post objects at a GET endpoint. You also want to load the respective authors of the posts, embedding an author object within each of the post objects. Here’s a naive way of doing it:

For each of the n Post objects being rendered, a query will run to fetch the corresponding User object. Hence we’ll run a total of n+1 queries. This is disastrous. And here’s how you fix it by eager loading the User object:

When A Simple Join Is Not Possible

Until now there’s been nothing new for the experienced.

But let’s complicate this. Let’s assume that the site’s users are not being stored in the same RDMS as the posts are. Rather, the users are documents stored in MongoDB (for whatever reason). How do we modify our Post serializer to fetch the user now, optimally? This would be going back to square one: