Increase Your Rails Application Reporting Speed by 10x with Sunspot's Stored Values

22 August 2016 on . 3 minutes to read

Storage

Intro

In my previous post I introduced Solr and Sunspot along with a basic use case to get up and running with Rails. Here, I’ll dive in to what originally brought me to Sunspot, stored attributes. Wouldn’t it be great if you could dynamically generate a field to save complex calculations on your models with having to add columns to your database, and be sure that these calculations are kept up to date? This would especially be helpful for generating reports or views where you have lots of complex data that would normally take long to both load and calculate. Look no further, Solr’s stored attributes on hits have you covered.

Stored Attributes

What is it, and why store an attribute?

A stored attribute is a field that’s saved on a Solr document. However, they are also returned with a search hit, so you can directly read the stored value without accessing the underlying result. This saves you a database query, which is consistently the slowest point in application performance.

What else can they be used for?

Besides being able to search solely persisted fields, Sunspot lets you generate dynamic attributes which are kept updated via callbacks on your model. Say you want to perform a computationally complex calculation (such as division) and be able to search by this calculation.

The Problem and Solution

Teams and Requirements

class Team < ActiveRecord::Base
# id            :integer
# wins          :integer
# losses        :integer

  def win_to_loss_ratio
    wins.to_d/losses.to_d
  end
end

That’s a basic setup of relevant information for creating a team, viewing their wins, losses, and calculating their W:L ratio. Imagine an app where instead of viewing team info one at a time, you’re displaying hundreds of teams’ W:L stats on a single page. Not optimal, but that’s what your client wants, so that’s what you’ll give them, dammit!

You render each team and calculate each :win_to_loss_ratio. However, instantiating all the teams and performing these calculations is expensive. It also has to be done every single time a page is refreshed. All these sports fanatics hammering the F5 key each game day… it’s brutal. Time to step up your own game.

Searchable Block

searchable do
  double :win_to_loss_ratio, stored: true do
    win_to_loss_ratio
  end
end

The above will store the calculated W:L on each solr document. Since Sunspot updates an instance’s associated doc every time it’s saved, you’re guaranteed that the cached version of W:L is always kept up to date with a team’s stats. Even better, it only has to be calculated when it matters; when the stats change.

Retrieving Stored Values

Given a search that returns all the search docs (and assuming you’ve reindexed solr to cache everything), you can access the stored attributes with the following code:

search.hits.each |hit|
  puts hit.stored(:win_to_loss_ratio)
end

As simple as that. You’ll now iterate through the hits and print all the W:L stats.

Final Thoughts

Sunspot makes it trivial to define, store, and access both dynamically and statically calculated attributes to a search document so you can not only search but access cached values for your models. 10/10 for ease of development.

Performance

I’m compiling a set of benchmarks run on production data for an application I’m developing. In a future post I’ll lay out a similar schema to what I’m working with, along with real world solr benchmarks to accompany the data. Check back for the results, but I’ll tell you right now, that using solr for even basic calculations like addition is 10x quicker than instantiating models and calling methods on them.


If you enjoy having free time and the peace of mind that a professional is on your side, then you’d love to have me work on your project.

Contact or view a list of available services to see how I’ll make your life better, easier and bring satisfaction back into you running your business.