A blog about Ruby, Rails and other Tech. Mostly.

5th Nov 2006, 1:34pm

Expiring the memcached cache in rails - tracking cached fragments

This is a brief description of how I solved my caching problem with rails. We have thousands of pages, which will all eventually be cached by the system. We list the actions to be cached, for example:

  caches_action :index, :list, :show

This works fine, but there are hundreds of ways to call each action with different parameters. So how do you sweep? Memcached does not allow you to iterate over the keys in the cache. So you need to keep track of cached items.

We added this after_filter:

  after_filter :cache_recorder, :only => [:index, :list, :show]

  # this records each cached page, so we can sweep them when needed
  # the records are stored in a number of memcache buckets to stop too much
  # traffic to memcached
  # see also sweep_cache
  def cache_recorder
    return if ENV['RAILS_ENV'] != "production"
    hash_digits = request.path.hash % CACHE_BUCKETS
    ca = CACHE.get "list_of_cached_pages#{hash_digits}"
    ca = [] unless Array === ca
    logger.info ca.to_yaml
    unless ca.include?(request.host + request.path)
      ca << (request.host + request.path)
      CACHE.set "list_of_cached_pages#{hash_digits}", ca
    end
  end

This adds a number if items to the cache that record the items stored in the cache. The records are kept in a number of buckets - I didn't fancy the idea of fetching a list of thousands of items from the cache each time I wanted to add something. So we split them up.

The number of buckets is a confirguation parameter, I set it in environment.rb:

CACHE_BUCKETS = 40  # number of buckets to use for cache of list of cached pages

Now the system remembers what was cached. We need to be able to sweep the cache, and it's easy because we have the lists of cached items to hand - here is my sweeper method accessible from our admin system:

  def sweep_cache
    if ENV['RAILS_ENV'] != "production"
      flash['notice'] = "Can't clean cache - not in production mode"
    else
      0.upto(CACHE_BUCKETS - 1) do |n|
        ca = CACHE.get "list_of_cached_pages#{n}"
        if Array === ca
          CACHE.delete "list_of_cached_pages#{n}"
          ca.each {|page| CACHE.delete page}
        end
      end
      flash['notice'] = "Cache cleaned"
    end
    redirect_to :action=>"index"
  end