1

I have several queries that leverage storing data in memcached. Does it makes and is it against best practices to run a cron job to run these queries say every 15 minutes to keep the cache updated? Currently our cache expires every 30 minutes, and refreshes if someone comes to visit a page that executes the query and stores it in cache. Was wondering what others thought about this approach.

KingKongFrog
  • 111
  • 2
  • What are your requirements as far as how up to date the data has to be? –  May 18 '15 at 19:19
  • Would be called warmup query in other systems. It's quite common. Alternately you could trigger it when the underlying data changes if that doesn't happen too often. – thorsten müller May 18 '15 at 19:19
  • The requirements are not strict for the data to be up to date, however I just wanted to see if this was a viable approach. – KingKongFrog May 18 '15 at 19:23

1 Answers1

1

Well, using a timed service to warm a cache is typically what applications do when they need to not be stale beyond N units of time. So this is good.

However I would like to warn you against a strict N minute routine especially if there are some queries that are bound to take longer than N minutes in any scenario. What would happen here is that you would overwhelm the system (cron processes) and the DB with requests when the prior ones haven't returned. I would suggest considering something like N units of time since the last attempt. Of course you will need to see if that fits your requirements but this way you will never have a case where 2 cron jobs or 2 threads trying to run the same query to warm the same cache. And as the system slows down the chances are there will be more processes trying to do the same thing at the same point in wasteful CPU and Memory cycles.

On a less important note I would also suggest you try and integrate the jobs in your application instead of the cron job - but this is a matter of personal taste and if you are confident the cron job deployment in every new environment won't me missed or messed up.

Y123
  • 494
  • 4
  • 12