Should we namespace Redis?

We are enthusiastic users of Redis: it's an awesome tool to solve a large range of issues. Nowadays, most of our Ruby/Rails projects use it, and with great success.

And when we're working on several projects, of course we try to avoid conflict and overriding data from one project by data of another one. With mysql or mongoid, we use different databases, named by the project + the environment. For example, for the project foobar, I'll have two databases on my computer, foobar_development and foobar_test.

But Redis doesn't let you have several databases with this naming schema. So my teammates have used something else for their projects: redis-namespace. The idea is to prefix all keys with a given string (foobar_development in my previous example) and a separator (:) by subclassing the Redis library for Ruby.

It works fine, but I was reluctant to this solution: in my opinion, redis-namespace is a hack… A clever one, but with its drawbacks:

  1. It's not code that I want to put in production and run for years.
  2. It takes several bytes for each key (foo_development: is 17 bytes, so it's an overhead of 1Gb for 60M keys).
  3. Not all the commands are very well namespaced… e.g. flushdb will delete all the entries of your database, those with your namespace as well as the others.

To illustrate the last point, I found during a code review that we recoded an equivalent of flushdb by iterating on keys starting with our namespace and deleting them one by one. Do I really need to say that it's a lot slower than flushdb?

One way to mitigate this drawback of redis-namespace could be to use its Lua scripting power. Although, I don't feel it's a right way to do this either.

So, what do I have to offer? Not much, but let's see the actual state: Redis already handles the notion of databases. They are not named but numbered from 0 to 15 by default - but if need be, you can increase their number in config. And it's also possible to launch several instances of redis, each one with its TCP port.

Thus, Redis actually has all the mechanisms for separating environments in their own databases. It's just not very convenient for local development and continuous integration (how do I know which databases are free?). But since Redis-namespace is somewhat popular, it seems to indicate that this a true issue for many developers.

Maybe all we need is a small wrapper for the redis library which transforms a database name (foobar_development) in a database number (42)? Or launches an instance of redis on its own port? It should not be very difficult to map the database names to numbers and it won't have the drawbacks of redis-namespace…

So, what are your thoughts about it? Should I code a proof of code to test this idea?

blog comments powered by Disqus