af83

Redis as a protocol

HTTP is the king of communication protocols, and it just won the internet war. Everything else is details; I love details.

Distant APIs

It has become almost indisputably common sense that non local services have to expose their APIs as REST APIs. From small private services to large cloud services, everybody speaks HTTP, the new lingua franca. You can debug with curl, handling authentification, compression, content negociation, caching and all other http wonders. But it has a cost, an overhead, and I don't even speak about SOAP here.

However there is a challenger for doing RPC without language restrictions.

Thrift was created by FaceBook for exposing services to different languages. It uses a neutral grammar to generate code for your language. The protocol specifies different communication layers (like TCP), rpc and errors; just like SOAP without XML and HTTP. It's a nice thing for class oriented developers.

You can do similar things with Google's Protobuf, albeit with more work, because only serialization and grammar are specified. There's also Avro, which is an attempt to simplify Thrift while cutting the link with Facebook. And it's more geared towards dynamic languages, Ilya Grigoric explains it better than me.

Minimalism

I wanted to try something distinct from HTTP. Something connected, with a simple message protocol, similar to communications between actors in the actor pattern.

Such a protocol is the memcache protocol. It's used to silently replace a memcache server (see CouchBase), or to expose a simple persistant connection (ElasticSearch).

When implementing a protocol used by many servers, you can use handcrafted and optimized client implementations…

So, if it works with Memcached, why not try Redis? The number of clients is huge, most of them based on hiredis, the official C client. And the protocol is extensible.

You can put a real Redis in your stack and mix Redis' patterns (queue, pubsub, incremental counter, cache…). All that while using a single protocol between application services and Redis services.

Expose your services as a Redis server

I chose nodejs to make some tests. Node is asynchronous and the king of POC; I could have chosen Erlang or EventMachine, but I wanted to see something running quickly.

The server is simple: a TCP server feeding a Redis parser. The parser is a hidden API, but node is not so strict. When a command is complete, a reply event is thrown, and you can handle the reply with the simple types available in Redis.

var redisd = require('redisd');

var server = redisd.createServer(function(command) {
    if (command[0] === 'info') {
        this.encode('redis_version:2.4.5');// I'm a liar
    } else {
        this.singleLine('OK');
    }
});

server.listen(4242, function() {
    console.log('server listening on port 4242');
});

You can test this server with redis-cli and your favorite language API. This project is small, it's just glue between well tested products. But now you can go ahead and build your own, with gevent, eventmachine, erlang … or implementing the MULTI/EXEC patterns…

blog comments powered by Disqus