Let Desi handle your menial Elastic Search chores!

As you may already have figured out, here at AF83, we like Elastic Search (and indeed, what's not to like in a distributed RESTful full-text search engine overloaded with features?).

However, having used Elastic Search project after project, either with the help of the extremely handy Tire gem or through direct HTTP access, we soon realized we were falling into the trap of copying and pasting crufty little snippets of code to deal with small but repetitive tasks. Tasks like deleting or emptying indices when running tests or starting the cluster on our laptops when beginning a work session for a given project. Or even just installing the sofware… (Mind you, it's incredibly easy — a wget to the archive hosted and Github and you are done — but as any self-respecting developer, we regard laziness as a cardinal virtue!)

Desi came out of the ensuing itch-scratching session. It is a very simple tool with limited ambitions but we find it handy for day-to-day tasks. It's useful to us and — who knows — it might be to you too!

What does it actually do, right now?

Nothing speaks more than a little hands-on experiment:

$ # First of all, let's install the gem, of course
$ gem install desi

$ # What can I do from there?
$ desi help

Available commands:
  help      Displays help for a command
  indices   List indices
  install   Install ES (to latest stable version by default)
  list      List locally installed Elastic Search releases
  releases  List latest ElasticSearch releases (latest 5 by default)
  restart   Start or restart Elastic Search (restart if already active)
  start     Start Elastic Search (do nothing if already active)
  status    Show current status
  stop      Stop Elastic Search

  -h, --help  Displays this help message

$ # OK. Let's install Elastic Search, then!
$ desi install
$ # We could also have explicitly requested a release number (0.19.9, for instance)

$ # What's its status? Is this thing on?
$ desi status
KO. No Elastic Search instance was found running on

$ # All right, let's get it started then.
$ desi start
 * Elastic Search 0.19.9 started
$ # The command did not return until the cluster became
$ # actually ready, so that we can chain commands.

$ # The fun begins. Let's index something
$ curl -XPUT http://localhost:9200/twitter/tweet/1 -d '{
    "user": "John Foobar",
    "message": "Heeello, world!"
$ # Is it indexed?
$ curl -XGET http://localhost:9200/twitter/tweet/1
{"_index":"twitter","_type":"tweet","_id":"1","_version":1,"exists":true, "_source" : {
    "user": "John Foobar",
    "message": "Heeello, world!"
} # It is!

$ # We should have one index at the moment. Let's check that out:
$ desi indices
Indices from host matching the pattern /.*/


$ # Let's create another index.
$ curl -XPUT http://localhost:9200/another_twitter_index/tweet/1 -d '{
    "user": "John Barfoo",
    "message": "Good morning, world!"

$ # And then a third
$ curl -XPUT http://localhost:9200/slashdot/comment/1 -d '{
    "user": "Anonymous Coward",
    "message": "First post!"

$ # Not only can we list indices, we can also delete and empy them:
$ desi help indices
Usage: desi indices [PATTERN]

  -d, --delete   Delete the specified indices (You've been warned!)
  -e, --empty    Delete all documents from the specified indices
  -h, --host     Elastic Search cluster URL
  -q, --quiet    Do not output anything
  -v, --verbose  Display information messages

$ # Let's remove all twitter something indices
$ desi indices twitter --delete
The following indices from host are now deleted
 * another_twitter_index
 * twitter

# We are then left with just one index
$ desi indices
Indices from host matching the pattern /.*/


# That's it for the little tour!

A command-line tool is fine, but what about its library API?

The README will show you little snippets of the most usual commands. And of course, the nice online documentation provided by rubydoc is there to get down to the nitty-gritty.

For the moment, the most useful feature API-wise is probably index manipulation. For example, we could wipe out the indices before and after a test suite and empty them before each test involving ES, which for Rspec would look somewhat like:

RSpec.configure do |config|
  config.before(:suite) do!("^fooapp-test")
  config.before(:each, :search => :true) do!("^fooapp-test")
  config.after(:suite) do!("^fooapp-test")

OK. It does a few things, but not much, what's next?

Well, it won't ever do that much, anyway. Its scope is voluntarily limited. However, a few more features would be nice, like for instance an upgrade command to automate data migration when installing a newer release. So feel free to create an issue if you want.

blog comments powered by Disqus