If it's your first night with EventMachine, you have to fight.
EventMachine in a nutshell
EventMachine is an implementation of the reactor pattern. It's not Twisted, nor Node, nor Erlang. It follows the Ruby way, hence you have to follow the EventMachine way.
The event-loop pattern is now mainstream
The event-loop pattern is quite simple. The actions are no more systematically sequential. It's not thread-based: there is only one thread for the event-loop, therefore execution is still sequential. This enforce a unique context, accessing variables is still atomic (no concurrency).
Classical asynchronous actions are waiting actions, such as IOs (network or disk access). This makes long-lived actions (CPU intensive) to jam the flow. Async actions are detached from the flow, and a callback is called when it finished. If you don't take care, you will have stairs of block (one for each callback) and troubles for handling errors, the now infamous callback spaghetti.
The EventMachine way
The first block handles the reactor, the second one is the callback.
What if I want to fetch more than one url?
As the event-loop pattern forces you to nest callbacks, you're building an infinite staircase. It also makes actions sequential, but you may don't want google to answer before yahoo. You also had to add one level of indentation per url.
A workaround is to handle it by hand:
Don't you love flashbacks in movies? This way, url fetches are parallel, and you've even get a callback when all actions are finished.
EventMachine provides a helper for such common patterns:
It is the official way, but IMO it turned out not as elegant as it could have been. The first
Proc handles the iterations and
return the response, the second
Proc is the finished callback.
Em-http-request provides a specific object for doing batch jobs in a simpler form:
Short, elegant, and specific, isn't it? As nobody cares about the specific requests' callbacks, they are out. The response is deferrable and MultiRequest collects the spreaded responses. You don't have to handle parallel or sequential multi-actions in your own API. It's really a job for EventMachine (or Synchrony)!
The mysterious Deferrable behavior
As seen above, an async action returns a deferrable response, which is a mysterious object bound a callback. Why not a classical
Proc? Why not a quick and dirty
DefaultDeferrable as you may have seen in EM's doc?
The callback is just a trigger, it is not responsible for giving the answer back: the deferrable object is the answer.
But Deferrable is a complex answer, providing a state, two (actual) triggers, a success callback and a error callback; and an optional timeout.
In this pattern, an async function returns a response immediately (a Deferrable one), but the actual value of the response will be available later. Think of it as a closed box that will open itself, later on.
Using API functions with a block is just a syntactic sugar, but don't forget to provide both a callback and an errorback, and more importantly, you must return a Deferrable.
Example of an API based on EventMachine
Let's take a simple example: airports are required to freely give weather information (no authentication layer to handle). So here is the first weather webservice I found in Google. It's PHP-based on the server side, response serialization is done the oldschool way, but it's free indeed and it just works.
And now an example with EventMachine's iterator:
Fiber, or the hidden-threaded way
It can be funny to fight with callback in daemon project, at least for some time. But it may come a time when you just yearn for sequential programming. Still you want to use this cool framework wich uses EventMachine to handle lots of parallels connections. How would you do then?
You could strive to parrallelize a few requests, but most of the time, you just want to be able to describe your needs in a sequential fashion, everywhere.
Fiber, shipped with Ruby 1.9, is the answer. A Fiber waits for the response, for you, pausing the execution in the middle of a flat code chunk using #yield and resuming execution at any time with #resume. We are using Synchrony here:
Usage is straightforward:
Sequential actions are now, well, sequential. Revolutionary.
In order to transform you previously async methods into a Fiber-aware one, you may prefix the method with a (as in *a*sync), then wraps it within a fiber which will be resumed in the callbacks. It can be made systematic, have a look of how Synchrony monkey patches common libraries.
You could also explicitly ask for parallel actions, using an async variant of your code.