h1. ØMQ transport layer plugin for Elasticsearch

This plugin exposes the REST interfaces of "Elasticsearch":http://www.elasticsearch.org over "ZeroMQ":http://www.zeromq.org/ messaging library.

h2. Versions

|_. ØMQ Transport Plugin|_. ElasticSearch	|_. ØMQ	|
|  master (0.0.3)   	|  master (0.19.2)	| 2.2	|
|     0.0.2         	|     0.18.2		| 2.1	|


h2. Installation

h3. Requirements

Before installing and using this plugin with Elasticsearch (0.19.2), you need to install ZeroMQ Java Binding library (2.2) on your system. This binding uses native library to work. 

The "ZeroMQ":http://www.zeromq.org/ website is a great place to start installing the libraries, specially the "Java binding page":http://www.zeromq.org/bindings:java

If you want to develop or modify this plugin, I encourage you to read the "ØMQ - The Guide":http://zguide.zeromq.org/page:all which is very well documented. 

h3. Installation

Type the command in your favorite shell :

<pre>
$ bin\plugin -install tlrx/transport-zeromq/0.0.3
</pre>

Elasticsearch automatically install the plugin:

<pre>
-> Installing tlrx/transport-zeromq/0.0.3...
Trying https://github.com/downloads/tlrx/transport-zeromq/transport-zeromq-0.0.3.zip...
Downloading ..........DONE
Installed transport-zeromq
</pre>

Then, you must replace the file *plugins/transport-zeromq/jzmq-1.0.0.jar* with the Java binding compiled for your system.

Finally, edit Elasticsearch configuration file @config/elasticsearch.yml@ and add the properties:

<pre>
# ZeroMQ Transport config
zeromq.router.bind: tcp://*:9700
zeromq.workers.threads: 2
zeromq.workers.bind: inproc://es_zeromq_workers
</pre>

Restart Elasticsearch.

If you have the following error
<pre>Initialization Failed ...
1) UnsatisfiedLinkError[no jzmq in java.library.path]2) NoClassDefFoundError[Could not initialize class org.zeromq.ZMQ]</pre>

You can try to update your LD_LIBRARY_PATH:
<pre>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib</pre>

You can also read the documentation of the "ØMQ Java binding":http://www.zeromq.org/bindings:java. 


h2. Behind the scene

The plugin exposes the REST interfaces of Elasticsearch over ØMQ sockets. The implementation uses a "router-dealer pattern":http://www.zeromq.org/sandbox:dealer, where multiple ROUTER sockets (2 by default, see @zeromq.workers.threads@) are listening to incoming messages (each in a dedicated thread) send by DEALER sockets on the @zeromq.router.bind@ address. This way, it is possible to send REST-like messages with ØMQ clients and get the replies back. 

For example, a ØMQ client can send the following message:
<pre>POST|/twitter/tweet/2|{"user" : "kimchy", "post_date" : "2009-11-15T14:12:12", "message" : "You know, for Search"}</pre>

It will receive the following response back: 
<pre>201|CREATED|{"ok" : true, "_index" : "twitter", "_type" : "tweet", "_id" : "2", "_version" : 1}</pre>

The transport layer converts ØMQ messages in a given format into REST request objects that can be handled by ES. 

The expected format for incoming messages is: 
<pre><code> <Method PUT,DELETE, POST...>|<URI, including parameters>|<JSON content> </code></pre>

The format for outcoming message is:
<pre><code> <Status code>|<Status name>|<JSON reply content> </code></pre>

But any other message format can be easely implemented if needed.



h3. Simple ØMQ client to test the plugin

The @SimpleClient@ Java class in test package shows how to create a simple ØMQ client and send messages. In your test, take care to add the native library to classpath (@-Djava.library.path=/usr/local/lib@).

h4. Add new document

Message:
<pre>java org.elasticsearch.zeromq.test.SimpleClient tcp://localhost:9700 PUT /twitter/tweet/2 "{\"user\": \"kimchy\",\"post_date\": \"2009-11-15T14:12:12\",\"message\": \"You know, for Search\"}"</pre>

Reply:
<pre>201|CREATED|{"ok" : true, "_index" : "twitter", "_type" : "tweet", "_id" : "2", "_version" : 1}</pre>


h4. Search for document

Message:
<pre>java org.elasticsearch.zeromq.test.SimpleClient tcp://localhost:9700 GET /twitter/tweet/_search?q=user:kimchy</pre>

Reply:
<pre>200|OK|{"took" : 29, "timed_out" : false, "_shards" : {"total" : 5, "successful" : 5, "failed" : 0}, "hits" : {"total" : 1, "max_score" : 0.30685282, "hits" : [{"_index" : "twitter", "_type" : "tweet", "_id" : "2", "_score" : 0.30685282, "_source" : {
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "You know, for Search"
}}]}}
</pre>


h4. Delete a document

Message:
<pre>java org.elasticsearch.zeromq.test.SimpleClient tcp://localhost:9700 DELETE /twitter/tweet/2</pre>

Reply:
<pre>200|OK|{"ok" : true, "found" : true, "_index" : "twitter", "_type" : "tweet", "_id" : "2", "_version" : 2}</pre>


h3. Other examples

The @ZMQTransportPluginTest@ Java class in test package has other examples.

-----

This software is licensed under the Apache License, version 2 ("ALv2").


Thanks to "David Pilato":https://github.com/dadoonet for the Maven pom and README files.