When Fuel meets Riak NoSQL database

Hi guys. I was fed up of writing a paper so I started to think what to show in the ESUG Awards with Fuel. As explained in my previous post, Fuel is a new general purpose and fast object graph serializer for Pharo.

The first thing it come to my mind, was the following: debug something from the workspace and in the middle of the debugging, serialize the debugger into a file. Then open another image, materialize the debugger, and continue debugging. The good news is that it worked from the first shoot! Nice! I will show it in ESUG.

Now, the second thing was something that was in my head since a couple of months: use Fuel to get a byte array representation of an object graph and store/retrieve it from a NoSQL database. Of course, there are several NoSQL databases, and even several wrappers for Pharo. For my experiments I choose Riak because I read it may have the best results. Runar Jordahl did the binding for such database and as many cases, the API is using HTTP protocol. Thanks to Sven Van Caekenberghe, we have now a wonderful HTTP library in Pharo Smalltalk: Zinc. Obviously, Riak client for Pharo uses Zinc.

Why is Riak important and how it is related to Fuel? Most web apps read intensive. Fuel is designed by scratch to be extremely fast during materialization. And Fuel fits nice with a NoSQL like Riak, since Riak reads fast as well. Minimizing latency on web pages is really important. With riak (commercial version) you can host your database on many datacenters. So you can have datacenters close to all customers. So…reading from Riak is fast, and materializing from Fuel is also fast. And now in Pharo we have a fast VM as well. 🙂

Ok….I should stop talking and show you the code. I will show you how to store and read a simple object graph into a Riak database using Fuel. Steps:

1) Install Riak database server. You can download it from here: http://downloads.basho.com/riak/CURRENT/. For more information about how to install and setup a Riak database you can read: http://wiki.basho.com/Installation-and-Setup.html

In my Mac OS, I just unzip the file and that was all. Let’s say  the directory where I have the unzip is /home/mariano/riak.

2) Start the server. In Mac I went to the directory /home/mariano/riak/bin. There you will find the executable Riak. So, just run “./riak start”.

3) Check the server is running. Just execute “./ping” and you should get a “pong”.  Another possibility is to go to your web browser and go to http://localhost:8098/riak/ or http://localhost:8098/ and you should see a couple of things.

4) Ok…..cool. Now, grab a Pharo image and install Fuel. Of course, if you are lazy like me, you can download an image from our greatest tester/slave, Jenkis: https://ci.lille.inria.fr/pharo/job/Fuel/. Anyway, if you want to take a Pharo image and install Fuel in it, just execute:

Gofer new
squeaksource: 'Fuel';
package: 'ConfigurationOfFuel';
((Smalltalk at: #ConfigurationOfFuel) project latestVersion) load.

5) Install Riak client by executing:

Gofer new
squeaksource: 'EpigentRiakInterface';
package: 'ConfigurationOfEpigentRiakInterface';

((Smalltalk at: #ConfigurationOfEpigentRiakInterface) project version: '0.2') load.

6) Since in step 2) we have started our server locally, our tests should use the local server. To make this work, you have to options: add a mapping between ‘riaktest’ and localhost in your “hosts” file or even simple, just modify the method #riakTestUrlString to something like:

^ 'http://localhost:8098/riak'

7) Now you should be able to run Riak tests and they should be green.  Of course, Fuel tests should be green as well.

Ok. Once you follow all those steps, you should have a Riak database running locally and a Riak client in Pharo using Zinc and being able to talk to the database. So…how do you store stuff in a NoSQL database? basically, data is stored with key / value. The key is usually a String and the value can be a string or something binary. You may have listen that NoSQL databases can be used as a persistency strategy for enterprise applications. But how can that be possible if Riak only accepts strings or bytes? Well, here is when the serializer plays its role. One possible solution is to use a JSON or XML serializer in which case you take your domain specific object graph and you serialize it to a text file which is then stored in the NoSQL database.

But with Fuel, we can serialize an object graph in a binary way, and we able to serialize it and materialize it really fast. Here is an example of how you can use Fuel with Riak:

| bucket storedObject materializedObject aRectangle client |
client := EpigentRiakClient newForRestConnection: 'http://localhost:8098/riak'.
aRectangle := Rectangle origin: (4@2) corner: (7@1).
storedObject := FLSerializer serializeInMemory: aRectangle.
bucket := client  bucketNamed:  'epigenttest' .
bucket atKey: 'test' put: storedObject type: 'application/binary'.
self assert: (bucket atKey: 'test') = storedObject.
materializedObject := FLMaterializer materializeFromByteArray: storedObject.
self assert: materializedObject origin = (4@2).
self assert: materializedObject corner = (7@1).

Basically, we use Fuel to serialize an object graph into a ByteArray. Then we store such object into a Blob in the database. And of course, we can then read it and materialize it. What do we gain with this? Now you are able to take any object graph and store/retrieve it from Riak database. Just replace “aRectangle” for whatever object you want. If you are developing a real application, this object may be your “root” which is usually a dictionary.

So….since I am a simple PhD student, I’ve already done my job: experiment. Now it is your time to get something real from this 😉

See you

6 thoughts on “When Fuel meets Riak NoSQL database

  1. Very interesting article Mariano, just in a few lines I learnt what Fuel is and a good example of use with Riak (another topic I’m interested, the NoSQL db’s).

    Like the last paragraph: “So….since I am a simple PhD student, I’ve already done my job: experiment. Now it is your time to get something real from this ;)” LOL

    1. Thanks German. I should have also said that I am totally newbie with NoSQL databases so I may have said something not completly correct. I also forget to say that this scheme should work with “any” serializer with “any” NoSQL database. What I mean is that the same way you can use Fuel for other NoSQL database, or you can use StOMP or whatever other serializer with Riak.

      1. Yes I understand. I was (also a work-to-complete-some-day) trying to develop a Cassandra Console using Pharo (just for learning). May be I could try the same example with Cassandra, in an installation I’ve somewhere….

Leave a Reply