Boosting SandstoneDB with Fuel

Hi guys. I continue to find nice Fuel examples to show at ESUG, and I while doing so, I post my results πŸ˜‰ . It may be useful for someone else apart from myself.Β  SandstoneDB is a database similar to Ruby’s ActiveRecord. SandstoneDB is a simple object database that uses (by default) SmartRefStreams to serialize clusters of objects to disk. It seems that currently there is also a GOODS database backend.

For more details about SandstoneDB please read: There is also an excellent screencast about it in PharoCasts.

In summary, SandstoneDB is a simple database which I think it may be useful for small/medium applications. Anyway….the goal of this short post is to show you how to speed-up SandstoneDB by using Fual serializer instead of SmartRefStream. Ramon Leon, author of SandstoneDB did the integration between the both. This was pretty straighforward and worked out of the box in the lass one hour. It was necessary to only create the class SDFuelSerializer with two methods: #serialize: anObject toFile: aFileΒ  and #materializeFromFile: aFile.

So….how to install and use SandstoneDB with Fuel? Quite easy in fact:

1) Grab a Pharo image and install Fuel. Otherwise, you can download an image from Jenkis: Anyway, if you want to take a Pharo image and install Fuel in it, just execute:

Gofer new
squeaksource: 'MetacelloRepository';
package: 'ConfigurationOfFuel';

((Smalltalk at: #ConfigurationOfFuel) project version: '1.6') load.

2) Install SandstoneDB and its integration with Fuel:

Gofer new
squeaksource: 'SandstoneDb';
package: 'SandstoneDb';
package: 'SandstoneDbTests';
package: 'SandstoneDbFuel';

3) If you run now the tests with SandstoneDB you will still be using SmartRefStream instead of Fuel, even if we have downloaded the package “SandstoneDbFuel”. So if you want to use Fuel as the default serializer you have to evaluate:

SDFileStore serializer: SDFuelSerializer new.

4) That’s all πŸ™‚ You can now run all SandstoneDB tests, they all should be using Fuel and they all should be green πŸ™‚

Now, how much speed-up you can have? Well…that depends on the graph you are serializing. The bigger it is the graph to serialize, the faster Fuel will be over SmartRefStream. If the graph is large, Fuel can be like 10x faster than SmartRefStream. If it is small, much less. The following is a benchmark where we individually serialize 500 persons. Each person is small, hence there is not that big difference. Nevertheless, in this test, Fuel is 2.5 times faster in serialization 1.5 times in materialization.

| commitTime people lookupTime loadTime |
 "Β  SDFileStore serializer:Β  SDSmartRefStreamSerializer new."
 SDFileStore serializer: SDFuelSerializer new.
 setStore: SDFileStore new;
 "only want to warm up test models, not anything else that might be in this image"
 SDFooObject warmUp.
 SDPersonMock withAllSubclasses do: [ :each | each warmUp ].

people := (1 to: 500) collect: [ :it | SDPersonMock testPerson ].
 commitTime := [ people do: [ :each | each save ] ] timeToRun.
 lookupTime := [ people do: [ :each | SDPersonMock atId: each id ] ] timeToRun.
 loadTime := [
 SDActiveRecord resetStoreForLoad.
 withAllSubclassesDo: [ :each | SDActiveRecord store ensureForClass: each ];
 withAllSubclassesDo: [ :each | each warmUp ].
 SDActiveRecord store ensureForClass: SDFooObject.
 SDFooObject warmUp ] timeToRun.
 show: 'Serialiation time: ', commitTime asString;
 show: 'Materialization time: ', loadTime asString;
 SDPersonMock do: [ :each | [each delete] on: Error do: [] ].
 SDPersonMock coolDown.
 SDFooObject do: [:each | [each delete] on: Error do: [] ].
 SDPersonMock allSubclassesDo: [ :each | each coolDown ].
 Smalltalk garbageCollect

If we take the same example what instead of serializing each person individually we save serialize an object that contains all those persons, Fuel is 12x faster in serialization and 6x in materialization.

Anyway…all this was just a proof of concept. All I can tell you is that all tests are green and that Fuel seems to speed-up SandstoneDB. If you want to give it a try, please go ahead. And if you already have an application running with SandstoneDB and SmartRefStream I would appreciate if you can benchmark both since it is interesting to see a real case.

See you.

10 thoughts on “Boosting SandstoneDB with Fuel

    1. Thanks Laurent. Well….SeasideHosting or CMSBox (don’t remember which one) used ImageSegment a lot. So…at the end maybe you need Fuel for SmallHarbour πŸ˜‰

    1. Thanks Stef πŸ™‚ Well….with all your energy I guess you must have more than one source πŸ˜‰
      Anyway, you know you are the coolest PhD supervisor hahaha.

    1. Hi. Weird. What did you exatly evaluate? If I take a Pharo 1.2 and evaluate the following it works out of the box. It seems http:/ is down now…maybe it is that? try later … πŸ™

  1. Hi,
    is there a version of SandstoneDB that would work in Pharo 2.0? I tried it in combination with Fuel – but of course the FileSystem part of SandstoneDB does not work in 2.0..

    1. Hi Helen. I don’t think there is a version that runs in 2.0. But it should be very easy to migrate to the new FileSystem. I have already migrated a couple of packages. There is usually 3 or 4 operations done with files and directories. Once you know how to map them, the rest is do again and again the same. If you send me some pieces of code of the old FileDirectory I can tell you how to do it with the new FS.
      Anyway, there is also a FileDirectory compatibility layer that Camillo wrote for the new FD. You way want to try that also

Leave a Reply