Hanumant’s Java Workshop

Turbo Charged Java Development!

Terracotta and GridGain comparison…

One of my objectives with this exercise was to be able to understand which one of these tools should we use in what situations. It looks like we can get some inferences with the help of this application.

Initially, before implementing this app on GridGain, I was not too sure on how to implement it so that we can have some similarity with the implementation on TerraCotta. In Terracotta, we shared a Jobs class instance accross JVMs and we started multiple Producers and Consumers who would look at the same Jobs instance and add/remove a Job to/from that instance. In effect, we were able to take advantage of multiple nodes by starting up either a Consumer or a Producer, as required, and achieved better performance. In other words, “sharing” enabled us take advantage of multiple machines. So I was hung up on finding out how to share things on GridGain.

After discussing this with Dimitriy, I learned that it would not be correct to look at GridGain from a “sharing” perspective. We should look at it from a “task” perspective…what task can be made a unit of work and can be executed on other  machines. In this application, Job.run() is such a task. In Terracotta, we isolated Producers and Consumers, while in GridGain, we isolated Job.run() method.

However, one drawback of this application is that Job.run() is a completely independent task and does not depend on anything. So no sharing or coordination between two JVMs is required. In Terracotta solution we were able to see how such coordination can be done among threads running on multiple machines but our application doesn’t touch this aspect on GridGain. I will try to modify it such that we can see how coordination can be achieved on GridGain. Any suggestions would be welcome!

Another important aspect of GridGain that we haven’t touched upon in this application is how to split a task, execute the parts on multiple nodes, bring back the results, join the results, and return the final output. I think this kind of a situation will take care of our sharing scenario as well.

Later…

October 1, 2007 Posted by | Java | 1 Comment

Off my mark with GridGain…

Dmitriy from GridGain was kind enough to point out that for a simple application like this, not much code needs to be written or modified. As he explains in his comment on my previous post, I used his suggestions and was able to run the application.

All I did was the following –

1. Made Job class implement Serializable.

2. Used @Gridify annotation on Job.run() method. (I think I should have named it execute instead of run to avoid unnecessary confusion with Thread.run()). 

3. In the Main, inserted GridFactory.start(), Thread.sleep() and GridFactory.end() .

public Main() {
        try {
            GridFactory.start();
            new Producer(jobs).start();
            new Consumer(jobs).start();

            //not sure how many Consumers should I create.
            //new Consumer(jobs).start(); 
            Thread.sleep(Long.MAX_VALUE);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            GridFactory.stop(true);
        }
    }

4. Added libraries (gridgain jar, other supporting jars, aspectjweaver jar) to the project.

5. Added -DGRIDGAIN_HOME and javaagent to VM parameters. BTW, for some reason, GridGain refused to start ( gridgain.bat from cmd line) when GRIDGAIN_HOME was set to “C:\Program Files\gridgain-1.5.1\bin”. But when I changed the blackslash to forward slash, it worked!  This is on WinXP.

So after these steps, I was able to see that Job.run() was being shipped off to different nodes. At this time, I have a few questions –

1. What happens when a Consumer picks up a Job from Jobs and calls job.run(). Since the run() method is gridified, when is that Consumer ready to pick up the next job? Immediately after the run() method is shipped off to be executed to another node? or after the run() method is done execution on the other node? What I am expecting is that since the execution of run() is shipped off to another node, the consumer should pick up the next available job and ship it off to another node. Is that a valid expectation? Is this happening is this sample application?

2. This question depends on the answer of the first one. How many Consumers should I start from Main? Starting a consumer is done through the code while number of grid nodes can be changed (by killing or starting new nodes) at anytime. So how do I make sure all the nodes of the grid are utilized. If a Consumer becomes ready to pick up a new Job as soon as it pick up one Job and sends it off to another node, then I just need one Consumer. If the Consumer.run() on the Main node waits until the Job.run() finishes execution on the remote node, then I need to start as many consumers as I have grid nodes.

May be Dmitriy can throw some light on these questions 🙂

October 1, 2007 Posted by | Java | 2 Comments