Hanumant’s Java Workshop

Turbo Charged Java Development!

Off my mark with GridGain…

Dmitriy from GridGain was kind enough to point out that for a simple application like this, not much code needs to be written or modified. As he explains in his comment on my previous post, I used his suggestions and was able to run the application.

All I did was the following –

1. Made Job class implement Serializable.

2. Used @Gridify annotation on Job.run() method. (I think I should have named it execute instead of run to avoid unnecessary confusion with Thread.run()). 

3. In the Main, inserted GridFactory.start(), Thread.sleep() and GridFactory.end() .

public Main() {
        try {
            GridFactory.start();
            new Producer(jobs).start();
            new Consumer(jobs).start();

            //not sure how many Consumers should I create.
            //new Consumer(jobs).start(); 
            Thread.sleep(Long.MAX_VALUE);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            GridFactory.stop(true);
        }
    }

4. Added libraries (gridgain jar, other supporting jars, aspectjweaver jar) to the project.

5. Added -DGRIDGAIN_HOME and javaagent to VM parameters. BTW, for some reason, GridGain refused to start ( gridgain.bat from cmd line) when GRIDGAIN_HOME was set to “C:\Program Files\gridgain-1.5.1\bin”. But when I changed the blackslash to forward slash, it worked!  This is on WinXP.

So after these steps, I was able to see that Job.run() was being shipped off to different nodes. At this time, I have a few questions –

1. What happens when a Consumer picks up a Job from Jobs and calls job.run(). Since the run() method is gridified, when is that Consumer ready to pick up the next job? Immediately after the run() method is shipped off to be executed to another node? or after the run() method is done execution on the other node? What I am expecting is that since the execution of run() is shipped off to another node, the consumer should pick up the next available job and ship it off to another node. Is that a valid expectation? Is this happening is this sample application?

2. This question depends on the answer of the first one. How many Consumers should I start from Main? Starting a consumer is done through the code while number of grid nodes can be changed (by killing or starting new nodes) at anytime. So how do I make sure all the nodes of the grid are utilized. If a Consumer becomes ready to pick up a new Job as soon as it pick up one Job and sends it off to another node, then I just need one Consumer. If the Consumer.run() on the Main node waits until the Job.run() finishes execution on the remote node, then I need to start as many consumers as I have grid nodes.

May be Dmitriy can throw some light on these questions 🙂

October 1, 2007 - Posted by | Java

2 Comments »

  1. Hi Hanumant,

    What happens when a Consumer picks up a Job from Jobs and calls job.run().

    The behavior of grid-enabled methods is just as it would be local – the method has to behave identically with or without grid. In your code, consumer synchronously waits for job completion and that’s exactly what happens after grid-enabling. What you are suggesting is that you want your consumer to wait asynchronously – but for that you have to update your own logic. The advantage of remote execution is that with addition of remote nodes you can afford to have many more consumers running locally in parallel.

    How many Consumers should I start from Main?

    In your Terracotta example you started one producer and one consumer. So one node was producing jobs in one thread and another node was consuming jobs also in one thread. So you simply were not able to control how many consumers or producers you could have, and, hence, didn’t ask this question then 🙂

    The answer to your question is that every application is different. If you see that application is under-performing, then start another node to help out. If you see that application is idling, then either start more consumers or remove nodes to get better CPU utilization.

    One of the big differences you didn’t mention is that with GridGain your remote nodes did not require any extra Jars or configuration – all your code was peer-loaded automatically onto remote nodes for execution. Just imagine what kind of time-saver it is during development.

    Best,
    Dmitriy Setrakyan
    GridGain – Grid Computing Made Simple

    Comment by Dmitriy Setrakyan | October 1, 2007 | Reply

  2. Hi Dmitriy, Thanks for your comments.

    >>In your Terracotta example you started one producer and one consumer. So one node was producing jobs in one thread and another node was consuming jobs also in one thread. So you simply were not able to control how many consumers or producers you could have, and, hence, didn’t ask this question then.

    I don’t think that is correct. I was able to control the number of producers or consumers by starting either a producer or a consumer on another node (see the main method). So I could have 1 producer and 2 consumers.

    It looks like I can’t do so in GridGain without changing the code. I can add more grid nodes but since I have 1 Producer and 1 Consumer, it doesn’t matter how many nodes I have, the Job instances will still be processed sequentially by that Consumer. While Job.run() may be executed on another node, and the main node and all other nodes are idling. So basically, I will have to know before hand how many nodes do I have and accordingly start the number of Consumers.

    Please let me know if that is not so.

    Comment by Hanumant Deshmukh | October 2, 2007 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: