subject: review of grid software
posted: Mon, 13 Dec 2004 13:09:57 -0000

This [short] article is to describe my current experiences with the
World Community Grid.

What is the World Community Grid?

The WCG is a distributed computing platform; it is attempting to
process large quantities of data by breaking the data up into chunks
and shipping it to client nodes for processing. When the client is
done, it uploads the results to the server and downloads a new task.

This process is similar in nature to the SETI screensaver. However,
the WCG software is different, in that it can run a variety of
different applications. The SETI screensaver had one function - SETI
- the WCG agent, as the client software is known, provides an API to
the grid application; in this way multiple applications can be
written for the API and they all simply plug in.

In this sense, the WCG agent is significantly more advanced than the
SETI software. Currently the WCG agent is processing a task called
"Human Proteome Folding" (info about the task is on the WCG website),
however it seems extremely likely that it will run other tasks in the
future. The SETI task could be run over the WCG, for example.

Resource usage

The agent behaves extremely well. When the host computer needs its
CPU for something else, the agent drops off its CPU utilisation
immediately (within half a second). When the host computer becomes
idle again, the agent immediately kicks back in. The agent also has
a snooze button which stops it number-crunching for 5 minutes, so if
you need full access to the CPU, it's available. The agent has
crashed only once in over 165 hours of operation. And it has proved
resilient to machine lockups and crashes - when the host computer
crashes the agent does not lose all its work and start over at 0%
complete. It appears to periodically write results to disk.

Diskspace used is currently minimal; the binaries are around 10Mb
total. Datafiles created by the task seem to be consuming less than
1Mb so far. Also, network bandwidth is not currently affected, since
the agent is almost always processing a task, which does not involve
interacting with the grid server.

Motivating factors

Participation in the grid is voluntary and earns no financial reward.
There is a scoring system, however this is only used to see which
agents have done the most work, and for the sense of personal
satisfaction that arises from seeing one's number move up the ranking
on the web. The wider benefit of course is to the planet as a whole,
as the tasks being executed are important to science. Participating
in the grid at this early stage of its life also helps the grid
engineers to improve the grid itself. The engineers of this grid
include IBM and United Devices.

The future

The WCG looks good to me. The agent is stable, and the technology
seems to work. The fact that the agent exports an API means it can
be used for multiple tasks. However, at least two obstacles remain
between widespread use of grids. This agent, which seems to be the
most advanced so far, only talks to one server, the WCG server. And,
the grid server software does not seem to be available for public
download. This means the WCG is essentially a *closed grid*, in that
it's not open for general-purpose use.

What Joe Public needs is an open grid. On an open grid, anyone can
submit a task for processing, and people are free to instruct their
agents to connect to any number of servers.

I don't believe that this grid currently exists. However, the tools
are available to build it. From :

"The Globus Toolkit is an open source software toolkit used for
building grids. It is being developed by the Globus Alliance and many
others all over the world. A growing number of projects and companies
are using the Globus Toolkit to unlock the potential of grids for
their cause."

That's it for now.


* Origin: [adminz] tech, security, support (192:168/0.2)

generated by msg2page 0.06 on Jul 21, 2006 at 19:04:00

this site only