subject: Grid gets down to business posted: Sat, 01 Dec 2007 16:52:19 -0000
[This article is 3 years old, but it seems to capture a few
limitations of grid networks that are likely to pose problems for
some time to come. In particular, this quote: "We're seeing adoption
of this technology largely for applications needing high
computational performance that also have application logic or data
sets that can be segmented" - implies that apps whose logic and data
cannot be segmented will have issues with the grid model. As an
example of what this means, imagine a massive amount of data, which
needs to have a single sum done to each data item. This data can be
split into, say, 1000 chunks and sent to 1000 different computers,
who all perform the sums and send back the results. Fine, the grid
works well here as the data is easily segmented, and we get our
results 1000 times faster. The problem will arise when the results
of the second chunk of data depend upon results of the first. In
this case computation on the second dataset cannot start until
computation on the first dataset is completed, ie, parallel
processing of the data cannot occur.
However, this is a simple grid design. A more complex (and as far as
I know, still theoretical) grid will be able to execute individual
threads from a given machine. This will require support at the
operating system level. At the moment, grids are a layer that is
stacked on top of the OS, client software that downloads jobs,
execute them and uploads the results to server software, also stacked
on top of the OS. An OS that integrates grid technology will not
require separate client or server software, and the use of grid
resources will be transparent to the applications running on it. The
OS joins the grid on startup, and then donates and consumes resources
as required. This model will require large amounts of network
bandwidth, as if it takes longer to ship the job to the grid, and
back, than it would to process it locally, there is no point in doing
it. However, network bandwidth is increasing at a rate even faster
than Moore's Law, particularly as the optics don't have the same
limitations as the electronics (eg. we can crank up the bandwidth on
any given segment of fibre, simply by increasing the sensitivity of
the photoreceptors at the end of the fibre).
Early enterprise adopters of grid computing praise benefits such as
ultra-speedy processing for heavy-duty applications.
By Alan Joch, Network World, 12/27/04
Just because a company traces its roots to 1775 doesn't mean it's
hidebound. Bowne & Co. certainly isn't. Earlier this year, the
centuries-old financial publishing firm did something few other
commercial companies have dared to do: Bowne launched a modest
production grid, the computing resource-sharing technology that's
received a lot of headlines but so far has seen little adoption
outside of academia and life sciences.
So far, Bowne's grid bet has paid off. With the grid, the company has
cut in half processing time for a key application that helps mutual
fund customers meet financial reporting regulations. Now, with the
help of DataSynapse's LiveCluster distributed-computing software,
reports that used to take two hours to complete are done in an hour.
That's a significant reduction for a company that has to churn out
hundreds of thousands of reports within production windows that might
last only a few days.
"We have a complicated production process with a lot of customers
trying to get work out at the same time. We get big spikes in
demand," says Ruth Harenchar, CIO at the New York company. "By
reducing bottlenecks, grid helps Bowne meet those demands and enables
it to seek out additional business."
As an added bonus, the speed boost came about using two Intel Xeon
processor machines running Windows 2000, which are less expensive
than the stand-alone proprietary Unix servers that used to do the
processing. LiveCluster resides on the Windows servers, not in a
separate middle tier.
Now Harenchar's IT staff is developing guidelines for future
activities, including identifying the applications that stand to
benefit most from grid and what technical hurdles might arise.
"Because grid is still immature, it's not plug and play," she says.
"I want an overall plan for a corporate-wide grid strategy before we
go further."
Harenchar's hesitation isn't unique. Despite some enthusiasm by early
adopters, IT managers at most large corporations still question
grid's ROI potential. "Much work is still required to convince the
more risk-averse majority of users across all verticals that grid
investments will pay dividends," The 451 Group concludes in its
recent report, "Grid Computing: Where is the Value?"
So far, grid has made the biggest inroads in the scientific
community, especially in life sciences, pharmaceuticals and seismic
processing for oil and gas. Financial organizations and manufacturers
in aerospace, automotive and electronics also have come onboard
slowly, with scattered implementations in other industries. Human
resources outsourcer Hewitt Associates recently put grid computing to
work on a pension calculation application (see story). The Enterprise
Grid Alliance, an industry consortium formed last spring, promotes
grid computing for any company that has to perform complex analyses
to get to market faster, says Peter Ffoulkes, chairman of the group's
marketing steering committee and a Sun executive.
These pioneers helped grow the worldwide market that includes grid
software to $6 billion in 2003, with gains of almost 20% compounded
annually expected through 2008, according to IDC. "We're seeing
adoption of this technology largely for applications needing high
computational performance that also have application logic or data
sets that can be segmented," says Dan Kusnetzky, program vice
president for system software at IDC. "Parallel processing techniques
can offer some benefit to these applications."
On-demand grids
These types of computing clusters, which string together processing
power from multiple networked computers to tackle demand spikes, is
just one grid incarnation. A second, more ambitious category that's
not yet commercialized, calls for massive sources of on-demand
computing power that corporations would draw on from a computing
services provider, much like drawing electricity from a traditional
power grid.
But except for tests in their R&D labs, non-scientific organizations
have been slow to adopt even the more mature clustering approach. The
hesitation stems in part from a host of unresolved data management
issues. IT executives aren't going to show widespread support for
enterprise-grade transactional environments on grids until data
management is a smoother process, The 451 Group says.
"Most of the available software is still focused on jobs that require
heavy computation, and it isn't good at trying to decide where or
when to carry out a task - one of the key requirements of real-time
transactional environments. It's also of little use in cases where a
task requirement is all communication and no computation, or where an
action is dependent upon the result of a previous one" such as in
ERP, CRM or accounting applications, the report says.
That could change as enterprise CIOs seek out new ways to break up
processing bottlenecks. Getting the most out of IT investments is
another grid selling point, Ffoulkes says. Some large organizations
might utilize less than 10% of their available computing resources at
any given time. By identifying and allocating idle processors across
a network, grid middleware can focus resources during demand spikes
and bring utilization rates close to 90%, he says. "If enterprises
can harness that potential, they can bring IT costs down and gain a
strategic advantage over competitors," Ffoulkes says.
But before that happens, even pioneers like Bowne need to see more in
the way of turnkey grid products. "I'm sure products will get better
over time, but before we put more grid technology into production
we'll have to do more testing to understand what we're dealing with,"
Harenchar says. "It's not like there are 5,000 other companies out
there that have already developed a checklist that says 'if you run
into this problem, flip these five switches and everything will work
fine.'"
Joch is a freelance writer in Francestown, N.H. He can be reached at
[email protected] .
