Earlier this month, at the Apache-UK academia workshop, I was
pushing Hadoop as something that mattered. In a not entirely
unrelated event, HP, Yahoo! and Intel have just announced a
Cloud Computing Test Bed, which will consist of 6+
datacentres, each for experimentation with cloud computing
applications. Hadoop and applications on top of this are going to
be a key part of this. But not the only things that run on it. It
really is a a testbed, not just a hadoop-to-go system. Which means,
if someone wants to do some OS fun, or play with completely new
applications, they can ask for time on some of the machines.
This makes the test bed interesting in two ways. Firstly, Hadoop
and the layers above it provide immediate value: map/reduce, data
mining, stuff on top. Secondly, nobody is saying Hadoop-only. If
someone wants to build a distributed object infrastructure on top
of WS-ResourceTransfer (who would do that), then they are free to
apply for test-bed time, alongside anyone else. This makes it
profoundly different from, say the OGSA-approved grid fabrics, and
gives it a bit of the flexibility of planet-lab.
There's still lots of details to get sorted out about how getting
access; the bias will be towards short-lived over long-life
computation, and initially it will be the companies and the partner
institutions that will be running code on the machines.
I certainly hope that alongside academic (including UK academic)
and industrial applications, open source projects get time too -not
just the Hadoop/HBase/Mahout + incubating layers, but things that
do interesting work with shared datasets on top of the tools.
Again, this is somewhere where some open-source/academic
collaboration would be interesting.
Some press:
This is really exciting stuff. I'm not going to add any more on
the topic right now, because I don't want to do anything that would
upset the press teams of the various companies, and make anything
resembling a forward looking statement. Certainly nothing I have
posted should be interpreted as any form of commitment by myself or
my employer. As usual.
