Steve: Developing on the Edge - It doesn't scale
Steve: Developing on the Edge
Thoughts on development, Web-services, technology and mountains.
12Sep
Sun2004
It doesn't scale

Oh, this is a beautiful article. A deconstruction of the bandwidth bias of HTTP and the ramifications for RSS.

I think the issue is broader than that. As Bill says, RSS is one example of machine-scraping rather than human browsing. And HTTP is biased towards humans. Not just in rate of fetch, but in the protocol itself. Like all those error codes whose text detail only makes sense to people. Or in the way that because a response doesnt need to specify message length, there is nothing to stop a server serving up half a picture, or incomplete text. Something people detect, but machines, well, dont.

If RSS is the big symptom of a problem, then it is tractable. I use BlogLines as my aggregator; it only polls a site once regardless of the number of users. So using this shared viewer is as eco-friendly to the IPv4 backbone as taking a tram to work is in the real world. It's tractable as the update notifications and read-only data can be propagated -tree like- with ease.

Scaling POST handling is a different game, which is an issue in SOAPland. Unless there is a grid of things out there to offer edge-computation, or some fundamental discovery in database distribution, you are going to have bottlenecks in the server/cluster/farm that handles your request. And of course, there is the load of parsing the XML payloads.

Clients could do what they help, which I suppose is why all those proposals for a binary form of XML popped up at the W3C workshop. I have one. I call it "Corba" :)

Comments