[First sent to mailing list subscribers on April 30th 2010.  To subscribe for early access join NetKernel portal https://cs.1060research.com/csp/]

What's new this week? xml-saxon update. NKP no show. 'Redirects' what's
all that about?

Repository Updates
------------------

xml-saxon:  Chris Cormack, the maintainer of xml-saxon, has published
some tweaks. Updated to use Saxon HE 9.2.0.6, fixed a possible
NullPointer in the catalog code and removed a println in the catalog code.

NKP Status Report
-----------------

We've been pushing the envelope with NKP.  We had a candidate release
prepared and to push it to the limits have implemented a general pubsub
framework with it. This shows that NKP is able to offer bi-directional
channels without the need for a callback server interface on the
subscribers end. (contrast with pubsubhubbub)

Implementing pubusb was outside of the original design goals and core
use cases that we wanted to cover. But it helped to highlight that,
whilst from the application level's point of view we can cleanly
implement the pattern, at the low-level we needed a more sophisticated
infrastructure.

In particular, each subscriber required a waiting thread in the
server-side hub. For small scale scenarios this is sort of ok, but we
aim for perfection, so we're close to completing a second generation
implementation with a true threadless asynchronous architecture.  This
will mean that NKP will handle very large numbers of concurrent
requests, including very long-lived requests, with a small impedance
matched set of threads corresponding to available cores.

So, sorry to say, nothing to play with yet.

[Incidentally, did you know that the NKEE enterprise http server
provides com.ten60.netkernel.tpt.http.NetKernelHandlerAsync. If you
switch to this handler in your etc/HTTPServerConfig.xml and combine it
with a throttle overlay after the HTTP transport, you have a threadless
async frontend for HTTP.  Talk to us if you want to know more about how
to set this up to handle extremely large HTTP concurrency.]

demo-redirect
-------------

We recently had a discussion about whether it made sense to implement a
redirect capability into the NK protocol.  A la HTTP's 30x response
codes and, at the Web-application level, the browser's detection of
redirects in the <meta> Refresh header of HTML representations.

It turns out that given that NKP transparently projects the ROC address
space abstraction between systems, and, ROC can treat
requests/identifiers as resources, then it doesn't make any sense to
make special cases in the protocol of what are really general ROC patterns.

To see what I mean. I knocked up a demo/tutorial "demo-redirect"
(available in the 4.1.0 apposite repos) which shows 4 different patterns
for implementing redirects in the ROC domain (and hence can be used for
NKP architectures too).  [Note these are ROC redirect patterns - nothing
at all to do with HTTP]

A static copy of the documentation for the demo is available here...

http://docs.netkernel.org/book/view/book:urn:org:netkernel:demo:redirect:guide/

(Note, the demo's don't work in the static copy - you'll need to install
'demo-redirect' with apposite to try it for real)

The demo is fairly mechanistic, in that, it shows the different physical
ways that you could implement redirection.  What's not addressed is the
philosophical perspective.  For example why doesn't NetKernel implement
redirect handling of requests at the kernel/protocol level?  It surely
could.

The answer to this lies in the old adage: one man's metadata is another
man's data.  A redirect is a resource representation, the abstraction's
job is to reify resource representations.  Things start to get very
muddy very quickly if we start to pollute layers and introduce
hard-coded context and/or semantic interpretation of resource values at
the kernel level.

As you'll see from the demo, its really very simple to introduce
architectural patterns that allow the application layer to perform
redirects.  That being so, its cleaner, and ultimately more consistent
and efficient (in terms of the re-use of computed state etc) to make it
an application architecture design decision.

Next question is, what purpose do redirects play in the ROC domain?

Probably the most important thing that redirects offer is the ability
for an application architecture to be decentralized (see search example
below).

But, as I mentioned previously, identifiers are relative, therefore
being able to redirect allows stable names to be used for transient
contextually defined identifiers.

Very interestingly, redirection allows loose, "approximate identifiers",
to be progressively tightened into more and more definitive and
authoritative identifiers. (That could be a semi-formal definition of
what we mean by "search".)

In fact we've used approximate identifier patterns in a number of parts
of NetKernel.  For example did you know that when you search, all the
results are actually redirection resources that only point to a resolver
for the specific category of documents?

When you click the search result you request the resolver which knows
about the rendering interface and redirects you to the identifier of the
rendered resource.  With this approach, the search sub-system's index
generation is focused souly on the textual source to index and has no
connection with the (completely unrelated) reification of HTML
representation of the content.

Another example in the doc system is the {luckysearch} wiki macro engine
we provide.  For example this...

"The {luckysearch}declarative request{/luckysearch} can be used to
implement redirect patterns"

Would replace the {luckysearch} with a link to the lucky search engine
with the specified term as its argument.  The lucky search engine is
just the standard search tool which selects the first search hit and
returns a redirect resource to take you to it.  The net effect of this
is that we have hypertext linkage capability within the system that is
entirely approximate! But, if the approximate identifier (search term),
is sufficiently well defined, it's as good as a definitive hard-coded link!

As you can see, using approximate identifiers, context and redirection
leads to incredibly robust long-lived systems. (Dare I say "loosely
coupled" - hell no, they're "decoupled").

Long story, but hopefully yet more grist to the mill of what we mean
when we say that identifiers are relative.

Now something for you to worry about: if you extrapolate the
philosophical implications, you'll quickly realise that relative
identity leads to an infinite contextual regression.  (I said its weird).

Now to stop worrying: While this is true, its no concern as the basis
for a real system since you can always introduce constraint boundaries
that "absolutize" the context (ie isolate it into a bounded set - that's
what DNS and the Web does. We don't get hung up on the parallel intranet
address spaces).  The beauty of NK and ROC is that unlike the web, you
can place the constraint where you want and allow address spaces and
context to flexibly and dynamically converge.

Next week, I'll show you some patterns with ROC (using NKP) that show
how the context established by a client associated with two
(independent) downstream NK servers allows each party to acquire the
aggregate address space of all three.  Makes pubsub look positively
primitive. (BJ are you watching ;-).
---

Have a great weekend!

The 1060 Team

---
You can unsubscribe from this mailing list here: https://cs.1060research.com/csp/settings/