[First sent to mailing list subscribers on April 23rd 2010.  To subscribe for early access join NetKernel portal https://cs.1060research.com/csp/]

What''s new this week?  Core infrastructure tweaks. Exceptions are
resources too. XRL tutorial. Thoughts on linked data runtimes.

Repository Updates (NKSE / NKEE 4.1.0)
--------------------------------------

Prompted by the NetKernel protocol development work, some minor tweaks
have been made to the core infrastructure.  These don''t affect the
user-side but were needed to ensure NKP can cleanly map the NK abstraction.

kernel: Optimized expiry accumulation.
layer0:  INKFRequestReadOnly now has getHeaders() method.
standard-module:  StandardTransportImpl now does not expose an arbitrary
logical endpoint.

Other updates...

xml-core:  DOMXDA transreptor updated to make SAX parse errors cacheable
(see discussion below).

nkse-doc-content:  Added links for xrl-tutorial installation.

Exceptions are resources too, Caching Exceptions
------------------------------------------------

Development of NKP is getting close to a public release (hopefully next
week).  We''ve been working through the many different error handling
scenarios and ensuring that when they happen the connection and ROC
abstraction ''do the right thing''.  One design intent is that the NKP
connection should handle errors and also, when the error condition is
put right, it should transparently reconnect.

One such error scenario is a badly formed configuration resource. ie an
XML config that will not parse.  So consider a request arriving at the
NKP client, if it is not yet connected to the server it must obtain its
configuration, establish a connection and issue the request to the
downstream side.

But, as I covered last week, the client endpoint is locally responsible
for resolution requests for the remote logical endpoints.  In this error
scenario, with no configuration, you can never connect. But our
requirement is to have the client automatically connect when errors are
fixed.  You can see that there is the potential for (hundreds) of
repeated attempts to parse the broken configuration resource - which
would be a huge waste.

Now you might think, ok, put state in the client endpoint and make it
back-off on config errors.  But that introduces hideous and escalating
state management/thread safety complexity in the endpoint. It also
misses a very powerful trick...

We say it over and over but maybe its implications aren''t easily seen:
Everything in the NK/ROC abstraction is a resource.

For example, we see that requests for resources from accessors are
obviously resources. But so too are Requests (think of the pluggable
overlay, the mapper configuration, the declarative request, f(g(x))
withRequest), Spaces are resources too (look at dynamic import
configuration).  Even, as I discussed last week in the comparison of NKP
with HTTP, the Metadata that provides the structure to the system is a
resource.  So the NK ROC abstraction, to first and second order, is
self-consistent.

Given the title of this section, you''ll not be shocked when I also
reveal that Exceptions are resources too. It might not feel like it
since the NKF API is designed to make it feel like exception throwing
and catching is innate to the language.   But did you ever wonder how
it''s possible that in NK we are able to throw exceptions in an endpoint
written in one language and catch them in a requesting endpoint written
in another?

The answer is that an Exception is not special. Its a resource
representation. You should think of an Exception as being that
representation you obtain when the contextual state that is available to
the requested endpoint results in an error.

So failing to parse badly formed configuration yields an Exception
resource representation.  Until someone goes and actually changes the
configuration (by editing it, or changing the code that dynamically
generates it), you''ll always get the same representation every time you
request it.

As you''re becoming familiar with ROC, you''ll immediately see the
following logical consequence...

NetKernel caches all requested resources representations.
Exception are resource representations.
NetKernel caches Exceptions.
NetKernel has a dependency model.  The Exception resource can come from
cache for as long as its dependents are not changed.

Therefore the computational cost of computing a resource that results in
an error can be paid once, for all time, until the problem is fixed.
This means hard-core production time error conditions do not impact the
overall system performance.

Now, everything I''ve just said is true.  Only its been turned round to
show you the potential when you think of Exceptions as resources.

The problem with errors is that they are usually very significant when
they occur at development time.  So caching exceptions during
development would make debugging a first class ticket to insanity.

For this reason, we''ve made it the default in all endpoints that
exception representations are expired (not cacheable).  Therefore, out
of the box, any endpoint you create will behave just like what you
expect and are familiar with in your chosen language.

But, if you''re writing an endpoint and you know that your error
condition will recur (ie like classical memoization it has referential
integrity and only depends on the input state). Then you can very simply
change the default and make the exception cacheable.

There''s a detailed guide to the NKF API''s for dealing with Exception
caching here...

http://docs.netkernel.org/book/view/book:guide:physicalreference/doc:physicalreference:exception

If you''ve followed this line of thought then this is something you can
introduce to your endpoints as an additional runtime optimization.  One
thing to beware of though. Languages don''t really have any
classification model for exceptions. Therefore you must be very aware if
an exception you are handling is generated by an "externality" you must
not cache it. For example, an IOException should never be cached since
it may be tat the external cause is transient, therefore for you have no
choice but to retry (externalities are not referentially transparent).

(Of course even with externalities, you can always add time or other
dependent expiry functions in order to implement back-off patterns.)

So now you know why we changed the DOMXDA parser.  It turns out that
with one line of code...

try
{ ...
}
catch (SAXParseException spe)
{   aContext.createResponseFrom(spe);
}

The transreptor now issues cacheable exceptions for badly formed XML
resources.  And now the whole stateful NKP client scenario disappears.
Bad config is sourced from cache and our NKP client''s configuration can
be completely dynamic.

[Incidentally NKP lets you throw exceptions between systems too - they''re
resource representations so it happens for free.]

XRL Tutorial,  Thoughts on linked data runtimes
-----------------------------------------------

Brian Sletten reminded me that in NK3 we had a tutorial that teaches
XRL.  This prompted me to translate it and freshen it up for NK4.  Its
now available as an interactive tutorial - just go to apposite, sync and
look for ''tutorial-xrl''.  Or search the docs for "XRL tutorial" after
you''ve got all today''s updates.

I''ve mentioned a few times recently that XRL (XML Recursion Language) is
a pretty powerful way to create composite resources by recursively
evaluating resource references - in essence it effectively constitutes a
linked data runtime.

Now I''m sure some people will get hung up on whether XML constitutes
linked data (I hear distant crys of "Its got to be RDF dummy!").  But
the truth be told, I couldn''t give a stuff what forms you choose for
representations*.

To the ROC abstraction, whatever your resource model, implementing a
linked data runtime boils down to identification and composition of
information resources.

At least, if you come from a Web/REST perspective, those are the two
eigen-vectors.  For example, a web browser is a linked data runtime.
When it receives an HTML page it dynamically sources the CSS, Image,
Script etc etc resources and forms a composite representation which is
rendered into the browser''s display.

I know I''ll get a reputation soon, but did I ever mention that REST is
not a complete model? The Web/REST is not complete because it makes a
presumption that there is a single uniform global context in which we
issue our resource requests.  The Web is a model in which a single
address space provides (implicit) resolution of point identifiers to
point resource representations (in the Maths of Set theory it is a
Bijection http://en.wikipedia.org/wiki/Bijection ).

Seems to me that the linked data community is aiming for something
similarly rich but for systems not requiring (quite so much) human
input.  Something like a simplified model of what we have when a human
and a web browser combine in the form of a stateful contextual endpoint.
   Notably, it is the human element that provides the context.

Now here''s something that Tony and I have been living with for the past
7 years.   Its a bit weird and certainly hard for the followers of the
doctrine of the absolute canonical *World Wide* Web to be comfortable
with. It turns out that identification of resources is relative.  Or put
another way, the names(symbols) that we use for things are relative to
context.

In your gut you know this already. Your cellphone probably has presets
for your Wife/Husband/Mother/Father etc. It has an address book in which
the short name is used instead of the full international telephone
number.  When you want to call someone you find their name and hit call.
  The local relative name is resolved to the telephone number, the
request issued to the local cell network, to the regional distribution
network, to the international trunk network, back up to the regional
network, to the cell and ping the request is resolved to the connected
call.  This is a simplified picture, but it shows an example of a
relative contextual resolution in an extended, non normalized address space.

I could go on (and on and on) with examples of relative addressing in
our everyday experience (see some of the ROC papers on the site for some
more http://www.1060research.com/netkernel/roc/ ).  But the point I''ve
been building to is that for linked data to work you have to recognise
the role of context.  Better yet, you have to be able to dynamically
*adapt* that context as you accumulate the state of the resource data too!

You''ll see in the first part of the XRL tutorial, that it doesn''t
actually talk at all about the mechanics of XRL recursive evaluation.
Instead it discusses how you can provide a context in which the XRL
evaluation yields the resources you require.

NetKernel generalizes the Web and REST.  It does it by making context a
a first order eigen-vector which you can control and dynamically adapt...

Gentlemen, start your runtimes.

[* To open a can of worms I know I''ll regret, my gut instinct tells me
that tree structures seem to have about the right "dimensionality" for
humans to cope with their creation and their consumption.  Libraries are
crammed with tree structured representations: we call ''em books.  A book
with an index is a tree containing its own inversion map - clearly also
a very efficient data structure for humans.  Machine efficient data
structures should always be transreptable to human efficient
representations (we prefer to code in Java and Groovy etc and not
assembler).]

---
Another long one to fall asleep over. Thanks to those of you that gave
me feedback last week - its nice to get encouragement and know that I''m
not just whistling in a vacuum.

Have a great weekend,

The 1060 Team