Overall I think NK is a brilliant design, but I have a nagging doubt about one aspect I want to get off my chest.  Or maybe someone can clarify and put my mind at ease.

The issue is that when a module A imports a module B, and then issues an active request that B handles, the argument resource addresses are first looked up in B's (private) address space, before ascending up the address space stack if not found in B to be evaluated in A.  It's this search of B's private namespace *before* A's that seems wrong to me.  After all the request came from A,  who knows nothing of B's *private* address space - so if a resource existed at the same address in both, I'd assume it was the one in A that was intended (where the request came from), and any matching in B would probably be accidental. 

As an example, see
   http://docs.1060.org/docs/3.3.0/book/gettingstarted/doc_intro_concept_context.html
and consider the discussion about where argument ffcpl:/fn.bsh is evaluated.  The possibility of accidental matching this resource in B might be clearer if the resource were ffcpl:/etc/ConfigRDBMS.xml instead.   Then e.g. in module A I may be intending to do some processing involving my ffcpl:/etc/ConfigRDBMS.xml and ask for B's help - if B also uses a database *privately* for its *own* purposes (say just to store it's own configuration preferences), then the ffcpl:/etc/ConfigRDMBS.xml would accidently match B's, not A's, contrary to A's intent - and which would be impossible for A to know of in advance without examining the internals of module B.

I really like the opportunity for lazy evaluation that having B decide how and whether to evaluate the args provides.  I just think the stack of address spaces that the arguments are evaluated in should be inverted.   Thoughts?

Thanks for the kind words.  You definitely have a good grasp of the address space relationships in NK3.   You are quite correct that if module-A goes down into module-B, and supplies pass by reference arguments, then the service in B should take care not to locate its internal resources in a space that is likely to be used by an application.  (If the arguments are pass-by-value then this is not an issue).  Incidentally we call this Push-State-Transfer.

Your example of the RDBMS tools looking for a standard resource ffcpl:/etc/ConfigRDBMS.xml is actually a case of Pull-State-Transfer - in which the resource state to be used is implicit and is requested from the tool if no push state-transfer is specified (eg in RDBMS an argument configuration@[myconfigresource] ).

This is a really old pattern that is used a lot in Unix.  The RDBMS have a Push-Default-Pull model.

Masking of resource/overriding of address spaces, is actually not really different to any other language - for example Java requires you to put your classes in a package using the inverse domain name convention, but depending on the order of your classloaders you can mask one class with another. This is actually a really nice technique to use to integrate a library into a NK module - since the module classloader will load classes from the physical module ahead of any classes found in jars in the lib/ directory - so you can seamlessly mask a class in a jar file for a local modified version (take a look at any language module and you'll see this used).

Unix-based languages (like Python etc) have the concept of environment PATH which is a list of direcotries that is progressively searched to load a class.  Again masking is not ruled out and requires care in the choice of address space.

So the rule of thumb is that if your module B is a 'library' - ie its called  as a set of services from one or more application level modules then its a good idea to put its internal resources in an unambiguous path.  We tend to use the Java inverse domain convention.

If you're using an existing library service that uses a default pull then either a) give it an explicit PushST argument located in your unambiguous local resource space or, if it really does want to use the configuration resource provided in the calling application's modules address space, make sure you are not matching that path in your own internal address space - otherwise you risk masking it.

So that's the technical details but the more interesting thing to discuss is the philosophical reason for the NK superstack.  In any language that we're aware of, the capability of a piece of code to call other code is defined in a downwards relationship - declared by import statements.  So for example

import java.util.Random;
r= new Random().nextInt();

For these languages, the code and its use of the function are locally bound by the import statement.

In NetKernel there is no physical binding between code - every resource that is requested is resolved and dynamically bound to physical code on-demand, just-in-time, each and every time.  Also philosophically, although we often think in terms of calling a 'service' (eg active:sqlQuery... or active:beanshell+.. which feels analogous to calling a software method) we are actually asking for a resource representation. (The name of the resource just happens to explicitly resolve to a binding to a physical piece  of code that can fulfill the resource - here these are the sqlQuery or beanshell accessors).

So now for a bit of magic.   Any physical piece of code that makes sub-requests for further resources does not have to have any physical import relationship with that resource. For example:

DPML does not have any concept of import. You don't need to do <import>active:xslt</import> in order to be able to perform an xslt transform.  Equally beanshell can make a request for a random number resource (if there was such an accessor in your app) needs no import ...

req=context.createSubRequest("active:random");
r=context.issueSubRequest(req);

All resources are contextually defined by the address space relationship which is extrinsic and dynamic.

The net result is that engines that process resources do not need to know anything about how the resources are physically fulfilled or even where they are located.  DPML can use every accessor in the system but knows about none of them.  The Unit test engine, the documentation engines etc can be given references to resources and operate on them in a uniform local way without any knowledge about there location - other than that they are in context via the dynamic superstack.

Needless to say the superstack is something that makes software in NK extremely flexible and tolerant to change.

However it goes further.  Once you start to get a feel for this stuff there are whole worlds of new patterns you can use.  As I talked about earlier, push/pull state transfer is just one example.  You can dynamically remap requests based on policy (eg the gatekeeper pattern and impl).  You can use dynamically generated config - for example you can very easily map your database config to an accessor that uses a database to store multiple database configurations like this...

<rewrite>
<match>ffcpl:/etc/ConfigRDBMS.xml</match>
<to>active:dynamicDBConfig+id@someApplication</to>
<rewrite>

where active:dynamicDBConfig would be implemented on top of active:sqlQuery with a specified configuration@[...] to point to the config database.  (You could do this deeper and deeper but you'd lose your mind!).

In use the active:sqlQuery tool won't know or care that its actually just made a secondary DB call to get its own config state. And it'll only happen once on first use and thereafter the contextual caching will mean it never gets invoked again.

Finally.  There is actually yet another bigger picture here.  The relationship between resources and context.  Unix made a first solid step, NK3 takes it further but there is a lot lot more.  That's what we've been working on for the last 3 years and which we will soon be ready to reveal in NK4.  Come along to the NK architects weekend in September - its going to be a lot of fun.

Hope this helps.

Peter

Peter,
thanks for the explanation. It's a great system and makes for modular solutions.  I wouldn't dream of suggesting any big changes to a system that works so well already.  I've been thinking a bit more, and I think there may be a pretty easy solution to the accidental uri collisions problem that doesn't require any fundamental changes at all.

So: would it be possible to have some sort of labeling of arguments (say with @! instead of @) that, in the module that first initiates the request, would automatically be rewritten to a unique and opaque uri that resolves to the same resource within the modules private address space? 

E.g. If I want to store *my* module's db config file in the database, while suspecting that the submodule I'm delegating to for doing the storage probably has its own, then I'd just write:

   active:store-file-to-db+theFileToStore@!ffcpl:/etc/ConfigRDMBS.xml

(note the @! instead of just @).

This would be equivalent to

    1)  Sinking the ffcpl:/etc/ConfigRDMBS.xml uri (itself not the resource it indicates) to a new unique and opaque uri in the private address space of the module:
               SINK "ffcpl:/etc/ConfigRDBMS.xml" -> ffcpl:/urn:org:mymodule:myapp:42

     2)  Rewriting the !ffcpl:/etc/ConfigRDBMS.xml argument in the request with (somthing like):
               curi:ffcpl:/urn:org:mymodule:myapp:42

     3)  Implementing (something like) the curi indirection as already supported in dpml

This would allow the requesting module to indicate that it means the resource indentified by the uri within *its own* address space, and would make accidental uri collisions in submodules it delegates to impossible.  As an added benefit, it also hides the original uri string from submodules in case parts of the uri itself are sensitive information.

I'm suspecting something like this may already be pretty easy. I think it would help a lot if it were just _dead_ easy though (like one extra character after the @), because I'd probably use it a lot.

Hopefully that makes sense...