Greetings Randy and Tony,

Thanks for taking the time to track down the problem. I have done the same in DPML and that worked, so I was stumped for quite some time there. Hopefully you will advise me how best to tackle my problem then (without having to resort to actually managing my files myself).

I'm trying to build a resource tree. The database is very simple, one record which contains the Bill of Material : ASSETBOM (shortdesc_owner, shortdesc_member);

I enter my program with (obviously) a shortdesc_owner. In one run of the program I read all the shortdesc_members and in a loop start the program recursively for all of them (going down the tree). I've got the recursion covered. What I don't have covered is that I need to keep all the results. Let me give an example output :
Level   Shortdesc
0       SVLIPC16
1       SVLIPC16_BOOTDISK
1       SVLIPC16_CPU
1       SVLIPC16_NETWORK
2       SVLIPC16_NETW0
2       XW010101_P001
1       SVLIPC16_SAN
(and then think some nice XSLT-transformation for the eye-candy).
Now, since I have recursion covered, those levelnumbers are easy. I just pass a param down the line, increase /nvp/level at each pass. Piece of cake. For my data I thought to be smart as well and pass the URI as /nvp/uri. In DPML this worked great (but relatively slow). In Python ... I ran into the optimisation problem (but with "ordinary" files I outperform DPML 3/1).

So Tony, while I do agree that this may not occur often in a real application, it does occur in mine ;-).

Any suggestions ?
Regards,
Tom

Hi Tom,
I've looked into the test case you have and confirm that it is working as expected (at least by NetKernel if not you ;-)

What is happening is that NKF API contains an optimisation to not re-request when it already has something and the response hasn't expired. So in this situation what happens is that the second source request isn't performed. (this explains why the visualizer doesn't show the request) Even if it was performed it would be plucked from the cache and the same result would ensue.

I suspect the reason why the response from the source isn't expired is because ffcpl module resources use a poll period to avoid excessive file IO that drains performance. This poll period can be changed on the "Kernel Parameter Manager" page on the control panel. However it isn't good to rely upon this because it is entirely possible that the granularity of the filesystem timestamp isn't high-enough to detect the change anyway.

Sorry this has tripped you up though I suspect this kind of behavior wouldn't happen much in a real application.

Cheers,
Tony

Tom,

Thank you for the program!

I can confirm that only that first SOURCE request is showing up in the visualizer on my system. I changed the program a bit to see if some initial ideas about the problem would resolve it and they do not.

We will now take a look at the issue and get back to you promptly.

Thanks!

-- Randy

Greetings Randy,

Here it is :

import sys import time sys.add_package("org.ten60.netkernel.layer1.representation") sys.add_package("org.ten60.netkernel.xml.representation") sys.add_package("org.ten60.netkernel.xml.xda") sys.add_package("org.ten60.netkernel.xml.util") sys.add_package("org.ten60.netkernel.layer1.nkf") sys.add_package("org.ten60.rdbms.representation") from org.ten60.netkernel.layer1.representation import StringAspect    ####################################################### temp_uri="ffcpl:/tmp/test_randy" # Write initial output outputs="BEGIN" context.sinkAspect(temp_uri,StringAspect(outputs)) ####################################################### output=context.sourceAspect(temp_uri,StringAspect) outputs=str(output.getString()) outputs+="\nTHIS LINE GOES LOST" context.sinkAspect(temp_uri,StringAspect(outputs)) ####################################################### output=context.sourceAspect(temp_uri,StringAspect) outputs=str(output.getString()) outputs+="\nEND" context.sinkAspect(temp_uri,StringAspect(outputs)) ####################################################### resp=context.createResponseFrom(StringAspect('RETURN')) resp.setMimeType("text/xml") context.setResponse(resp)

And my output in /tmp/test_randy :
BEGIN
END

Regards,
Tom

Thanks Peter.

I'd like to point out that the Preview 1 .jar file is exactly the same one that was distributed on the key at the Architect's Weekend (the source code for the lib modules is new).

So to install from this .jar file, follow the instructions from the readme.txt file (on the key only), which are as follows:

1060 NetKernel v4.0 Preview 1 ----------------------------- The NetKernel preview is packaged in an executable jar. NetKernel is pre-built to run from the jar - like a Linux-Live-CD. There's an installer tool that allows NetKernel to install itself to a local disk. System Requirements: -------------------- NetKernel 4 requires Java 5 or higher. To view the SVG diagrams in the documentation you will need an XHTML/SVG capable browser such as Firefox or Opera. Boot ---- To boot NetKernel: java -jar 1060-NetKernel-4.0.0-preview-1.jar and point your browser at http://localhost:1060 Install ------- To install NetKernel, when booted, point your browser at http://localhost:1060/installer/ Follow the prompts to tell NetKernel where to install itself.

enjoy,
-tom

Tom,

I am not a python programmer, however, I am now setup with a debug module in which I want to replicate your problem with sink/source.

Would you mind posting a complete, short python script that replicates the problem with some simple strings? I'll use that in my module and see if I can figure out what is going on.

Thanks -- Randy[/i]

I just sent email with details of where to get the limited access preview. If you didn't get it (or I screwed up the mailing list ;-) please drop me a note and I'll point you to it.

Cheers and enjoy.

Peter

Greetings Randy, Tony,

Its indeed a jdbc problem. It does work in HSQLDB. So I'm considering it a limitation of sqlite (you can't have everything in so small an engine).
Thanks for the suggestions that put me in the right track ... moving on to the other fish I got to fry with this application ;-)

Regards,
Tom

P.S. Randy, if you still want to have a look at the code your welcome to it (but I presume there's plenty of problems waiting at the moment). Its the same code I have the sink/source problem with.

Hi Tom,

I'd say it looks like your jdbc driver isn't threadsafe and you are hitting it with concurrent requests.

A solution might be to look for options on it's configuration or look at limiting the pool size for connections to 1.

If none of that works maybe you could create a data layer in your application to do database ops and place a throttle over it.

Cheers,
Tony

Tom,

The exception report seems to be coming from your database, if so my comments may help...

In your code snippet you are iterating on the results of a query. Since the DOM has already been created there should be no active connection to the database for that query. So the exception can't be caused by that code.

I'll presume that the code "impact_tree.py" is connecting to your database also. Since you are calling the code asynchronously you may be getting the database into a state it doesn't like. I don't know SQLLite. However I do know that some databases require column access in a certain order if BLOB or CLOB columns are involved. (The columns must be accessed in the order they are mentioned in the SQL statement - this has to do with buffering, etc in JDBC and the driver).

If you can provide the code (or a summary of it) from "impact_tree.py" maybe I can be of further assistance.

Randy