|
 python filehandling sinkAspect sourceAspect newbie question |
Joined: 22-April-2007 Posts: 79 Location: Belgium | Posted:
30-September-2008 08:55 Greetings, It's probably a newbie question, but why would this work : # Write initial output
if level==0:
outputs="{\r\n identifier: 'shortdesc',\r\n label: 'shortdesc',\r\n items: ["
context.sinkAspect(temp_uri,StringAspect(outputs))
#
output=context.sourceAspect(temp_uri,StringAspect)
outputs=str(output.getString())
outputs+= "\r\n { shortdesc:'"
outputs+= description
outputs+= "', numberOfItems:0,"
context.sinkAspect(temp_uri,StringAspect(outputs)) |
And would this not work : # Write initial output
if level==0:
outputs="{\r\n identifier: 'shortdesc',\r\n label: 'shortdesc',\r\n items: ["
context.sinkAspect(temp_uri,StringAspect(outputs))
#
# output from this block goes lost
output=context.sourceAspect(temp_uri,StringAspect)
outputs=str(output.getString())
outputs+= "\r\n { shortdesc:'"
outputs+= description
outputs+= "', numberOfItems:0,"
context.sinkAspect(temp_uri,StringAspect(outputs))
#
# output from this block goes lost
output=context.sourceAspect(temp_uri,StringAspect)
outputs=str(output.getString())
outputs+= "\r\n }"
context.sinkAspect(temp_uri,StringAspect(outputs))
# Write final output (this output I get)
if level==0:
output=context.sourceAspect(temp_uri,StringAspect)
outputs=str(output.getString())
outputs+= "\r\n ]\r\n}"
context.sinkAspect(temp_uri,StringAspect(outputs)) |
These are just preliminary blocks, so yes, I know that its not the most efficient code possible. I basically want to do multiple writes (sinks ?) to the same fileresource at different stages in my code. But I seem to run into some sort of caching (?) problem. Can somebody explain how I should best do this ? Regards, Tom | | request visualizer |
Joined: 22-April-2007 Posts: 79 Location: Belgium | Posted:
30-September-2008 10:13 Greetings,
-- Additional information
I used the Request Visualizer on this code ...
All the "sinks" are there ... but only the first "source" is listed ...
So my "result" is logical ... but I still don't understand why the other "sourceAspect" requests are not done ...
Regards,
Tom | | Visualizer |
Joined: 15-February-2005 Posts: 127 Location: Fort Collins, CO | Posted:
30-September-2008 13:39 Tom,
We are just now getting back from the Architects' Weekend and it may take us a little while to unpack and setup in our offices before we can answer your question.
While you are waiting I'll ask this quick question - have you viewed this request processing in the visualizer? You might want to do so to confirm that all sub-requests are being issued, the arguments are as you think they should be, etc.
-- Randy
| | well ... |
Joined: 22-April-2007 Posts: 79 Location: Belgium | Posted:
30-September-2008 14:30 Randy, No worries. Take your time to unpack ... If I put only one source-request (I tried both source and sourceAspect) in my code it works as I expect : if level==0:
outputs="{\r\n identifier: 'shortdesc',\r\n label: 'shortdesc',\r\n items: ["
context.sinkAspect(temp_uri,StringAspect(outputs))
# So this first source request works
resp=context.source(temp_uri,StringAspect)
output=resp.getAspect(StringAspect)
outputs=str(output.getString())
outputs+= "\r\n { shortdesc:'"
outputs+= description
outputs+= "', numberOfItems:0,"
context.sinkAspect(temp_uri,StringAspect(outputs))
#
# And here I now comment the source request ...
#resp=context.source(temp_uri,StringAspect)
#output=resp.getAspect(StringAspect)
#outputs=str(output.getString())
outputs+="\r\n }"
context.sinkAspect(temp_uri,StringAspect(outputs))
#
# Write final output and again I comment out the source request
if level==0:
#resp=context.source(temp_uri,StringAspect)
#output=resp.getAspect(StringAspect)
#outputs=str(output.getString())
outputs+= "\r\n ]\r\n}"
context.sinkAspect(temp_uri,StringAspect(outputs))
context.sinkAspect(save_uri,StringAspect(outputs)) |
This works perfectly. However. You'll notice that I've still got way too many sinks. This is because this is just skeleton code. In between the two main requests I want to do a lot more processing, amongst other things some asynchronous subrequests that will in turn access the same uri. So, before each sink I want to make sure I have the latest source (I'm going to add locks and unlocks too). But beyond the first source request none of the others (when I uncomment them) shows up in the visualizer. Before you show me the error of my ways, let me explain what I want to accomplish. I'm building up a recursive tree (so my script will call itself recursively). I've managed this perfectly in DPML where I build up the tree in a temporary file. I don't have management of that file, curi takes care of that perfectly (and I pass the uri in the param of each recursive request). Now, I want to speed-up my code by using Python (am thinking of asynchronous requests handled by a table). But now the filehandling seems to be my problem ... Long story. Like I said, take your time, in the mean time I'm learning lots by going to all the netkernel code and trying to find similar stuff. Regards, Tom | | read write works ... source sink does not |
Joined: 22-April-2007 Posts: 79 Location: Belgium | Posted:
2-October-2008 05:34 Greetings,
I replaced the source/sinks with "normal" python file access ... and that works. However I don't like it. I've got to specify where my files go, need specific routines for copying and deleting ... Somehow this does not feel like what I've gotten used to when using Netkernel.
So ... still looking for a solution to my "source"-problem ...
Regards,
Tom | | Python program |
Joined: 15-February-2005 Posts: 127 Location: Fort Collins, CO | Posted:
2-October-2008 15:57 Tom,
I am not a python programmer, however, I am now setup with a debug module in which I want to replicate your problem with sink/source.
Would you mind posting a complete, short python script that replicates the problem with some simple strings? I'll use that in my module and see if I can figure out what is going on.
Thanks -- Randy[/i] | | the source |
Joined: 22-April-2007 Posts: 79 Location: Belgium | Posted:
3-October-2008 06:35 Greetings Randy, Here it is : import sys
import time
sys.add_package("org.ten60.netkernel.layer1.representation")
sys.add_package("org.ten60.netkernel.xml.representation")
sys.add_package("org.ten60.netkernel.xml.xda")
sys.add_package("org.ten60.netkernel.xml.util")
sys.add_package("org.ten60.netkernel.layer1.nkf")
sys.add_package("org.ten60.rdbms.representation")
from org.ten60.netkernel.layer1.representation import StringAspect
#######################################################
temp_uri="ffcpl:/tmp/test_randy"
# Write initial output
outputs="BEGIN"
context.sinkAspect(temp_uri,StringAspect(outputs))
#######################################################
output=context.sourceAspect(temp_uri,StringAspect)
outputs=str(output.getString())
outputs+="\nTHIS LINE GOES LOST"
context.sinkAspect(temp_uri,StringAspect(outputs))
#######################################################
output=context.sourceAspect(temp_uri,StringAspect)
outputs=str(output.getString())
outputs+="\nEND"
context.sinkAspect(temp_uri,StringAspect(outputs))
#######################################################
resp=context.createResponseFrom(StringAspect('RETURN'))
resp.setMimeType("text/xml")
context.setResponse(resp) |
And my output in /tmp/test_randy : BEGIN END Regards, Tom | | Confirmed behavior |
Joined: 15-February-2005 Posts: 127 Location: Fort Collins, CO | Posted:
3-October-2008 14:35 Tom,
Thank you for the program!
I can confirm that only that first SOURCE request is showing up in the visualizer on my system. I changed the program a bit to see if some initial ideas about the problem would resolve it and they do not.
We will now take a look at the issue and get back to you promptly.
Thanks!
-- Randy | | The cause | 
Joined: 7-February-2005 Posts: 249 Location: Uncharted territory | Posted:
3-October-2008 16:19 Hi Tom,
I've looked into the test case you have and confirm that it is working as expected (at least by NetKernel if not you ;-)
What is happening is that NKF API contains an optimisation to not re-request when it already has something and the response hasn't expired. So in this situation what happens is that the second source request isn't performed. (this explains why the visualizer doesn't show the request) Even if it was performed it would be plucked from the cache and the same result would ensue.
I suspect the reason why the response from the source isn't expired is because ffcpl module resources use a poll period to avoid excessive file IO that drains performance. This poll period can be changed on the "Kernel Parameter Manager" page on the control panel. However it isn't good to rely upon this because it is entirely possible that the granularity of the filesystem timestamp isn't high-enough to detect the change anyway.
Sorry this has tripped you up though I suspect this kind of behavior wouldn't happen much in a real application.
Cheers,
Tony | | ok ... so how do I ... |
Joined: 22-April-2007 Posts: 79 Location: Belgium | Posted:
6-October-2008 06:38 Greetings Randy and Tony,
Thanks for taking the time to track down the problem. I have done the same in DPML and that worked, so I was stumped for quite some time there. Hopefully you will advise me how best to tackle my problem then (without having to resort to actually managing my files myself).
I'm trying to build a resource tree. The database is very simple, one record which contains the Bill of Material : ASSETBOM (shortdesc_owner, shortdesc_member);
I enter my program with (obviously) a shortdesc_owner. In one run of the program I read all the shortdesc_members and in a loop start the program recursively for all of them (going down the tree). I've got the recursion covered. What I don't have covered is that I need to keep all the results. Let me give an example output :
Level Shortdesc
0 SVLIPC16
1 SVLIPC16_BOOTDISK
1 SVLIPC16_CPU
1 SVLIPC16_NETWORK
2 SVLIPC16_NETW0
2 XW010101_P001
1 SVLIPC16_SAN
(and then think some nice XSLT-transformation for the eye-candy).
Now, since I have recursion covered, those levelnumbers are easy. I just pass a param down the line, increase /nvp/level at each pass. Piece of cake. For my data I thought to be smart as well and pass the URI as /nvp/uri. In DPML this worked great (but relatively slow). In Python ... I ran into the optimisation problem (but with "ordinary" files I outperform DPML 3/1).
So Tony, while I do agree that this may not occur often in a real application, it does occur in mine ;-).
Any suggestions ?
Regards,
Tom |
|
|
|
