(updated, fixed an incomplete sentence, stubbed out better results from PowerPointUnit).

Jorgen Thielen, one of Savas' colleagues, argues that WS-* interop plugfests ensure that WS-* standards work better than Corba., indeed, that Workshops apply software testing disciplines to the way Web services specifications are developed, and that implement-standardise-interop is a better process.

Clearly I am mistaken, and the "WS" protocols that he is working on must be a different set from "Web Services". Because it is blatantly obvious that they don't apply test-driven development to those specifications, and it now seems inevitable that they are digging an even deeper hole, the more complex the specs get.

My proposed Apachecon lightning talk was going to be "Testing for Architects", in which after haranguing the members of the WSDL-1.0, 2.0 and WSA spec teams in the audience (Sanjiva, Dims and Glenn), I was going to argue that if we could not bring the architects to the test tools, we could bring the tools to the architects with PowerPointUnit being a test framework combining powerpoint macros and the truth-resolution feature of the semantic web, one that assuming that all non-resolvable statements are false. (i.e. we are working with Horn clauses). The test framework would evaluate the setUp and tearDown slides first, then run through the presentation parsing and evaluating all statements in text boxes given an label beginning with the word "test". Not having got this working on the short BRS-Dublin flight (Ryanair doesnt have room to unfold a laptop), I will have to do the work by hand, with some suitable text.

And what do I have to hand but a little note in my soapbuilders inbox, sent on June 30 from one Jorgen Thelin from a microsoft.com email address. Jorgen, I'm afraid my prototype RDF-truth-resolution tool cannot resolve the statement

for either X=corba or X=ws. Let's look at some of the error results of the email-analsis.

Microsoft Corporation would like to invite you to a 3-day Web Services Interop Plug-fest on Tuesday September 26, 2006 to Thursday September 28, 2006 from 09:30am to 5pm each day.

[PPTUnit] PASS

In terms of barrier to entry, the cost of a trip to SEA, the commute round I405 and then hanging round building whatever is probably the same as attending a standards meeting. Not worth participating, unless you have money to burn, and a high perceived value in attending, or you live in sight of Mount Ranier and the cost of attending is just the commute out to Redmond. But if you are in a standards meet, you have some chance of influencing the outcome and improving the process.

The Interop Plug-fest is an ad-hoc, open forum for companies who have implementations of various Web Services protocols to meet with engineers from the Connected Systems Group and test against the upcoming version of Microsoft's Windows Communication Foundation (formerly known as codename Indigo) product.

[PPTUnit] FAIL. Could not resolve has_tests(ws). Warning: process may not be test-driven.

Test against indigo. Which tests precisely? The tests that came with the specs? Oh, wait a minute, there are no such tests.

The focus of these plug-fest meetings is on interop testing, so no feedback will be taken on the WS-* specifications themselves.

[PPTUnit] FAIL. Expected process(ws,"agile"), found process(ws,"slowly iterative waterfall")

That is, "we won't listen if you say WSA doesn't make sense, it has taken something as simple as a URL and replaced it with 2003/03, 2004/05 and 2005/08 interim and 2005/08 drafts, all of which you can encounter in the field". Instead we are meant to be greatful for messages like this bunny, currently sitting at the top of my live endpoint tracelog (there's a link to it in the testing slides, go read them)

<Envelope
  xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns="http://schemas.xmlsoap.org/soap/envelope/"> 
<Header
 xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing"> 
<To soap:mustUnderstand="1"
  xmlns="http://schemas.xmlsoap.org/ws/2003/03/addressing"
   >http://deployapi.iseran.com:8080/alpine/system/?system=uuid_6ba4318b_23ce_46d7_ae38_b2500b443dcf</To>
<MessageID soap:mustUnderstand="1"
  xmlns="http://schemas.xmlsoap.org/ws/2003/03/addressing"
  >uuid:c0e61c77-ec5e-4d4b-89f6-2360fcdfe7d2-0</MessageID>
<Action soap:mustUnderstand="1"
  xmlns="http://schemas.xmlsoap.org/ws/2003/03/addressing"
  >http://docs.oasis-open.org/wsrf/2004/06/wsrf-WS-ResourceLifetime-1.2-draft-01.xsd/Destroy</Action>
</Header> 
 <Body> 
 <Destroy
  xmlns="http://docs.oasis-open.org/wsrf/2004/06/wsrf-WS-ResourceLifetime-1.2-draft-01.xsd"/>
</Body> 
</Envelope>

Admittedly, MS won't be expecting WSRF operations, but messages like this, in which most of the payload is WS-A headers and needlessly repeated xmlns declarations will undoubtedly be floating around. And feedback like "what is the point, just tell me?" is clearly not allowed.

This plug-fest event is open to anyone who desires to participate and who can bring an implementation to test interoperability with Microsoft's Windows Communication Foundation product any or all of the following areas: )

(listing including SOAP1.1, SOAP1.2, WS-Addressing 2004/08, MTOM, WS Secure Conversation 2005/02, WS-Trust 2005/02,WS-RM 2005/02, and many others

[PPTUnit] FAIL. Expected interoperable(P,P) where P=[ibm,sun,jboss,bea]; found interoperable(P,ms).

Interoperability with Indigo. Not interop between, say, Axis2 and the JBoss stack. Or between XFire and Alpine. No, those are our problems.

This shows what is wrong with the process. With no formal, machine readable and yet normative definition of the semantics of the communication other than XSD or WSDL, there is no way to prove the correctness of an implementation. You can write tests, but since there is no way to prove your tests match the spec, well, your tests may be wrong too. If there were normative tests included in the spec, then the test suite itself would be part of the official definition of system behaviour. There would be no real need for interop testing plugfests against a single vendor's implementation, because everyone would know that the test suite worked against their endpoints, and therefore that their impls matched the specs. The interop events would just be demos of this, and to catch little surprises like MTU inconsistencies and other stuff that kills performance.

The fact that instead WS-* implementors are reduced to flying to seattle, just to get their stuff working with MS and not with each other, shows that the process for achieving workable, interoperable implementations is not only crippled by the focus on O/X mapping for datatypes, but in the way that specs get written without thinking about testing or proofs of correctness.

I shall not be attending BTW, no snow there at that time of year. I managed to spend some time showing Alpine to the Axis2 folk and Dan Diephouse, author of XFire. Dan and I did some code walkthroughs, swapping design ideas and improving both our stacks through peer code review, even though neither project is under Apache. Because ApacheCon doesnt care what project you code for, or whom you test against, as long as you have fun and do useful things at the conference. WCF interop plugfests seem to have a different agenda, like declaring victory from having so many stack vendors demo interop with indigo.

NB, for anyone who does go, stay in the Silver Cloud just south of the campus and demand the MS rate. Free LAN; some restaurants in walking range.