Steve: Developing on the Edge - MSDN on Continuous Integration

Steve: Developing on the Edge

Thoughts on development, Web-services, technology and mountains.

7Feb
Tue2006

MSDN on Continuous Integration

On the subject of building things, MSDN has an article on adding continous integration to Visual Studio Team Server

VSTS doesn't ship with such a thing built in; remember it still lives in a world where testers and architects have separate versions of Visual Studio. This paper shows how to easily fit the stuff in.

I gave the page 4/10 for trying. It isnt the software, its the process. Like this bit

BuildType:The build type to be used while doing continuous integration builds. Typically this is configured to just do builds. Optionally, you can also add some basic level of testing and static analysis.

That is: they think that running basic tests on a CI server is an optional extra. If you aren't going to delegate running your full test suite to a machine with time on its hands, that leaves (a) people or (b) nothing. People have better things to do, even testers, like writing better tests.

But what really hurt was this "disadvantage"

Some teams find the level of discipline required for continuous integration a bottleneck and would require a shift in the mindset.

Now remember, the paper has just encouraged your server to just build, not test. and now they say the level of discipline required for continuous integration is a bottleneck and would require a shift in the mindset.". Let's rephrase that. Some teams are happy checking in broken code, and are not prepared to fix this behaviour. That's why I gave it 4/10. Tech good; process bad. If the team is checking in broken code it means that you can never check out good code. Which means that you can never be sure in the morning whether or not the stuff in the SCM repo is any good or not, So you branch for weeks at a time, have integration hell at the end, no stable images in SCM, etc. etc. And you probably don't have test cases that are any good either.

So yes, CI does imply a change in mindset for those projects. But its either that or ship broken code, late.

I wonder if this MSDN article is going to crash and burn as badly as the one on test-driven development?

Posted by steve at 12:45

Comments

new comment thread


	On the MSDN article on CIreply to this thread On 7 February 2006 at 17: 00 Khushboo commented: Hey Steve Thanks for the feedback on the paper. You do make some interesting and very valid points. Not to defend but just to clarify the intention of each of the above mentioned points 1. Though it may sound like that - but definitely none of Team Build talks or presentations have 'discouraged' test integration. I am completely with you on this - running tests is a must for a disciplined build system - CI or no CI. If you have noticed and played with Team Build - it is because of this that we declare a build 'Failed' even if one of tests fail - without giving a choice to the users. All that this point mentions is that while configuring a build type - you can 'choose' not to run tests - there can be some scenarios where a customer would want this flexibility. 2) On the second point - the sub heading of the point mentions "Others find some disadvantages..." - just to clarify again this was clearly some points that people raise when they speak pros and cons of CI - not our views. But again thanks a ton for the feedback and the points you make. Thumbs up for both of them !! :) - I couldnt agree more.
	Re:MSDN on CIreply to this thread On 8 February 2006 at 12: 20 Anu commented: I dont understand how providing an option to run tests in the build process could mean that MSDN is "discouraging" you to run tests. You must keep in mind that there are several users who may not want to run tests as part of Build process for some reason or another. So, providing an option only means that you cater to both sets of the audience. Just goes to show that no matter what options you provide in any product, you can't please everyone! On 8 February 2006 at 13: 11 Steve Loughran commented: Anu, The point I was trying to make is that moving to test driven dev is the most profoundly valuable thing a project can do, far more important than deciding which toolchain to use for a build. Now, if the accompanying docs to go with the tools (such as this MSDN article) don't go out there way to emphasise that TDD matters, then people aren't going to get the real benefits of automation. Having an automated build without tests just checks that your stuff compiles; having an automated build with tests is an order of magnitude more valuable, as it keeps track of whether or not your program actually works. The other thing is that I believe that VSTS costs many dollars/pounds/euros/rupees/yen, whatever. It's a high-touch product to compete with Eclipse+Rational, targeted at enterprise projects. If those projects aren't going for testing, then they are wasting their software budget, and their engineering resources.