This is not another critique of any particular build tool, instead it's part of a long essay on where they will go in the future. I could maybe make a paper of it, though then I'd have to get some citations, do some prototyping, etc.

The first concept is this: build tools are created to fix the problems of the languages that are used, the platforms on which they run, and the applications that are developed.. More succintly: they are driven by user need.

The classic Make tool was written to get things building on Unix, C programs. Programs which include header files and need to be rebuilt when they change, binaries that need linking together.

IDEs came about in the nineties, when the focus was on drag-drop client guis: you'd design your GUI in a gui tool, hit the make button (Borland C++:F9 -still the key I use today), and have some windows .exe pumped out that you could deliver to your customers. No testing, just run and debug. The IDEs were fantastic at debugging, and once you learned enough x86 assember to know that xor eax,eax set the EAX register to zero, you could even step through the object code to find your pointer bugs.

When Java came out, there were two ways to build. In-IDE, and makefile. IDEs suffered from being pure java and slow, barely usable AWT things (Symantec Visual Cafe?) or nippy but non-portable native apps, of which Visual J++ is probably the ultimate example. It still had the old IDE mindset to the extent that you could bring up the compiled java bytecodes and step through it. That is, they took the old way of working, and applied it to a new problem: Java. There was still make of course, which was fiddly to use in Java because of its hierarchial package layout. It was also single platform. Gnu make+Unix, windows and NMAKE, etc. While single-IDE or single-platform development worked in small, in-house projects, it didn't cut it for open source dev, where one person may have a solaris box with make on, but the other developer is on a windows system, another on a version of debian-unstable they built themselves. There's is/was not enough unity of infrastructure to stabilise on tools, so James Duncan Davidson had to write one. Ant.

It certainly addressed the x-platform problem, but it had a more subtle effect -it let JUnit -which was written around the same time- integrate with the build. Instead of being a portable, command-line alternative to the IDE, it suddenly added something new: the ability to declare the rule that you don't ship until the tests pass.

Having a portable library for building stuff (under an integration-friendly license), had some interesting side effects. Firstly, IDEs picked it up. This lowers the cost of entry of a new IDE into the ecosystem, as it still runs existing builds. And, because the build files can run the tests and the packaging, they let IDE-hosted builds do everything a Java project needed to do. Secondly, integrated testing led the way towards Continuous Integration (CI) servers, which would build the code and run the tests. Without widespread testing, there's little value in CI. Its good, it detects signature changes between classes, but not incompatible semantics.

The best metric of Ant's success has to be that MSBuild is very similar; the foundation for building modern .NET projects would be immediately familiar to an Ant user. There are differences, and MSBuild is designed to integrate with IDEs better; there's more explicit listing of input and output files.

Of course, once you can build one Java app, you need to build more, and that's where both Maven has come into existence and Ant has accrued features. They both take different routes to minimising duplication of XML configuration files, and a somewhat similar approach to managing library dependencies: a big central repository of files and metadata, with the option for private versions. Now, one of the problems we have here is security of the artifacts and the quality of the metadata, but these are not faults in Maven per-se, as Ivy is at risk from them too.

The other change in Java land is Eclipse, which, for better or worse, is about as much an inevitability in a big project as unrealistic deadlines. IBM and the other Eclipse players are clearly trying to hard to make a tool that competes head to head with MS Visual Studio. But as a consequence, Eclipse actually removes some of the pressure for IDE-neutral, cross-platform build tools. Why bother with Ant or Maven when you can go back to hitting "build" or even "test" in the IDE? Possibly the main compelling reasons to stay with command line tools is to retain that command-line option, and for CI tools to build your app. However, I would expect some of the CI tools will already support Eclipse builds, and if not, soon will. IntelliJ's TeamCity integrates well with IntelliJ IDEA, as an example, where not only does it run off IDEA configuration files, it can feed back coverage and analysis data from the server-side runs into the IDE.

If Eclipse makes the IDE the centre of the build universe again, I'd also hope to see more in the way of integration with central artifact repositories. There should be no reason not for an IDE to offer you a list of libraries you can use, with the library binaries, source and javadocs pulled down on demand. That would be very slick: imagine stepping in to third-party Java code and having a classname lookup triggering a source file fetch on demand.

If we return back to the IDE, we are almost back in the early nineties for development, this time with testing and a bit of deployment in the mix. But applications have moved on -and it's this future of development that is going to drive the tooling.

The first biggest change is that dynamic languages are back. Regardless of how well the benefits really measure up, if they take off, they change how you develop. You dont need a compile phase anymore, just a deploy-and-test phase. If you edit the stuff in situ, saving the file can give you deployment, leaving only testing. At the same time, the client app is now JavaScript in three different browsers, so testing and debugging just got harder. This means that build tools are going to have to change. Instead of being something to compile static code, they have to adapt to become (or be replaced by) things that focus on deploy and test, rather than just compile-and-package.