Managing automated build dependencies with Maven and Hudson!!

One of the tricky parts of setting up a Continuous Integration build server is managing dependencies between build jobs. Many organisations have projects made up of tens or hundreds of different, interrelated modules, with complex dependencies between them. So when you change a module somewhere, you may need to rebuild and retest other modules that depend on this module.

One of the great features of Maven is the way it handles dependency management. Maven detractors may moan and groan about black magic and not being able to see what happens under the hood, but for me, declarative, transitive dependency management is a real life-saver.

Now, when you add Maven dependency management to Hudson automated builds, the result is nothing short of miraculous. Well, maybe I'm exagerating slightly, but it's certainly very cool. In short, if you use "Maven" build jobs in Hudson instead of the more frequently-used "free-style" build jobs, Hudson will figure out the correct build order all by itself - and rebuild all the other projects directly or indirectly affected by your changes.

Let's look at an example. Suppose my team is working on a Maven project called "core-api". We've already released one version of the core-api (version 1.0.0), which is available on the Maven enterprise repository for other teams. However, we are currently busy adding new features in the 1.0.1-SNAPSHOT version.

Now suppose we also have an application called "super-web-app", which we are building in conjunction with the core-api module. Now these two modules are essentially part of the same project, so we make the super-web-app depend on a snapshot release of core-api (1.0.1-SNAPSHOT). That way, I can make any changes in the core-api immediately available in the super-web-app application.

We are using Hudson as our CI server, using the "Maven" build jobs in Hudson to automatically build both these projects whenever a change goes to Subversion. Since they are intimately related, we would like the super-web-app build job to kick off whenever the core-api changes. Traditionally, you would do this by creating a post-build goal in your core-api build job to kick of the super-web-app job. But what if there are twenty other modules that depend, directly or indirectly, on the core-api? Do you have to add (and maintain) all of these post-build triggers by hand? And what if some of these other modules depend on older versions of my API, and don't give a hoot if I release a new snapshot version?

For example, across the hall, some colleages are working on the killer-web-app project. Their application still depends on core-api version 1.0.0. They are free to upgrade their version of the core-api when they need to, but it's pretty much up to them as to when they do so. So, as long as their dependency is on core-api version 1.0.0, the rebuilds of core-api-1.0.1-SNAPSHOT on Hudson should not trigger a build of their project.

The Hudson Maven projects handle this sort of build depencency management beautifully. When you kick off a build for a Maven project, it will automatically look at the other Maven build jobs on your server. If any of these depend on the Maven project currently being built, they in turn will be scheduled to be rebuilt. Note that Hudson takes into account the version number in the dependency - if core-api 1.0.1-SNAPSHOT is rebuilt, then only the modules depending on core-api-1.0.1-SNAPSHOT will be scheduled to be rebuilt - the modules depending on the older core-api-1.0.0 version will be left alone.

Note that this also works transitively, as you would imagine - if a change in project A causes Hudson to rebuild project B, and project C depends on project B, then project C will be rebuilt as well. This is a very elegent solution to a quite complex problem, and one that requires no specific configuration or maintenance in the Hudson build jobs - it relies entirely on the Maven dependencies to get the job done.