One thing that really surprised me was that when tests fail in my build script the build script happily ran along after that!? Isn’t one of the ideas with Continuous Integration that the build script only will emit working versions?
Well in this article Aaron Hallberg (Swedish relatives?) shows us how to handle this. What he basically does is to check the status of the test-run and then update the status of the complete build.
I extended his code to use the test-status-flag on the tasks that follow the test, i.e. the publish task. This task deploys the executables to my test server, update the documentation etc.
So now I’ve got my desired result which is that failing test leads to failing the whole build.