As more and more people use Badboy and integrate it into their development and testing workflows a question that has been becoming more and more frequent has been “how do I build Badboy into my automated build scripts so that it can run as part of my normal build process?” This question becomes even more important when you attempt into integrate Badboy into a continuous integration environment such as Hudson (or Jenkins, as it is now called).
The good news is that with the latest versions of Badboy (as of this writing, Badboy 2.1.1 and up) using Badboy with one build tool - Ant – just got a lot easier. In fact, even if you use a different build tool, new features in Badboy 2.1.1 will help you better integrate Badboy into your workflows because there are new command line options for Badboy that significantly streamline the process. However this post will focus on support for Ant as this extremely common build tool now has direct support via some very handy Ant plugin tasks that make it extremely simple to integrate Badboy.
<taskdef name="runscript" classname="com.badboy.ant.RunScriptTask"/>
<taskdef name="runTestPlan" classname="com.badboy.ant.WTMTask"/>
Once you’ve done this you’re ready to use the tags just like any other inside your build file.
Using the RunScript Tag
Once you’ve declared the tasks putting the tags to use is easy. Running a simple script using Badboy on the same computer as the build script is running is, in fact, just a single tag:
If you prefer different behavior you can stop the build from failing when a script fails by simply adding a failOnError attribute and setting it to false:
<runscript script="test.bb" failOnError="false"/>
If you want to pass variables to your script then you can easily add them as child elements:
<runscript script="test.bb"> <var name="foo">bar</var> <var name="cat">dog</var> </runscript>
Using the Wave Test Manager Tag
The RunScript tag is simple and easy to use, but it has one glaring disadvantage: it runs the script on the same computer as that where the build script itself is running. This may not be a problem in some situations but there are a range of scenarios where it just doesn’t work at all – perhaps you like to do useful work while your build is running and don’t like a giant Badboy window popping up in the middle of the build, or perhaps you need to run multiple builds concurrently and having many Badboy instances popping up together causes problems – or perhaps you want to support developers who aren’t using Windows and can’t run Badboy at all. All these issues are easily dealt with using Wave Test Manager and the runTestPlan task. Before you can do this, of course, you need to install Wave Test Manager somewhere – anywhere will do, including on the same computer as the build is running on. For the purpose of this post, I’ll assume that you have the server running on a server called “wave.acme.com” – but it could be on any computer, the build script doesn’t care. Assuming you’ve got a live Wave Test Manager server, running your script is once again, just a single tag:
<runtestplan name="Example" server="wave.acme.com:8030"/>
There are some differences to how this task works to the previous one that need to be noted. Firstly, with Wave Test Manager, you don’t run script files directly but rather Test Plans which are essentially just a list of scripts with preconfigured settings for how they should run. So instead of specifying a script file name now you are specifying a Test Plan name that matches the name one of your Test Plans in Wave Test Manager.
Another important difference is that instead of pausing while the test plan executes, the build will, by default, continue on in parallel while the test executes, and the build script will not fail if the script has errors or assertion failures. The reasoning here is that a script scheduled to run in Wave Test Manager is simply placed in queue and there is not a guarantee that it will begin executing straight away – that depends on the availability of a Badboy Agent to run the script. Wave Test Manager also supports its own notification mechanisms to tell you about script failures so you don’t necessarily need a build failure to tell you about it.However if you prefer your build script to wait while the script executes and fail when it fails, you can add attributes to make that behaviour occur:
<runtestplan name="Example" server="wave.acme.com:8030" waitFor="true" failOnError="true"/>
Now your build will patiently wait until the script finishes executing and it will fail the build if any of the scripts in the Test Plan has errors or assertion failures.
Hopefully these Ant tasks will make integrating Badboy into automated builds much easier and help Badboy users streamline execution of their test scripts. Of course, there are many other build tools out there and many more features that could be added to these tags as well, so let us know what you want at the Badboy Feedback Page and we’ll endeavour to oblige!