Grails & Hudson Part 3: Testing

Since Grails incorporated the testing plugin into core it provides good unit & integration testing support (via the test-app script). There are also additional plugins to support BDD tools (e.g. EasyB) and functional testing (e.g. Canoo WebTest).
One of the useful roles that Hudson fulfils is helping to manage quality. Consequently it has plugins available for most of the popular testing tools (we saw an example of the Violations plugin in part 1 of this series).
We’ll start with the standard unit & integration tests, then add in test coverage and functional tests.

Unit tests
Assuming you already have unit/integration tests in your Grails project – we need to add the targets into the Grails builder options.
clean compile "test-app -unit -integration"

Notice how the server port has been set to prevent port collision with the servlet container running Hudson.
Tip: you'll need to set this differently on each job too.

Then set the post-build action:

After a couple of builds (so that there is a trend), Hudson will show:

With blue for tests that passed and red for failed tests.
You can also drill into the test results just as with the standard HTML junit reports.

Test coverage
As we're looking to use Hudson to help ensure the quality of the software project, we'll also inspect the level of test coverage as one of our quality metrics.

Warning: determining test coverage isn't foolproof and high levels of coverage don't guarantee good code or good tests. It is possible to achieve a high percentage with poor tests (I'm writing a separate post on this topic).

For this post, we'll be using Cobertura to check the code coverage. There are other options, but we'll be using Cobertura because:

  1. There is a Grails plugin
  2. There is a Hudson plugin too.

Install the Grails code-coverage plugin
grails install-plugin code-coverage

You may want to configure additional exclusions in BuildConfig.groovy, e.g. to avoid testing 3rd party code such as a plugin taglib that isn't used. (the plugin scripts/_Events.groovy includes a default list)

Install the Hudson Cobertura plugin
This follows the same process that we covered in part 2 of this series (we'll omit the screenshots this time):

  1. From the main Hudson dashboard, click on the "Manage Hudson" menu item
  2. Click on "Manage plugins"
  3. Go to the available tab, find & select Cobertura, click on the install button.
  4. When the plugin has downloaded and installed, you will need to restart Hudson (you can use the button provided).

Configure your Hudson job
There are 2 things that need to be done here:

  1. Set up the Grails build action
  2. Enable the post-build action for Cobertura

1. This is a case of adding "-coverage -xml" into the build targets within the test-app double quoted section.
2. Enable the Cobertura post-build processing, tell it where the XML report is and set thresholds:

Leverage the information
Watch the trends:

and drill down to see the in-context coverage:

When you have the insight as to where you are missing coverage, then you can take the appropriate action to remedy the situation...

Functional tests
There are a number of options for functional testing your application. We'll cover Canoo WebTest in this post for the following reasons:

  1. WebTest is htmlunit based - portable across different environments
  2. Canoo have written a Firefox WebTest Recorder add-on - allowing less technical users to create the basis of the tests
  3. There is also a WebTest Presenter plugin for Hudson.

Note that as WebTest isn't browser-based, you may encounter some issues with JavaScript on particular test scenarios.

Install the Grails webtest plugin
grails install-plugin webtest
If you are using older versions of Grails, you may need to do an interactive plugin install on the Hudson server (the plugin used to download a WebTest bundle, rather than having it as a managed dependency).

Install the Firefox WebTest Recorder add-on
You can install the WebTest Recorder in Firefox from:
https://www.canoo.com/webtestrecorder-dist/webtestrecorder.xpi

Create some web tests
You can come back to this bit later
The process we'll follow here uses the WebTest Recorder to create the initial test steps.

1. Run "grails create-webtest name" e.g.
grails create-webtest HomepageLogin

This plugin script will create us the placeholder test file under test/webtest.

2. grails run-app

3. Bring up Firefox
4. Navigate to the starting page for your test scenario.
5. Enable the WebTest Recorder Sidebar (Tools > Webtest Recorder Sidebar)
6. Click on the Groovy tab.

This should already contain "invoke http://localhost:8080/YourGrailsApp/" (you'll probably want to trim this down to just a relative path e.g. /YourGrailsApp/)
Using the Webtest Recorder Sidebar, you can click around the site and also easily add verification of content on pages (as a test isn't a test if you don't verify that the output is correct for the given input). This can then be placed into a method in the groovy file we created earlier (note: you may need to trim some instructions e.g. where the recorder has created different versions for the same input field).
For more information using Webtest, see the manual.

I'd strongly advise you to run your tests locally first before checking them in - in the interests of time/space we'll assume you've done that using:
grails test-app -functional

Install the Hudson WebTest presenter plugin
This follows the same process as the Cobertura plugin above (see part 2 of this series for screenshots):

  1. From the main Hudson dashboard, click on the "Manage Hudson" menu item
  2. Click on "Manage plugins"
  3. Go to the available tab, find & select WebTest Presenter, click on the install button.
  4. When the plugin has downloaded and installed, you will need to restart Hudson (you can use the button provided).

Configure your Hudson job
There are 2 things that need to be done here:

  1. Set up the Grails build action
  2. Enable the post-build action for the WebTest Presenter

1. This is a case of adding "-functional -headless" into the build targets within the test-app double quoted section (The headless option tells Webtest to not run the test monitor nor to load the test results into the browser).

2. The Webtest Presenter post-build action needs enabling and telling where to find the results

After the Hudson job has executed your webtests, you'll see that these have been included in your overall test count (and may even have improved your test coverage if you've followed the Cobertura section above).

If you click onto an individual build, the left hand menu will now have "Webtest Results"

This will give you an overview report:

and a detailed step-by-step report for each test (including resulting pages and any errors):

So there you go, a whistle stop tour of setting up test execution, coverage analysis and result reporting with trends.
Next in the series, we move into using Hudson for continuous deployment of Grails projects.

About these ads

5 responses to “Grails & Hudson Part 3: Testing

  1. Pingback: Grails & Hudson Part 3: Testing

  2. Pingback: Grails & Hudson / Jenkins Part 5: Monitoring build status | Lean Java Engineering

  3. Just wondering if you didn’t have any issues with this issue: https://issues.jenkins-ci.org/browse/JENKINS-8535

    I know I do… did you find a way to get around it? Hard to do TDD this way…

    • Hi Hans,

      Obviously I’ve encountered that behaviour, but as mentioned elsewhere I use the Green Balls plugin – so a failed build due to a failed test matches a decades worth of JUnit “keep the line green to keep the code clean” expectation.

      I appreciate your view that with TDD you should write a failing test & then fix it. However, even with TDD, I expect team members to execute the tests before checking in and only check in when all is working. This is to prevent disrupting fellow team members (“do as you would be done by”). So the CI job catches errors/regressions that have leaked into the source code control system.

      From that perspective a failed test in Hudson/Jenkins is a sign that the HEAD cannot be trusted which is a problem that needs sorting before it impacts the delivery schedule. Build failure provides notification by email to the project leads and the person(s) who checked in code to that build.

      The CI Game plugin can be used to encourage good behaviour as you gain points for adding/fixing tests and lose points for breaking the build. The one downside is that on large teams with multiple concurrent check-ins, you can be docked points for other team members mistakes!

  4. Pingback: C.R.A.P. metrics for Grails | Lean Java Engineering

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s