2009/12/10
Wow. A late night - but I got Confluence back up and running.
Off to the Groovy and Grails Exchange in the morning. Looking forward to it.
Introduction
In this series of blogs I'm going to
- introduce the Concordion framework - a simple yet powerful way to write presentable, durable automated acceptance tests that run on the JVM
- show how those Concordion tests can be written in Groovy and used to drive the Selenium testing API and test a Grails web application. (Why Grails? Well it could be any web app stack but Grails is what I'm currently using in my projects and I think it rocks in terms of productivity!)
- show how easy it is to use Maven2 to run the whole thing in a CI compatible set of pom files
Full code examples will be attached for you to download and try! I hope you enjoy them...
Part 1 - Introducing the Concordion framework
 | Source Code Full Source code for this part is available here. |
| Part 2 Part2 of this series is available here |
| Prerequisites To understand this blog best you're going to need to download the attached source code and refer to it. You'll also need to download maven2 (and better still have some background knowledge of how to use it). |
I'm currently involved in architecture and development of a couple of applications where I was looking for an acceptance framework that would bring the users in. After all they're the ones who need to accept the system.
I've used Fitnesse in the past and I liked it because
- tests were fundamentally easy to read with red / green highlighting
- tests could be changed and run again fast - so a tight feedback loop
- tests could be run as part of a CI process (just)
but I really didn't like it because
- took a long time to set up and get effective even with a Fitnesse expert on the team
- no way to store the tests in Source Control separate from the wiki server itself - so ended up storing the whole thing in Source Control!
- only way to run the tests is to have a running Fitnesse server somewhere (with associated CI ceremony)
Then a good friend of mine suggested I have a look at Concordion which I instantly liked. This is mainly because it has the same good points as Fitnesse but I found was
- very simple to use
- integrated easily with JUnit
- integrated easily with source control
oh and it looked very slick too. Which is always nice if the users / customers are going to be seeing it. For more details have a look on the Concordion site - it's pretty compact and simple, and refreshingly it's author David Peterson is pretty opinionated on what makes a good User Acceptance test.
Specs And Scripts
One thing I'd noticed with my early forays into automating front end testing was how brittle these tests can be. Especially if the front end design isn't stable - which it never is in the early days. I'd heard a lot about agile testing and getting early feedback from these kind of tests seemed like a good idea but the reality was the testers ended up doing a lot of rework for even simple changes to the UI.
Concordion has a novel solution to this problem. Separate your Test Spec from your Test Scripts. So the Test Spec should just describe the behaviour of a feature in terms that don't bind the spec to an actual implementation of the feature. So you could say
Given a user who wants to create a new book
when the user provides a valid book name "Moby Dick"
then the system will create the book "Moby Dick"
Notice I don't go into detail as to how you supply the book name (eg. textfields, clicking buttons, what should be displayed etc.) Not only do these things make the tests less readable for a user, they also make the tests less durable. David Peterson suggests using the "Given, when, then " syntax that is popular in other emerging intention driven testing frameworks like EasyB.
Now of course you do need to bind your spec to an actual scripted implementation at some point, but the key thing is that the spec is separated from the script itself. So how do we actually write a spec and a script?
Example Spec and Script
To write a spec in Concordion you use html.
html might seem like a strange choice but it works well because of its limitations. Here's the example test above as a Concordion test
<head>
<title>Book Test</title>
</head>
<body>
<h1>Create Book</h1>
<p>A user should be able to create a book using the bookstore.</p>
<div class="example">
<h3>Examples</h3>
<p>Given a user who wants to create a new book</p>
<p>when the user provides a valid book name of "Moby Dick"</p>
<p>then the system will create the book "Moby Dick".</p>
</div>
</body>
</html>
The nice thing about this page is that I can view it straight away in a browser. I can read it and it makes sense.
So how do I execute this test? I don't even have a system yet! The next stage is to "instrument" that html so that it can be executed by Concordion.
"Instrument"? Sounds painful...
it isn't.
<!-- Add Concordion instrumentation --> <html xmlns:c="http://www.concordion.org/2007/concordion"> <head> <title>Book test</title> </head> <body> <h1>Create Book</h1> <p>A user should be able to create a book using the bookstore.</p> <div class="example"> <h3>Examples</h3> <p c:execute="prepareToCreateBook()">Given a user who wants to create a new book</p> <p>when the user provides a valid book name of "<span c:set="#bookName">Moby Dick</span>"</p> <p c:execute="#book = createBook(#bookName)">then the system will create the book "<span c:assertEquals="#book.name">Moby Dick</span>". </p> </div> </body> </html>
What this effectively does is
- prepare the scenario with a prepareToCreateBook() method call
- set up a variable called bookName with a value "Moby Dick"
- call the method createBook() with our bookName variable and store the result in a new variable book
- assert the result book's name equals "Moby Dick""
Notice that even without us writing any JUnit test code, the scripts are still readable as html. They aren't executable but they can be written and reviewed. This allows testers to write their Acceptance Test Specs early as features are worked on.
Runnning The Test
Concordion is based on JUnit. That means you can run Concordion tests in all the places you can run JUnit - from CI, the CLI or within your IDE.
But so far all we have is a Test Spec written in html - now we need to write a JUnit test that can execute the spec.
Concordion follows convention over configuration - it expects you to place html files and Java classes in the same package/directory structure.
I'm going to be using Maven2 for my build system and Groovy to generate my Java classes. So what does my maven2 directory structure look like?
+ concordion-example-integration-tests .pom.xml + src + test + groovy + com.dish2dish.concordion.examples . BookTest.groovy + resources + com.dish2dish.concordion.examples . Book.html
Notice that I put a BookTest.groovy class into the test/groovy maven2 groovy source path and a Book.html file into the maven2 resources path. The files must have the same root name (e.g. Book) and package/path by convention with the actual class appending Test to the file name.
So what does BookTest.groovy look like?
package com.dish2dish.concordion.examples import org.concordion.integration.junit4.ConcordionRunner import org.junit.runner.RunWith @RunWith (ConcordionRunner.class) class BookTest { void prepareToCreateBook() { println "prepareToCreateBook"//cheat initially } Map createBook(String bookName) { [name: bookName]//really cheat } }
In this first example, since we don't have a running web application yet I'm going to simply cheat the test. This at least shows the Concordion test running and interacting with our Test Spec. The output in IntelliJ looks like this
prepareToCreateBook C:\DOCUME~1\ThorneNe\LOCALS~1\Temp\concordion\com\dish2dish\concordion\examples\Book.html Successes: 1, Failures: 0
and if I look at the file C:\DOCUME~1\ThorneNe\LOCALS~1\Temp\concordion\com\dish2dish\concordion\examples\Book.html I get the following:
Concordion has applied a simple and clean stylesheet to the test to make it look good and you can see the result in green.
What's next?
With a quick bit of maven2 pom magic we can get this simple test under CI. I'm using a multi-module project to allow for the grails project to be built separately as a war (more on this in the next part).
+ concordion-example-parent .pom.xml + concordion-example-integration-tests . pom.xml
Using the poms you should be able to run the tests using mvn clean install from the parent pom to ensure the integration test phase is run.
The main things to note around the maven2 usage are that I'm using the great gmaven plugin to turn the Groovy code into classes, and I'm forcing the surefire plugin to run the Concordion tests in the integration-test phase. This will become important once we start to test the Grails web app.
We've still got a lot to do. Coming up in part2 I'll go over
- generating a Grails bookstore app for testing
- using the Selenium API from our Concordion test to test the Grails application
- using maven2 to control the whole process