...
I've used [Fitnesse|http://fitnesse.org/] in the past and I liked it because
* tests were fundamentally easy to read with red / green highlighting
* tests could be changed and run again fast - so a tight feedback loop
* tests could be run as part of a CI process (just)
but I really didn't like it because
* took a long time to set up and get effective even with a Fitnesse expert on the team
* no way to store the tests in Source Control separate from the wiki server itself - so ended up storing the whole thing in Source Control!
* only way to run the tests is to have a running Fitnesse server somewhere (with associated CI ceremony)
Then a good friend of mine suggested I have a look at [Concordion|http://www.concordion.org/] which I instantly liked. This is mainly because it has the same good points as Fitnesse but I found was
* very simple to use
* integrated easily with JUnit
* integrated easily with source control
oh and it looked very slick too. Which is always nice if the users / customers are going to be seeing it. For more details have a look on the Concordion site - it's pretty compact and simple, and refreshingly it's author David Peterson is pretty opinionated on what makes a good User Acceptance test.
h1. Specs And Scripts
One thing I'd noticed with my early forays into automating front end testing was how brittle these tests can be. Especially if the front end design isn't stable - which it never is in the early days. I'd heard a lot about agile testing and getting early feedback from these kind of tests seemed like a good idea but the reality was the testers ended up doing a lot of rework for even simple changes to the UI.
Concordion has a novel solution to this problem. Separate your Test Spec from your Test Scripts. So the Test Spec should just describe the behaviour of a feature in terms that don't bind the spec to an actual implementation of the feature. So you could say
{panel}
Given a user who wants to create a new book
when the user provides a valid book name "Moby Dick"
then the system will create the book "Moby Dick"
{panel}
Notice I don't go into detail as to how you supply the book name (eg. textfields, clicking buttons, what should be displayed etc.) Not only do these things make the tests less readable for a user, they also make the tests less durable. David Peterson suggests using the "Given, when, then " syntax that is popular in other emerging intention driven testing frameworks like [EasyB|http://www.easyb.org/].
Now of course you do need to bind your spec to an actual scripted implementation at some point, but the key thing is that the spec is separated from the script itself. So how do we actually write a spec and a script?
h1. Example Spec and Script
To write a spec in Concordion you use {{html}}.
{{html}} might seem like a strange choice but it works well because of its limitations. Here's the example test above as a Concordion test
{code:title=com.dish2dish.concordion.examples.Book.html}
<head>
<title>Book Test</title>
</head>
<body>
<h1>Create Book</h1>
<p>A user should be able to create a book using the bookstore.</p>
<div class="example">
<h3>Examples</h3>
<p>Given a user who wants to create a new book</p>
<p>when the user provides a valid book name of "Moby Dick"</p>
<p>then the system will create the book "Moby Dick".</p>
</div>
</body>
</html>
{code}
The nice thing about this page is that I can view it straight away in a browser. I can read it and it makes sense.
So how do I execute this test? I don't even have a system yet! The next stage is to "instrument" that html so that it can be executed by Concordion.
"Instrument"? Sounds painful...
it isn't.
{code:title=com.dish2dish.concordion.examples.Book.html}
<!-- Add Concordion instrumentation -->
<html xmlns:c="http://www.concordion.org/2007/concordion">
<head>
<title>Book test</title>
</head>
<body>
<h1>Create Book</h1>
<p>A user should be able to create a book using the bookstore.</p>
<div class="example">
<h3>Examples</h3>
<p c:execute="prepareToCreateBook()">Given a user who wants to create a new book</p>
<p>when the user provides a valid book name of "<span c:set="#bookName">Moby Dick</span>"</p>
<p c:execute="#book = createBook(#bookName)">then the system will create the book
"<span c:assertEquals="#book.name">Moby Dick</span>".
</p>
</div>
</body>
</html>
{code}
What this effectively does is
* prepare the scenario with a {{prepareToCreateBook()}} method call
* set up a variable called {{bookName}} with a value {{"Moby Dick"}}
* call the method {{createBook()}} with our {{bookName}} variable and store the result in a new variable {{book}}
* assert the result {{book}}'s name equals "Moby Dick""
Notice that even without us writing any JUnit test code, the scripts are still readable as html. They aren't executable but they can be written and reviewed. This allows testers to write their Acceptance Test Specs early as features are worked on.
h1. Runnning The Test
Concordion is based on JUnit. That means you can run Concordion tests in all the places you can run JUnit - from CI, the CLI or within your IDE.
But so far all we have is a Test Spec written in {{html}} - now we need to write a JUnit test that can execute the spec.
Concordion follows convention over configuration - it expects you to place html files and Java classes in the same package/directory structure.
I'm going to be using Maven2 for my build system and Groovy to generate my Java classes. So what does my maven2 directory structure look like?
{code:title=Concordion maven2 project structure}
+ concordion-example-integration-tests
.pom.xml
+ src
+ test
+ groovy
+ com.dish2dish.concordion.examples
. BookTest.groovy
+ resources
+ com.dish2dish.concordion.examples
. Book.html
{code}
Notice that I put a {{BookTest.groovy}} class into the {{test/groovy}} maven2 groovy source path and a {{Book.html}} file into the maven2 resources path. The files must have the same root name (e.g. Book) and package/path by convention with the actual class appending {{Test}} to the file name.
So what does BookTest.groovy look like?
{code:title=com.dish2dish.concordion.examples.BookTest.groovy}
package com.dish2dish.concordion.examples
import org.concordion.integration.junit4.ConcordionRunner
import org.junit.runner.RunWith
@RunWith (ConcordionRunner.class)
class BookTest {
void prepareToCreateBook() {
println "prepareToCreateBook"//cheat initially
}
Map createBook(String bookName) {
[name: bookName]//really cheat
}
}
{code}
In this first example, since we don't have a running web application yet I'm going to simply cheat the test. This at least shows the Concordion test running and interacting with our Test Spec. The output in IntelliJ looks like this
{code}
prepareToCreateBook
C:\DOCUME~1\ThorneNe\LOCALS~1\Temp\concordion\com\dish2dish\concordion\examples\Book.html
Successes: 1, Failures: 0
{code}
and if I look at the file {{C:\DOCUME~1\ThorneNe\LOCALS~1\Temp\concordion\com\dish2dish\concordion\examples\Book.html}} I get the following:
!BookTestResult.PNG!
Concordion has applied a simple and clean stylesheet to the test to make it look good and you can see the result in green.
h1. What's next?
With a quick bit of maven2 pom magic we can get this simple test under CI. I'm using a multi-module project to allow for the grails project to be built separately as a war (more on this in the next part).
{code:title=Concordion maven2 multi module structure}
+ concordion-example-parent
.pom.xml
+ concordion-example-integration-tests
. pom.xml
{code}
Using the poms you should be able to run the tests using {{mvn clean install}} from the parent pom to ensure the integration test phase is run.
The main things to note around the maven2 usage are that I'm using the great gmaven plugin to turn the Groovy code into classes, and I'm forcing the surefire plugin to run the Concordion tests in the integration-test phase. This will become important once we start to test the Grails web app.
We've still got a lot to do. Coming up in part2 I'll go over
* generating a Grails bookstore app for testing
* using the Selenium API from our Concordion test to test the Grails application
* using maven2 to control the whole process
