Nancy.Testing - executable specifications through the full stack, in memory

· February 7, 2013

When I showed the code from the last post to a colleague (@HugoHaggmark), he remarked: Nice - but that no unit test. And he’s absolutely right.

A unit test should test a unit. The tests in the last post, and the test that Nancy.Testing allows us to easily write, flexes the whole stack of our application. In memory (which is super cool) but the full stack. That’s no unit test.

This can still be VERY useful and in this post I’ll show you how to put one of my favorite tools, SpecFlow, in front of what we wrote in the last post, to get an executable specification. Not only that - I’ll do it in a manner that let’s you swap it and hit the HTML page if you wanted to.

As the tradition calls - let's look in wonder at Nancy's silhouette. If you came here and want to read up on what we have spoken about so far, the other posts in the series can be found here: 1. Intro to testing with Nancy 2. The Configurable bootstrapper 3. The Browser and Response objects 4. Hat and shoeless testing with Simple.Data 5. SpecFlow and Nancy(this post) In this post I'll introduce you, short - very short, to Specification by example and SpecFlow and some practices that makes automating your application ... bearable ### Specification by example and SpecFlow Specification by example is a way to analyze how your application should behave, by posing concrete examples. You do this together with the business or client in order to understand what they need. By doing so you can come up with a solution that is more in line with what they have envisioned. You're increasing the possiblity that you are building the thing **right.** ** ** As a nice side effect of having these discussions with the client you end up with a lot of examples that can easily be used as test cases to validate your code. Using certain formats and tools (hrmf SpecFlow hrmf) you could even have these example executed against your system. ** ** That means that you could run the examples you talked about against your system and they will tell you if you have implemented the feature as discussed or not. Your examples have become an **executable specification**. ** ** This was a super short introduction to the vast subject Specification by example (aka Behaviour Driven Development BDD). If you haven't heard about it before I encourage you to check it out. It changed my life - and will change yours too if you give it a change. It's a more fun way to do better code! ** ** Specification by example is not about the tools. It's actually most about the conversations that you and your client have around the examples. So to honor that very important fact - let's talk tools: The tools I'm talking about here are tools that let you take the examples you talked about with your client and execute them, in a format that the non-technical client can read and understand. One of the more prominent tools like that is called Cucumber and the .NET implementation of that is called SpecFlow. Simply put the flow of SpecFlow is like this: 1. You write your examples in a .feature-file, using the Gherkin language. It's basically lines starting with Given, When or Then and then any english (or 40 other supported languages) line. "Given the database is empty" for example.   2. When you save the .feature-file SpecFlow generates a test, in a framework of your choice behind the scenes (in a .feature.cs-file). The .feature-file can now be executed by a test runner. 3. For each line the .feature-file ("When I log in" for example) SpecFlow will look for a method with an attribute like \[When("I log in")\] over it and execute it. 4. SpecFlow will continue to execute all the lines in the .feature-file. Just as for normal unit test it will report success if no error is thrown and failure otherwise. As you probably know, or understand, Asserts in test frameworks throw errors when the assertion fails. Summing up this simply means that each row, or step, in your .feature-files, that contains your examples, is mapped to a corresponding step definition, a method that defines what should happen in the step. For the step "When I log in" we automate the application to do log-in. SpecFlow has excellent support in Visual Studio and is completely free. ### Maintainable test code and Page wrappers When I first heard about specification by example I, like many others, wanted to automate as much as possible. And that's a good idea - surely you'd want your specifications to be run against as big a part of the system as you possible can. But that means, for all ~~loosers~~ the people not using Nancy, we have to automate against the UI. That's not very pleasant, since it moves a lot (in other word: give you brittle tests) and testing the UI means that you have very little control over the execution of a test. Finally testing against the UI is slow. In short: you'll mostly end up shooting yourself in the foot (thanks Gojko) and finally resign with not using the tests. There are some ways to get around some of the maintainability issues and structuring the test code a bit. The best one I know is what's known as the Page Wrapper. It's basically encapsulating access to a page in a separate class with all the UI automation logic. This wrapper (that you write) can be thought of as a Domain Specific Langauge (DSL) that you use to automate your application. What's great about this is that your step definitions are ligthweight and simple to write - you just delegate into the automation layer where all the automation happens. Not only that, but you could write a new DSL that for example didn't automate against the UI, without changing the .feature-file or the steps. Cool - huh? This can be seen to great effect in this excellent talk by Matt Wynne. Of course, it would be even better if we could skip the top layer, the UI, the hat if you will, and just test the application logic and not the rendering in the browser. If only ... now wait a minute here - that's exactly what Nancy.Testing gives us. That's just leaves us with test data that is inherently difficult to setup when hitting the production code. That would be your shoes and can also be handled with [Simple.Data](, as described before. We now have all the parts for what I wanted to talk about in this post. Longest intro ever, but there's some background in there as well, to my defense. Let's do a diagram over this:
The green boxes is code that you need to write (oh, well you need to write the .feature-file as well but that's not really code right). The rest of the picture is the frameworks work for us. ### Example Let's see how I've stitched this together. Mind you - this is my take on it. There's probably other ways to do this but this worked for me with reasonable complexity. The example has quite a lot of moving parts for such small test, but that pays off in the long run. Trust me on that one. For our example we will take a look at the behavior around creating new fairy tale figures. Here's my feature: As you can see there's just a little introduction at the top, stating the reason of this functionality. What then follows are two distinct scenarios with concrete values. In the first one we're focusing on creating a fairytale figure with no hangarounds. Sad one. In that latter we create another figure (note that we don't care about the name and hence don't use one) and make sure that the hangaround get's registered and is as evil as there master. Before we can take a look at the step definitions there's a little infrastructure you should know about: Here's we're using a SpecFlow event called BeforeScenario that is executed before ... you guessed it: each scenario. That gives us an excellent place to do some cleaning. In this case we're setting Simple.Data up to use the InMemoryAdapter. Now let's dive a step deeper and take a look at the step definitions: These are the method that SpecFlow calls for us for each step in the .feature-file. As you can see it's really nothing strange here. Notice that the step definitions can take in variables in thats extracted via regexp's from the steps in the .feature-file. Pretty useful. On the first line we create a FairyTaleModuleWrapper that hands us a nice API to interact with the functionality of the module. In fact that the kind of API to automation that you always wanted, right. Just what you want to: method to register and validate result. I wonder how the automation looks like. Who's writing that now again? We? Better get to it. Now we're talking: here's the automation code we have have seen examples from throughout the series. Let's walk through the code. It's a mouth full but by no means complicated On line 6-15 is the constructor of the wrapper. Here we set up a Nancy.Testing.Browser indicating that we'll be using the FairyTaleFigureModule to test against. We store the browser object in a private field since we'll be using that all over the class. - On line 13 (brrr) we're setting up the dependency for the module, our FairyTaleFigureRepository s. As before we using the production repository since we have already mocked out the data access with Simple.Data (see the Event-file above) Line 17-36 is a method used to perform the actual registration. The signature takes in the Name of the figure and the evilness of it and finally a number of hangarounds. - Line 21-26 creates and post the form with all our data. I use a couple of helper methods to create some values for the form. In one case in a very very ugly way. All help to make that nice is appreciated. - We then verify that we got redirected (line 34) to the right place - and finally do a GET to the newly created fairy tale figure. The results of the GET is stored in the \_latestResponse variable for use in later methods. The rest of the methods (line 38-64) are small verification methods in which we assert against the stored response (\_latestResponse) and verify that the name and evilness of the figured got rendered on the page properly. Or that all the hangarounds are as evil as their master (line 60-64). The final part is just my small private helper methods to create the form. Nothing particular exciting there. As I said - that's a mouth full when you first look at it. The number of lines in that code has partly to do with my formatting and my comments. The actual code is not that hard and built on everything we've talked about in this series. That just leaves the production code: Just a few notes on this: - I'm trying to keep the code in the "routes" to a minimum and hand it of to private methods. I think that's a good idea for readability. - Not proud of the code in BindHangarounds but I hope that someone will correct and help me writing something more ... Nancy-like there - The repository is opening the Simple.Data -database in it's constructor. You probably don't want to do that in real life. Better might have been to inject it into the constructor. In any way there's where the InMemoryAdapter is starting to get used by Simple.Data (the Database.Open()-stuff) ### Summary There's you have it. Testing the entire application stack, from a human (and business) readable format, with the help of SpecFlow, all the way to the ends of the dataaccess code (with the help of Simple.Data). All in memory. I don't think that can be done in many other frameworks (at least not on the .NET stack). Color me impressed! That brings us to an end of this series. I've learned a great deal about both Nancy and testing Nancy. I can now reveal that when I started to write these blog post I had some problems grokking it. That was why I jumped in at the deep end like that. Now that I know a bit more I can only say that I'm continioulsy being impressed with Nancy and her awesome testabilities. Great stuff indeed. Oh yeah - almost forgot. The code is up here.

Twitter, Facebook