With your SpecFlow installation comes SpecFlow.exe that is a program that can be used to generate tests from the scenarios AND to create nicely formatted reports from the generated test results. There’s been a lot written on how to generate these reports when your using NUnit (see this and this for example), but when it comes to managing this for MsTest there’s been almost silent. And facing this problem I can sure see why… It’s a bit trickier.
In this blog post I want to show you two things; how to generate MsTest’s from your .feature-files and how to create a report from the generated results. Finally I’ll show you how to put the two together and link it up to a nice “External tool”-button in Visual Studio. Here we go:
Generate MsTest’s from your .feature-file
With this step you can generate the test from you scenarios. SpecFlow.exe picks up your configuration and generates the test in your test framework of choice. From the look of it, it seems quite straight forward. Using the “help command” of SpecFlow.exe, “specflow help generateall” produces this help:
Generate tests from all feature files in a project usage: specflow generateall projectFile [/force] [/verbose] projectFile Visual Studio Project File containing features
OK. I put together a .bat file that does that. (Note that I’m on a 64-bit machine and had to use some funky DOS-shortcut to get there. No simple way to be architecture agnostic I’m afraid). Here is my file:
"%ProgramFiles(x86)%\TechTalk\SpecFlow\SpecFlow.exe" generateAll Specs\Specs.csproj /force /verbose pause
I’ve added some line breaks for readability.
Well – that was easy.
Create a report from the generated results
To get this step to work we have to run the test and get hold of the location of the test report file (.trx). When you do this from within Visual Studio the test reporting is done in a TestResult folder and the file get a name with a timestamp. That is not very script-friendly sadly and we’re forced into writing a bat-file that also run the tests.
I used the MsTest Command line reference to put together this .bat-file:
if Exist TestResult.trx del TestResult.trx
"%ProgramFiles(x86)%\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe"
Again, watch out for me using a 64-bit Windows and use %programfiles% if you’re not.
Some strangeness with MsTest made me delete the testResults.trx before each run. Also MsTest create some folders (username_testrun…) but that doesn’t bother me now.
When we now have a created .trx file we can run the command that creates the report for us. According to the “documentation” (specflow help mstestexecutionreport”):
usage: specflow mstestexecutionreport projectFile [/testResult:value] [/xsltFile:value] [/out:value]
- The projectFile is the Visual Studio Project File containing features and specifications.
- Test Result refers to the .trx file generated by MsTest And it defaults to TestResult.trx
- Xslt file to use, defaults to built-in stylesheet if not provided
- out is the generated output file. Defaults to TestResult.html
I’m happy with the defaults of the last three since I choose my .trx-file name … wisely. :) So my whole .bat-file becomes this:
"%ProgramFiles(x86)%\TechTalk\SpecFlow\SpecFlow.exe" mstestexecutionreport Specs\Specs.csproj pause
A low and behold; it actually produces the nice report we wanted: