Thursday, June 07, 2018

Easy and free unit test coverage reporting for .NET projects without having to buy Visual Studio Enterprise

Visual Studio has a nice little feature called Code Coverage Results, which you can find in the Test menu. However, it does absolutely nothing unless you have the Enterprise edition. If you have it, then probably this post is not for you.

How can we test unit test coverage on .NET projects without spending a ton of money?

I have first tried AxoCover, which is a quick install from NuGet. However, while it looks like it does the job, it requires xUnit 2.2.0 and the developer doesn't have the resources to upgrade the extension to the latest xUnit version (which in my case is 2.3.1). In case you don't use xUnit, AxoCover looks like a good solution.

Then I tried OpenCover, which is what AxoCover uses in the background anyway, which is a library that is suspiciously showing in the Visual Studio Manage NuGet Packages as published in Monday, February 8, 2016 (2/8/2016). However, the latest commits on the GitHub site are from April, only three months ago, which means the library is still being maintained. Unfortunately OpenCover doesn't have any graphical interface. This is where ReportGenerator comes in and some batch file coding :)

Bottom line, these are the steps that you need to go through to enable code coverage reporting on your projects:
  1. install the latest version of OpenCover in your unit test project
  2. install the latest version of ReportGenerator in your unit test project
  3. create a folder in your project root called _coverage (or whatever)
  4. add a new batch file in this folder containing the code to run OpenCover, then ReportGenerator. Below I will publish the code for xUnit
  5. run the batch file

Batch file to run coverage and generate a report for xUnit tests:
@echo off
setlocal EnableDelayedExpansion
 
for /d %%f in (..\..\packages\*.*) do (
    set name=%%f
    set name="!name:~15!"
    if "!name:~1,9!" equ "OpenCover" (
        set opencover=%%f
    )
    if "!name:~1,15!" equ "ReportGenerator" (
        set reportgenerator=%%f
    )
    if "!name:~1,20!" equ "xunit.runner.console" (
        set xrunner=%%f
    )
)
SET errorlevel=0
if "!opencover!" equ "" (
    echo OpenCover not found in ..\..\packages !
    SET errorlevel=1
) else (
    echo Found OpenCover at !opencover!
)
if "!reportgenerator!" equ "" (
    echo ReportGenerator not found in ..\..\packages !
    SET errorlevel=1
) else (
    echo Found ReportGenerator at !reportgenerator!
)
if "!xrunner!" equ "" (
    SET errorlevel=1
    echo xunit.runner.console not found in ..\..\packages !
) else (
    echo Found xunit.runner.console at !xrunner!
)
if %errorlevel% neq 0 exit /b %errorlevel%
 
set cmdCoverage="!opencover:\=\\!\\tools\\opencover.console -register:user -output:coverage.xml -target:\"!xrunner:\=\\!\\tools\\net452\\xunit.console.exe\" \"-targetargs: ..\\bin\\x64\\Debug\\MYPROJECT.Tests.dll -noshadow -parallel none\" \"-filter:+[MYPROJECT.*]* -[MYPROJECT.Tests]* -[MYPROJECT.IntegrationTests]*\" | tee coverage_output.txt"
set cmdReport="!reportgenerator:\=\\!\\tools\\reportgenerator -reports:coverage.xml -targetdir:.\\html"
 
powershell "!cmdCoverage!"
powershell "!cmdReport!"
start "MYPROJECT test coverage" .\html\index.htm

Notes:
  • replace MYPROJECT with your project name
  • the filter says that I want to calculate the coverage for all the assemblies starting with MYPROJECT, but not Tests and IntegrationTests
  • most of the code above is used to find the OpenCover, ReportGenerator and xunit.runner.console folders in the NuGet packages folder
  • powershell is used to execute the command strings and for the tee command, which displays on the console while also writing to a file
  • the little square characters are Escape characters defining ANSI sequences for showing a red text in case of error.

Please feel free to comment with the lines you would use for other unit testing frameworks, I will update the post.

0 comments: