Code Coverage in Visual Studio Team Suite

I recently started using Visual Studio Team Suite 2008. Normally these tools are not very affordable, but through the magic of  Microsoft Bizspark, they are within reach for the software startup.

One of the VSTS features that interests me most is code coverage metrics for my unit tests. I normally use Resharper as my test runner, but to enable code coverage I needed to use the VSTS test runner. My first challenge was to get my NUnit 2.5 unit tests to work properly in VSTS.

Here are the steps I used to make this all work for me:

1. Download and install NUnitForVS, an open source plugin that enables the Microsoft test runner to see the NUnit project as a test project.
2. Next modify the project (.csproj or .vbproj file) to be marked as a test project. This can be accomplished by:
   • Right clicking on the project and selecting Unload.
   • Right clicking again on the project and selecting Edit (project name) to open up the project xml file.
   • For a C# project, insert
     <ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
     as a child of the first <PropertyGroup>. For a VB project, insert
     <ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{F184B08F-C81C-45F6-A57F-5ABD9991F28F}</ProjectTypeGuids>
   instead.
3. Re-Build the project. The tests won't show up in the Test View (Test -> Windows -> Test View) if the test project assemblies are not built.
4. Create the Test Run Configuration. This is a solution item and needs to be created at the solution level.
   • Select the solution
   • Right click on it and select New Item
   • Select the Test Run Configuration and then select the Test Run Configuration template.
   • Type in a suitable file name and press the Add button.
5. The Test Run Configuration editor should now be displayed. The important settings to configure are located under the Code Coverage section. Select all the assemblies you want track code coverage on. Save the configuration. See NOTE below for some caveats regarding selecting the proper assemblies.
6. Run the tests: Go to the Test View again. Select all the tests (Ctrl+A). Right click and Run Selection.
7. Check the code coverage results: Open the Code Coverage window (Test -> Windows -> Code Coverage Results).

Some Gotchas For Selecting Assemblies For Code Coverage

My typical layout for a Visual Studio solution is something like:

ProjectToTestA
ProjectToTestB
UnitTestProject

The UnitTestProject has project references to ProjectToTestA and ProjectToTestB. When I build the solution I get output files as follows:

1: ProjectToTestA\bin\x86\debug\
2:    ProjectToTestA.dll
3: ProjectToTestB\bin\x86\debug\
4:    ProjectToTestB.dll
5: UnitTestProject\bin\x86\debug\
6:    ProjectToTestA.dll
7:    ProjectToTestB.dll
8:    UnitTestProject.dll

This works fine for unit testing but doesn't work so well for coverage testing. The reason is that the Test Run Configuration requires that you select each assembly to instrument for coverage testing, but only presents you items 2, 4, 8 from above. If you select only lines 2 & 4 to instrument you will get no results in the coverage tests. This is because when the unit test in line 8 is run, only assemblies 6, 7, and 8 are loaded. Lines 2 & 4 are not loaded and you get no coverage on them. To solve this problem, select line 8 from the list above and the click the Add Assembly... button. Now select line 6 & 7 as well. Everything should now work as expected.