Holmes with Visual Studio Test Framework

Open your solution in Visual Studio 2010.

Under Tools menu, you will find the Holmes Statistical Debugging menu option. Click this menu option and then choose Configuration Holmes instrumentation. This step open a windows that lists all projects in the solution.

 

Check the projects for which you would like instrumentation to be enabled. Note that enabling/disabling instrumentation changes the project configuration files. You may be asked to reload the project.

Rebuild your solution. Holmes assigns a timestamp to each build. You will use this timestamp to identify builds during Holmes analysis.

Create/edit your test settings to enable Holmes Path Coverage data collector, which should appear under the Data and Diagnostics tab in the test settings editor.

Run unit tests as you would normally. While running tests, Holmes collects path coverage data and test results for further analysis.

Open Holmes analysis window from Tools – Holmes Statistical Debugging – Holmes Window menu option and launch a new analysis.

In the new analysis window, select the type of coverage data, the project ("Local Solution" for non-TFS projects), the build and the test run that you would like to analysis. Optionally, you can select any of the previous builds as a baseline build.

Now click the Analyze button. This will start a new analysis. This might take a while depending on the number of test cases in the test run.

Once the analysis finishes, Holmes reports a list of code paths that may be causing the test failures. Each row in the window corresponds to a potential root cause. Also displayed is the source file and the method containing the root cause.

Double click on any of the rows to view the code path in the Visual Studio editor. The code paths are highlighted in red. Holmes associates and sorts the suspicious code paths with a confidence score (indicated by the red bar next to each code path). The confidence indicates the degree of correlation between the code path and test failures. Holmes also associates each code path with a set of failures that are explained by the code path.