Pex - Automated Whitebox Testing for .NET : coverage

Coverage


Dynamic Coverage


As a side effect of the runtime monitoring, Pex computes collects dynamic code coverage data. We call it dynamic because Pex knows only about code that has been executed, therefore it cannot give absolute values for coverage such as other coverage tool usually do. For example, when Pex reports that the dynamic coverage as 5/10 basic blocks, then this means that 5 blocks out of 10 were covered, where 10 is the total number of blocks in all methods that have been reached so far by the analysis. (As opposed to all methods that exist in the assembly-under-test.) Later in the analysis, not only the nominator (here: 5), but also the denominator (here: 10) can increase, when the analysis discovered more reachable methods.

Coverage Domains


Pex partitions code into the following coverage domains: user code under test, user or test code or system code.
  • user code under test: this is the code under test for the current exploration. This code is included in the computation of the coverage numbers (block counts, etc...) and the source coverage will be rendered in the reports. Usually, this is the product code.
  • user or test code: this is code that is part of your projects or tests but that is not tested by the current exploration. This code is not included in the computation of the coverage numbers but the source coverage is still rendered in the reports (with different colors). Usually, this is the test code since you don't care about your test coverage but yet it is interresting to see the coverage of tests.
  • system code: any framework or third code for which you do not want to collect coverage data. This code is not included in the coverage numbers and source coloring.

To change the defaults, use attributes deriving from PexCoverageFilterAttributeBase or the Type Under Test feature.

Coverage Units


These are the different ways that Pex uses to characterize coverage:
  • blocks, basic blocks,
  • method touched, the number of methods that were touched at all.

Sample


Let's assume we have the following code given.

    class SystemCode {
        public void Run() {
            Console.WriteLine("I should show NOT show up!");
        }
    }
    class UserOrTestCode {
        public void Run() {
            bool b = true;
            if (b)
                Console.WriteLine("I should show up blue!");
            else
                Console.WriteLine("I should show up gray!");
        }
    }
    class UserCodeUnderTest {
        public void Run() {
            bool b = true;
            if (b)
                Console.WriteLine("I should show up green!");
            else
                Console.WriteLine("I should show up red!");
        }
    }
Let's consider the following simple exploration (without any parameter), to illustrate the coverage reports:
    [TestClass]
    [PexClass(
        // type under test is in UserCodeUnderTest domain
        typeof(UserCodeUnderTest), 
        Suite = "checkin")]
    // UserOrTestCode is the default domain
    // [PexCoverageFilterType(PexCoverageDomain.UserOrTestCode, typeof(UserOrTestCode))]
    [PexCoverageFilterType(PexCoverageDomain.SystemCode, typeof(SystemCode))]
    public partial class CoverageFilterTest {
        [PexMethod]
        public void CallExcludeMe() {
            new UserCodeUnderTest().Run();
            new UserOrTestCode().Run();
            new SystemCode().Run();
        }
    }

The resulting coverage is as follows:

coveragecolors.png

See Also

PexCoverageFilterTypeAttribute, TestClassAttribute, PexClassAttribute, PexMethodAttribute
(c) Microsoft Corporation. All rights reserved. pex Wiki Documentation 0.93.50813.0