Pex - Automated Whitebox Testing for .NET : Test Failures

Test Failures

When does a test case fail?

  1. If it does not terminate within the configured path bounds, it is considered as a failure, unless the TestExcludePathBoundsExceeded option is set.
  2. If the test throws a PexAssumeFailedException, it succeeds. (But it is usually filtered out, unless TestEmissionFilter is set to All.)
  3. If the test violates an assertion, e.g. by throwing an assertion violation exception of a unit testing framework, it fails.
  4. Otherwise, all registered IPexExceptionValidator are asked.
    • Every validator can decide to either let the test fail or pass.
    • One can use the common attributes which implement this the validator interface:
    • One can define it's own triage logic by definig a custom exception validator
If none of the above give a decision, a test succeeds if and only if it does not throw an exception. Assertion-violations are treated like exceptions.

Sometimes, when a test case fails, Pex can suggest a fix.

Diagnostic Messages

(c) Microsoft Corporation. All rights reserved. pex Wiki Documentation 0.93.50813.0