Signs That Your Unit Tests Are Not Helpful
Here are two metrics that you can look at with your unit tests:
- Percentage of test failures that are only a problem with the unit test, not a problem with the code running in a real environment. This often happens when you are mocking interfaces.
- Percentage of times that a bug fix or otherwise non breaking change requires an update to a unit test.
- The goal would be to have both these percentages at 0%. This would indicate that every test failure represents a real issue in your code and that only feature changes require updates to the unit tests.
- Every test failure that reflects a real code issue exactly out weighs every false positive.
- Every unit test update that comes from a non feature change is out weighed by the benefits of having a unit test that comes from a feature change.