Testing Code: Intentional vs. Incidental Behavior

Sometimes a maintenance programmer is another developer with an axe. Sometimes the maintenance programmer is ourselves after we have forgotten all about the project. I want whoever gets it to be happy and confident when they make changes.

Having inherited projects in the past (both from others and from past-George), I think it’s important to make sure that a project comes with good automated tests.

Hopefully, in 2016 that isn’t a controversial statement. I do want to explicitly enumerate some of the value tests provide the maintenance programmer:

  1. Testing forces us to simplify and decouple our code, which makes maintaining a system easier. If it’s hard to test, it’s going to be hard to maintain.
  2. Tests provide example usage of the code, so they serve as living examples of how to use the code.
  3. Testing infrastructure. It’s easier to add one more test when they already have tests for other parts. It also provides examples to mimic when someone is getting up to speed.
  4. Good tests define what behavior is intentional.

Let’s talk about #4.

Virtually all code has is intentional behavior and incidental behavior. Incidental behavior is what you get when a detail doesn’t matter to the functionality being developed. An example would be when your UI displays a list in whatever order the datasource provides it. You probably shouldn’t write a test for that; there’s no specified behavior.

In a codebase with tests for the intentional behavior, a maintainer can be confident that any changes they make aren’t unknowingly undoing past decisions. The tests answer the question “is it supposed to be that way?”

When retroactively adding tests (post-hoc testing) you run the risk of documenting the code instead of the requirements. If you find yourself writing tests with the goal of verifying the code works, you aren’t confirming that the code meets the requirements. You’re confirming that the code is the code.

Should you find yourself writing post-hoc tests, what can you do? Try to put yourself in the shoes of the original author. Use git blame to discover why the code was written. Write a test that verifies the original feature and no more.

Finally, I suspect that only testing intentional behavior is in conflict with 100% (or any) code coverage targets. If you have 100% coverage, either you are writing tests to cover code that isn’t part of the specification or you have a specification that is comprehensive to a fault.