TDD Antipatterns

(font is brianne's hand) James Carr enlisted a group of fellow travelers to define a list of TDD Antipatterns, errors in judgement common in TDD practice. He provided an initial list for us to base our work on, and did a very fine job of filtering out the duplicates and near-duplicates, providing catchy names, and writing up the result. Note that his list is longer than ours since we have a terseness constraint. Read the full list. I wish I had thought of it first.

• The Liar is a test that runs, but does not test what it claims to test. It could be named after a class, but actually be testing another. The test might be called ShouldNotThrowExceptionsForPositiveValues, but actually use the natural numbers less than 5. Liars give a false sense of security.
• Excessive Setup is common when the architecture is badly coupled and mocking is not well-used. This is often evidence of insufficient pre-factoring. A little dependency injection and a little interface use can go a long way. This is also common when programmers give in to the urge to test software "in context." One hopes that when they get to the assert they'll remember what scenario they were testing.
• The Giant is a single test that tests more than a single scenario. It may have excessive setup, but then follow with a large number of manipulations and assertions. It may test entire subsystems. Adding a new assertion requires a programmer to reverse-engineer his way through the Giant to find an assertion insertion point, and requries the programmer to exercise care to not leave side-effects that will cause the last half of the Giant to fail. Giants are ticklish and hard to understand.
• The Mockery is a real piece of work. A pair/team/programmer has actually replaced the system under test with a test double. The test proves only that the mocks worked as expected.
• Generous Leftovers are left behind by tests that dont clean up after themselves and cause later tests to fail when run in the suite but not when run in isolation. The leftovers are typicall in static memory, on disk, in a database (for shame!) or some other persistent store. When leftovers are found, it's often puzzling whose they are, and what they are.
• Local Hero is a test that runs well, and tests a system well, but only passes on the author's machine or network. Being environmentally-sensitive, such tests fail for peers and CI systems. Typical failures involve hardcoded paths, locally-installed libraries, and OS-specific assumptions.
• The Loudmouth is a test that produces copious output. It is like the boor at the party who thinks every trivial event in his live is worthy of an epic tale. At one time, the loudmouth's story may have been worth hearing, but now it's just idle chatter that gets in the way of a real conversation with more interesting guests/tests.
• The Secret Catcher is a test that seems to do nothing at all, but is secretly (implicitly) depending on any errors to produce exceptions. The fact that the code executed without exception is expected to evidence that the code works. These tests can eventually be reverse-engineered by every programmer on the team. Failure of the test always results in code spelunking, an unpopular passtime.
• The Hidden Dependency is a test that secretly requires setup that is not present in the test itself. Perhaps it requires a certain data setup script, or a change in a configuration file, or another test to have already run with generous leftovers. Hidden Dependency tests are a special kind of evil.
• The Stranger is a perfectly good test in a perfectly wrong place. It is not a liar, it's just not testing the same system (SUT) that the other tests are testing. Strangers don't cause problem except when you're looking for tests for class X and it's hiding among the tests for class Y.
• Success Against All Odds is a test that will always pass, no matter what. Due to a series of missteps, the test won't fail even if the code is wrong. Often these turn into complicated indirect versions of "assert true == true" or "false == false". These can be a variation on Mockery or The Liar. It is likely that a test with Success Against All Odds was written as a green test, without ever having seen the "red" part of the Red->Green->Refactor loop.
• The Slow Poke is a test that takes too long to run. If you have 15000 tests, and you want them to run in less than 45 seconds, you have a budget of 0.003 seconds per test. A five-second test will cause some irritation. A few 10-second tests will make people think twice about running the tests at all. A test that takes a minute is unlikely to ever be run by a programmer. Imagine what hell awaits the author of some of the three-to-ten minute monstrosities that exist in the wild! In a TDD shop, slow pokes cannot be tolerated. Note that many slow pokes are also Giants with Excessive Setup. Be warned.

Tim recommends that readers follow the link to the original article, which covers more territory than the index card can allow.