Test SCUM


Font is "Pen of Truth"

Source is Ben Rady and Rod Coffin of Improving Works, given in an agile presentation at Agile2009.

In sharp contrast to the FIRST principles of good unit tests, Rady and Coffin give us these properties of completely bad (scummy) unit tests.

  • Slow: We discussed now important speed is in unit tests. I find that when tests are more than 40 seconds, people have to decide to run them, rather than feeling free to run them immediately after every small step. Rady's Infinitest runs tests automatically and analyzes the tests to selectively exclude those which have no (discernable) transitive dependency on the system under test, just to eek out some extra speed. While it is preferable to run all the tests all the time, in many circumstances culling the unrelated tests will give back some speed.
    Tests are generally slow because thy provide insufficient insulation from the environment. They tend to spend a lot of time waiting on clocks, file systems, databases, web services, or the like. Slow tests are best rewritten with isolation from the environment.
    How fast is fast? I find that I can tolerate a usefully large battery of tests if each of them takes about .002 seconds. I can tolerate any number that starts with a dot and two zeroes. I might be able to tolerate a very few with only one zero immediately after the dot. More than that is "slow". It doesn't take many multi-second tests before developers feel reluctance to run them.
  • Confusing: We have emphasized that good tests isolate errors and their causes. A confusing test is one that fails to isolate a specific error. This may be because it has too many things going on, or because it is simply unreadable. It is preferable that the combination of the test class name, the test method name, and the assertion gives all the information a developer needs in order to correct a misstep. In a confusing test, those three items are not enough an the entire body of the test may not shed light on the failure mode. Confusing tests are best rewritten as smaller, more clear tests.
  • Unreliable: Tests should be repeatable: they should always fail the same way, or always pass for the same reasons. Tests that run only in isolation or on certain days or times of day are not reliable tests. One of the best ways to judge reliability is to run tests repeatedly. Note that overspecification often causes tests to fail in unexpected and non-useful ways.
  • Missing: The tests that you don't run provide you no benefit, and quite a bit of harm. Tests help you to design your production code, and also help you to tell if you are making decisions that break the system in unexpected ways. Not having the tests means that you just don't know. Maybe your design is iffy in ways you hadn't noticed. Maybe you've broken other code with your last line of code. Maybe the behavior you hand-verified last hour is no longer occurring as you expected it to. The tests you did not write are the ones that are really holding back your productivity.
While we could list other unwanted qualities of tests, these seem to describe the most painful sins.

12 Principles for Agile Software Development


Font (once again): MechanicalPencil

The canonical source for these principles is the agile manifesto website.

These are the basic agile principles, abbreviated to fit onto a 3x5 card without requiring the reader to hold a magnifying glass. They are "sound bites" and not the whole story. Each of these principles can (or has) launched myriad blogs/articles, and indeed many of the other Agile in a Flash cards touch on these principles. We could easily build a distinct card for each, though that would inflate the size of our eventual card deck quite a lot.

It has long been Tim's perspective that Agile is about having a short reach so we are allowing that point of view to color our summary of agile principles.
  1. Satisfy the customer through early and continuous delivery--We shorten the distance between requirements gathering and customer feedback. The period is shorter because we plan less change at a time, and in return we get more opportunities to steer the software in a direction the customer appreciates. Notice that the principle actually says "continuous delivery", not just "quarterly" or "bimonthly." Early and continuous delivery is not about working faster, only about working sooner.
  2. Welcome changing requirements, even late in development--We shorten the distance between conceiving and implementing an important change. We don't have to wait for a redesign of the whole system, or for the next system to be built. This is not to say that no features will be delayed; feature requests are frequently reordered or even dropped. The agile difference is that such changes take effect sooner.
  3. Deliver working software frequently--We shorten the distance between the system-as-designed and the system-as-built. Both will evolve as we learn more about the system as it should be built. We also shorten the distance between planning and delivery, giving more opportunity to improve the efficiency and effectiveness of our work.
  4. Business people and developers work together daily--We shorten the physical distance between a question and its answer. Moving the customer to a different building, area, room, or even to a cubicle just around the corner dramatically reduces the number of questions we ask the customer. With collocation, the business and technology sides learn to better understand each other and to make more mutually-beneficial decisions.
  5. Build projects around motivated individuals--We shorten the distance between intent and action. The goal is to build an agile team of skilled professionals who care about all the business concerns including schedule and content, and who will work in a highly-engaged and result-oriented way. Such individuals need no babysitting and very little direction. Such a team will refused to be blocked, and will produce their best work at all times. Management does not have to spend time on motivational issues or "cat herding" so common to less-motivated teams.
  6. Convey information via face-to-face conversation--We shorten the time between a question and its answer. Agile teams work in bullpens because it makes it much easier to ask questions and offer suggestions. Things that should be communicated get communicated, not forgotten, diluted, or otherwise insufficiently communicated. While there are many attempts to go agile without co-locating team members, we are not aware of any which have the kind of success which is typical in co-located teams.
  7. Working software is the primary measure of progress--We shorten the distance between thinking we're done and knowing we're done. The team should be judged by the product it produces, not by the rate of typing or number of degrees, or hours worked per month, or how quickly the members walk from the parking lot, or how quietly they work from their individual cubicles. A good team frequently produces quality software the customer wants. All other measures are subordinate or irrelevant.
  8. Maintain a constant pace indefinitely--We shorten the distance between productive bursts. This doesn't mean that management sets a large minimum hour limit for developers to endure months or years at a time. Excessive overtime cannot continue indefinitely without severely impacting quality. Instead, we choose a pace that allows us to go home tired and satisfied in the evening, and then return fresh and ready to rock in the morning. For normal people with families and bodily needs, that pace will not be 90 hours a week, and it probably won't even be 50 hours a week. We are never impossibly far from our life-sustaining relationships and activities.
  9. Give continuous attention to technical excellence--We shorten the distance between implementation and ideal. An agile developer is never more than a few minutes away from the last time all the tests passed. Collaborating classes are not at the opposite ends of long chains of contains/references/inherits relationships (e.g. "train wrecks"). Developers need not wait to clean up redundant or confusing code. If working code is the measure of agility, then excellent code must be defined as code that accepts changes gracefully. An agile team takes steps to ensure that code gets better with each iteration.
  10. Simplify: maximize the amount of work not done--We shorten the distance between comprehension and completion. We eschew things that don't matter. If we're less encumbered by unhelpful tasks and unwanted features, we've shortened the reach to useful work. We also attempt to simplify code (reducing the amount of reverse-engineering other programmers must do). Agile programmers tend to follow the rules of simple design.
  11. Teams self-organize--We shorten the distance between need and action. We don't wait around to be told who does what. We do what needs to be done, not waiting for direction or supervision. We attack problems with fervor, mitigating risks and clearing obstacles.
  12. Teams retrospect and tune their behaviors--We shorten the distance between introspection and adaptation. Improvement is never far away. Each iteration, we explicitly find ways to improve process simplicity, code quality, technical excellence, and predictability of results. We analyze problems and obstacles, and look for root causes and their solutions. We actively plan to use better techniques, tools, and process flows. We act on the plan in the subsequent iteration, without delay.

These principles are fairly simple in concept, but are profoundly deep in practice. If you are transitioning to an Agile work style or are looking for ways to improve your current Agile practice, we suggest you begin again with the principles espoused here.