Coding Standards

Coding standards? That tired, old chestnut? Who talks about coding standards these days?

You do. You are talking about coding standards when you see someone has their tabs set wrong, when you complain about bracing, when you ask someone why they put their spaces where they do, when you add comments and someone else says delete them, ...

The need for a common style guide was glossed over in Collective Code Ownership, in the following paragraph:

The team has a single style guide and coding standard not because some arrogant so-and-so pushed it down their throat, but rather the team adopts a single style so that they can freely work on any part of the system. They don't have to obey personal standards when visiting "another person's" code (silly idea, that). They don't have to keep more than one group of editor settings. They don't have to argue over K&R bracing or ANSI, they don't have to worry about whether they need a prefix or suffix wart. They can just work. When silly issues are out of the way, more important ones take their place.

So what kind of document is the code standard? The authors have seen plenty of large, complex, detailed guides that strive to be comprehensive. Who has the time? Perhaps the answer should be to look for the least documentation one can afford. How much is too much? How much is too little?

The first recommendation is that a team should standardize to avoid waste. In this case, "waste" includes rework, arguments, and work stops while issues are settled. In this regard, it is better to have even a bad decision than a variety of opinions. If we find that we are enduring waste, then we add a line to the standard. Again, we have to build collective ownership and any choice is better than arguing the same points repeatedly.

The team should start with an accepted community standard if one can be found. If there are multiple, choose one of them. For the python community, PEP 8 is a wonderful starting point. One may look at a ubiquitous tool's default styling as a community standard (community of tool users), since it seldom pays to to fight your tools. Notice that public style guides tend not to be the smallest document one can afford, but the goal is really to make development time productive and non-contentious (with regard to silly issues) so even a longer document may be successful.

If there is some particular contentious difference of opinion, then the team should time-box the argument. They might choose to debate for an hour, agree, and write it down. Some teams don't time-box the argument, and others do not reach resolution. Do not let contentious issues remain, or they will remain contentious. Once agreed and recorded, the issue should not be revisited. Nor should a pair partner allow his partner to spend iteration time arguing over decisions already made. Nobody has to love the decision, but they should admit to a fight well fought and a final answer that should not get in the way of getting real work done well and often.

Ideally, the standard should be minuscule. A standard-by-example will have a better chance of being concise and obvious. Therefore, the standard should be mostly code and fit on one page. This is especially true of the initial version of the team's style guide. To some degree, arguments and uncertainty will lead to the accumulation of additional guidelines but it is wise to start small. Our working documents should be like our code in the sense that it is as small, simple, and unambiguous as possible.

The style guide is an agile artifact. it is subject to corrective steering. A team should revisit the standard every iteration until no one cares any more. Iteration time is too valuable for these arguments, but retrospective time exists to help the team eliminate waste and turbulence. If further discussion of style points will help smooth the coming iteration, then it should be brought up.

The rule for simplifying any system is to obviate and then remove steps, processes, and instructions. If the code is written to standard, then the code ultimately becomes the standard and the standard becomes redundant. It is perfectly reasonable for a mature team to discard their documented style guide and keep the style. You will know the style guide is unnecessary when all new code looks like the existing code. And it all looks good.

Top 10 Agile Failure Factors

In working with teams attempting to be agile, we tend to see them make many of the same "mistakes" over and over. This list captures some of the most common pitfalls we've seen teams stepping into.

Please take a moment to help refine this card by completing a short poll. It'll be open for a week, and we'll feed the results back into this list--both to help prioritize it and possibly swap out some factors for others. If you want to challenge any of the elements, you can of course add your thoughts in a comment.

Courage

Sources: Beck, Kent. Extreme Programming Explained, Addison Wesley, 2000; Ottinger, Tim; Langr, Jeff.

When I first started teaching XP, I remember having more than enough to say about the other three XP values (communication, feedback, simplicity). Courage? Uh... well, you know, you need it to be able to stick to your guns with respect to the other three values. Moving on...

So I particularly like this card for its specifics on when you need to be courageous. It also highlights the opposite dependency of courage on the other three values: An environment where you're not communicating, acting simply, or generating lots of feedback is going to make you feel like Tommy (or Helen Keller) piloting a helicopter.

• To make architectural corrections - It takes little courage to put a model on paper and make everyone sign their name in blood, insisting that we cannot change anything going forward. We're all courageous about the distant future (which we might never be a part of).


• To throw away tests and code - "I worked on that mess all day long, are you kidding?" Often the best results are produced by discarding a poor solution and reworking it. Even short-term, this takes far less time than most people think, and long-term it usually returns many times over the modest rework.


• To be transparent, whether favorable or not - It's so easy to hide in your cube, or to use a long, drawn out development period that makes it seem like real progress is being made. Yes, short iterations can make it obvious that you know nothing about things like getting the software successfully integrated in a timely fashion.


• To deliver complete, quality work in the face of time pressure - Push back. If your manager tells you, "never mind with the testing, just get it out the door," push back. "We don't have time to pair or review this iteration." Push back.


• To never discard essential practices - Ditto, from the last one. If your manager says, "you're not allowed to refactor the code," push back. You will pay for omission of essential activities, to the point that having done them at all in the first place will have seemed like a waste of time. (In other words, if you're going to sling code like a cowboy, just do it and stop pretending you're agile or at all professional.)


• To simplify code at every turn - Does this really take courage? Fortitude, maybe. Perhaps the courage here is learning to accept that your code probably does stink, and that you can almost always improve upon it.


• To attack whatever code the team fears most - The usual reaction is to tread lightly. What's particularly interesting is that treading lightly can often lead to the worst possible design choices. For example, I need 3 lines of alternate behavior in the middle of a 2000 line method. My fear leads me to believe that the safest thing is to copy the entire method and change the three lines in the copy. Courage would allow me to refactor to a template method pattern or some other solution that resulted in only three additional lines of code.


• To take credit only for complete work - The business can't use software that's 99.99% complete. If it doesn't do what the customer asked for, it's not complete. The courage here is to accept that incomplete work delivers no value, and admitting incomplete work must be reflected on the very visible plan.

Courage is excessively important with respect to communication in an agile environment, so there are many more specific elements that we could add here. It requires courage to speak up about the challenges we face on a development effort. Retrospectives, for example, are not at all useful without the XP value of courage.

Red-Green-Refactor

Drawing: Tim Ottinger

Photo: Libby Ottinger

Cleanup: Jeff L.

We show flash cards to students in order to help them completely ingrain a concept, to the point where they don't think about something, they just know it. The classic flash card presents a student with a vocabulary word or a math expression for which we expect almost immediate recognition and response. I show you the Spanish vocabulary word "acuario" and you blurt out "fish tank!"

So, red-green-refactor. By definition, TDD says write the tests first. They should fail, since you've not built the functionality that the tests specify, and a GUI test tool will show red at this point. That's useful feedback that tells you to write just enough code to get all existing tests to pass; the GUI test tool shows us green. Finally, you can ensure that the code has an optimal design during a refactoring step, since you have tests to give you the confidence to change things. Spend a few minutes, improve the design, and re-run your tests, which should all still be green. The entire cycle should take about 5 minutes on average, and no more than 10 minutes.

If you've been doing TDD for more than a few days--I mean really doing it and not writing code first and then sneaking in a few tests afterward and then telling people "I got muh TDD's done"--this cycle should be starting to sink in. Those of us who've done TDD a little longer--maybe a few months--don't think twice about it. Red-green-refactor feels like the natural way for us to build software.

• Red - A common mistake for newbies to TDD is to gloss over the need to see the test fail first. It's part of keeping on course; the red is extremely valuable feedback that tells us our assumptions still hold true. Once in a while, they won't, and you'll have saved yourself a lot of time by finding that out immediately.


• Green - Keep rough track of how long it takes you to derive a solution that passes all your tests. If you're taking 5-10 minutes on average, or more, start figuring out ways to take smaller steps.


• Refactor - You should always take advantage of the refactoring step. Even if you added perfect code (that doesn't duplicate any other code in the system), treat the refactoring step as "mandatory bonus time." Poke around the area you're changing. Get rid of a warning. Rename a test. Improve the readability of an existing method. Follow the boy scout rule: Make things a little better when you leave than they were when you arrived. Note that the refactor step doesn't necessarily represent a single green test run: You should look to decompose refactoring efforts into even tinier steps, getting a rapid succession of green bars before moving on.

Writing Characterization Tests

Source: Feathers, Michael. Working Effectively With Legacy Code (WELC), Prentice Hall PTR, 2005.

Test-after development (TAD) is tough; dependencies and non-cohesive code can make for a significant challenge. That's why Michael Feathers' book Working Effectively With Legacy Code is so useful.

Does working with legacy code really require a different set of skills than doing TDD? Well, sprout method and sprout class are the first avenues you should always explore, and they are strictly about test-driving the new code. But most of the other WELC techniques require less test driving and more digging about, trying different things. Some of the techniques even border on being too clever--they're about problem solving, and you do what you gotta do!

An important step in modifying existing code is understanding what it does before you make any changes. Feathers says characterization tests are like putting a vise around code: You want to pin it down before you attempt changing it, so that you know if it slips a bit when you do. So in a sense, we are back to test-first.

How many characterization tests do you need to write? You can read the WELC chapter on effects analysis, and start to get scared that you're going to have to write a lot of tests. Or you can use confidence as a guideline, writing as many tests as you think you need to understand the code you're about to change.

The flash card describes what are pretty obvious steps, but they back one of the underlying themes in Feathers' book: be methodical, be safe.

Story Format

Well, it had to happen sooner or later: a card that bothers me quite a bit. Mike Cohn's book Agile Estimating and Planning, a book I recommend for just about anyone working on an agile project, popularized this story card format.

"As an actor, I want to accomplish some goal, for some reason"--very similar to what we did with use cases in the early 90s. And back then, we spilled time bickering over format and wording, so I give credit to whoever came up with a template to help sidestep those wasteful arguments.

The problem is that the story card ain't the thing, it's a placeholder and "promise for more communication," per Ron Jeffries. The card could have one word on it, and that would be sufficient. Bob Koss often told a story about a story for the U.S. Navy regarding how to adjust large gun firing based on sonic feedback. The story card said simply "boom splash!" That was enough to initiate talking about it, and no one forgot who the story was for, what it meant, or why it existed over the course of the iteration.

I realize I'm railing against a potentially useful flash card. As a tool, the format template can help us considerably in improving our collections of stories. Some same lessons from use cases apply: Thinking about the actors involved (the "as a" people) can help trigger the introduction of important stories that might be missed. The phrase "I want to" reinforces that stories are goals for customers the system, and not just technical pipe dreams. And "so that," well, as we write out dozens of stories in release planning, we'll want to remind ourselves of the rationale behind certain stories (but not all of them!). And sometimes this "why" can trigger other useful considerations.

To be fair to Cohn, he says many similar things in his book. But you know how people are. I've already encountered spreadsheets and other software tools that rigidly insist on this format. Never mind that the actor is the same for every last story, or that the reasons are pretty obvious for most stories, or that typing all that stuff is just redundant crud that we would stamp out if it were in our code.

Remember: The cards we present as part of the Agile In a Flash project are tools, not gospel. They're here to help prod you if you get stuck, to give you ideas, and to give you guidance. In most cases, you should follow them unless you have darn good reasons--and even then, you should follow them until you know why the rules exist, and only then should you consider taking your own path.

Breaking Down Larger Stories

Source: Jeff Langr, Tim Ottinger; also, Cohn, Michael. Agile Estimating and Planning.

One ideal for agile development would be to be able to deliver a "done done" story every day. We get a lot of resistance on this thought, and we push right back. Our goal is not to insist on the ideal, but instead to get you to move toward that goal, and not to defend stories that we think are too large. "Too large?" There's no consensus. Anything taking over a half iteration undoubtedly should be scrutinized.

One way to deliver stories more frequently is collaborate a bit more. Instead of one developer working a large story alone (sadly common), consider adding another developer or two or three. If they can help deliver the story sooner without inordinate overhead costs (including coordination efforts to avoid stepping on each other's toes), go for it.

The other route is to split stories. Breaking down larger stories is an often challenging proposition, but the more you do it, the easier it gets. Next time you have a large story, step through the list on the card.

The most challenging stories are "iceberg" stories. These are stories where the customer sees only a small impact, but the algorithmic or data complexity required to implement the story is large.

Sometimes a split isn't worth it. For example, you look to consider an alternate case as a separate story, but the developer says, "well, it's only going to take me ten minutes to implement that."

A key thing to think about when looking to split stories is the tests.

• Defer alternate paths / edge cases - If anything, thinking about how to split stories around alternate cases will force you to make sure you've captured all of them!


• Defer introducing supporting fields - If the story involves user input of good-sized chunks of data, support a few key and/or significant fields and introduce the remainder later.


• Defer validation of input data - Demonstrate that you can capture the information; add the ability to prevent the system from accepting invalid data later.


• Defer generation of side effects - For example, creating a downstream feed when the user updates information.


• Stub other dependencies - "Fake it until you make it."


• Split along operational boundaries (e.g. CRUD) - This is directly from Cohn's book. Note that you don't necessarily have to have "C" (Create) done before you implement "U" (Update).


• Defer cross-cutting concerns (logging, error handling) - This generates an interesting challenge around an important point--devising acceptance tests that verify the addition of robustness concerns.


• Defer performance or other non-functional constraints - Sometimes it's possible to bang out an implementation using an overly simplistic algorithm. A similar challenge: How do you devise acceptance criteria for the performance improvement?


• Verify against audit trail that demonstrates progress - I'm not enamored with this one, but sometimes there are no other obvious solutions. Sometimes this will expose more implementation details than you would like. Perhaps these tests can disappear once the entire story gets completed.


• Defer variant data cases - Will it simplify the logic if we don't have to worry about special data cases? Or more complex data variants? For example, it's probably a lot easier to devise a delivery scheduling system that supports only one destination.


• Inject dummy data - In some cases, data availability or volume can be a barrier to full implementation of a story.


• Ask the customer! - You may be pleasantly surprised!

Well, this list isn't necessarily complete, and needs a bit of work. What other story splitting mechanisms have you used successfully?

Refactoring

Refactoring is a large topic with enough material to fill a few books. In fact, it has filled a few good books. Clearly we don't have room for all of that on an index card but at least we can provide some meta-guidance on the topic.

Done only with a green bar (all tests passing) never with failing tests. No matter how much you want to rename, extract, or modify the code you really should have green (even an ugly green) before you try to refactor.

Always run the tests before and after the refactoring steps. Make sure you know that the system worked before and after each step. Check before, so you aren't confused about origin of breakage, and after so that you know you've done a good job. Skip this at the peril of having to use debugging or having to revert a set of changes. Always have a single reason to fail.

Refactor incrementally: have only one reason to fail. This cannot be over-recommended. Michael Feathers describes programming in his book as "the art of doing one thing at a time", which has been a very good bit of advice through the years. There is a good reason that the refactoring book gives recipes of tiny steps. This incrementalism is worthwhile and hard-won habit.

Add before removing when you are replacing a line. For instance, if there is a complex calculation and you are replacing it with a function that calculates the same result, place the new call & variable assignment right after the old calculation. Run the tests. If the result is different, comment out the call and try again. Don't delete the original until you know for sure that the new code is equivalent.

If you move a method to a new class, you must move or create a test. Otherwise your tests become increasingly indirect. Indirect tests do not produce a good smell.

Changing functionality is not refactoring. That would be correction or implementation. Calling a functional change a "refactoring" is not entirely honest. Refactoring does not change what the code does, it only changes where the code is located.

Increasing performance is a story, not refactoring. This is another misuse of the term "refactoring." The point of refactoring is to improve the structure of the code. While performance can accidentally result from simplifying the code, it is the simplification and not the performance that is the object.

The books outlined above will give a better idea of how to perform refactoring. At least the index card can help remember the circumstances in which one should perform the task.

Collective Code Ownership

See Extreme Programming's writeup on the rule of collective ownership.

Next to pairing, the Agile practice that causes consternation among developers is the idea of collective code ownership. It is not a difficult concept to grasp, but individual ownership is a difficult idea to let go. Developers inclined or acclimated to working solo find this four-bullet card quite threatening, and the motto above it positively chilling.

But what if we were truly working as a community? We would give up having the final say about how our code is formatted and how variables were named. We might find other people's pet peeves getting equal say. We might find that other people can take over the code we wrote. We might be able to be sick for a day or go on vacation without work ("that only we can do") piling up. We might be able to simplify the team's work by having a single work queue. We might find that we can help make other people's code better. We might find that they can improve ours. But we have to get over the idea that the code is ours.

It's not ours. It really belongs to the customer or company the second it rolls off our fingers. It won't follow us home, and we can't take it with us when we leave. It is copyrighted by the company as a work for hire. Let's not lose sight of that nugget of reality.

Anyone can modify any code at any time so that nobody has to seek permission to simplify and speed the code, or to add features that might touch code they don't own. Tests can be written because code can be made testable. This frees the development team from fear of a certain variety of personal conflict getting in the way of getting real work tested and delivered.

The team has a single style guide and coding standardnot because some arrogant so-and-so pushed it down their throat, but rather the team adopts a single style so that they can freely work on any part of the system. They don't have to obey personal standards when visiting "another person's" code (silly idea, that). They don't have to keep more than one group of editor settings. They don't have to argue over K&R bracing or ANSI, they don't have to worry about whether they need a prefix or suffix wart. They can just work. When silly issues are out of the way, more important ones take their place.

Original authorship is immaterial because "it's my code" has never been a valid reason to avoid improving or modifying a system that doesn't work or which doesn't do enough. In agile teams, we try not to talk about the original author when we are working on code. It simply is. It had an original intent, and we hope to strengthen it and make it more clear. The code exists and works. We want it to work better. This focus on the behavior of the code over the personality of the author is freeing as well. We can learn together once we let the guards down.

Plentiful automated tests increase confidence and safety so that we can learn while coding. If code is test-driven, every violation of the original intent (as expressed in the tests) is met with a test failure. Rather than pre-worrying every possible code change, the developer pair can make progress in short order and can refactor without fear. This is better for the company, better for the programming pair, better for the individual.

Reading over this card, I wonder if we missed the more important point that work does not have to queue for individuals. Too often we find work piled up for one or two siloed developers while two more are nearly idle because their silos are not busy. This is inevitable when our team structure involves multiple queues, and it is a shame. If we broke down the silos and followed pairing, testing, and collective ownership practices for just a little while we would find ourselves in a situation where any number of developers would be able to pick up the jobs that are waiting.

Examine a few common types of situations:

Scenario 1: Fred can produce automated filing reports in a day. Abe and Margaret together could get it done in two and a half days.. It makes sense to have Fred do this work, yes? But it's Wednesday, the work is due Friday, and Fred has 16 hours of more important work that is also queued for him. Now it seems foolish to assign Fred, because it will cause us to miss a release or else Fred will work copious overtime while Abe and Margaret go to the bar at 3:30pm. A good deal for Abe and Margaret, bad for Fred and risky for the company.

Scenario 2: Fred has three days of work to get done, and only two days to complete it. Nobody else can help, because Fred has always maintained this system alone. His wife calls to tell him that their newborn has pneumonia. Should he work late and fail as a father, or go home and fail as an employee?*

Scenario 3: Abe and Margaret already have worked with Fred in the automated filing area many times over the past three months, and feel pretty comfortable there. There are a lot of tests, and these tests help them to avoid checking mistakes into the code line. Fred has three days of work to be done, and a new two-day assignment was added to the pile. Margaret feels pretty comfortable with the new work, and should be done in two days. Fred and Abe team up on the remaining work, and at the end of the slightly-longer-than-usual day they're half-done. They should be able to finish up tomorrow. Fred's had bad Thai food. He begs off and dashes home. Charlie comes in to pair with Abe and Margaret as needed so that the release can finish on time.

This is an argument not only that a team should pair and explore the project's code, but also that they should be doing so at all times. The team should always be able to cover for members who are incovenienced by family emergencies or personal illness.

Some common objections:

It is dehumanizing and will reduce good programmers to undifferentiated ”cogs”. It has been our experience that it improves the human side of the operation, and that great programmers are not great programmers because they keep a walled garden of code working, but because they are good at programming. Why hide greatness under a bucket? Why limit a great programmer to a small corner of a system if you want the system to be great? Wouldn't a great programmer be better recognized by peers if they get to share in his great work?

It won't work in a shared trunk/branch. It does, all the time. The developers will share code more often, which means that they will integrate changes into their branches sooner rather than later. By sharing more, they have smaller and simpler integrations. I

It doesn't work in the face of refactoring and renaming code. You would think, yet somehow it still does. Common code and frequent integration make this largely a non-issue. And if people must work in the same small area of code, they can locate closer together and talk over their changes on-the-fly to make integration easier yet.

It will be hard to get any design done. A team does settle into shared design with surprisingly little thrash. The idea that good design is the work of a single enlightened individual and that it cannot be understood or appreciated by his peers is largely unsupported. An incremental project does small acts of design all of the time, and often hold impromptu stand-up meetings to talk over changes that impact the system on a larger scale.

You can't really get people to do it. However unlikely it seems, many work precisely this way each day. More surprisingly, people come to truly love the flexibility and expanded influence that collective code ownership gives them. This is particularly true in agile shops where they practice pair programming and unit testing.

*Rhetorical question. You always choose the child. Of course, you may still face dire consequences at the job. Personally, I would hate to be the manager who has to tell clients that he can't make the release date because he only has one programmer competent to do the work. Who should really be in trouble here?

INVEST

Source (tentative): Cohn, Mike. Agile Estimating and Planning. [Anyone know the original source? ]

The original XP take on what made a good story was that it had to have business value, it had to be estimable, and it had to be testable. That might have made for a simpler and also appropriate acronym, VET. But the three newer elements (I, N, S) add considerable value in helping you shape a candidate story into a real story that can be accepted in an iteration.

• Independent - Any story could be the next one done; the customer should have the final say. As stories complete, some may become cheaper and others more expensive. Tim recommends estimating all stories as if they were first, and re-evaluating estimates before iteration planning game.


• Negotiable - A story is not a contract! It is a "promise for communication," as we used to say. It shouldn't be flush with every last detail.


• Valuable to the customer - Don't create technical stories! A primary goal of using stories is to demonstrate to the customer that we can deliver business value incrementally, so that the customer can help steer us and provide us with feedback. I'll repeat, because it's very important: Don't create technical stories!


• Estimable - A story might be impossible to estimate if it's too big, or if we have no idea what's involved, in which case we probably need to go off and to a bit more research before we present this story.


• Small - Some sources replace "small" with "sized appropriately." Size will vary on your shop, but obviously, a story can't represent more than a single iteration's worth of work. A better rule of thumb would be that no story would represent more than half the iteration. To me, an ideal size would mean that your team could crank out a story every day. It is possible to make stories too small, but very rare, so the general rule is "smaller."


• Testable - If you can't verify it in some manner, how do you know it's done?