Priority Mechanisms

Font is Marker SD

Setting priority for tasks is essential to agile/lean software development. Agile teams strictly limit the number of tasks in flight in order to have greater efficiency and effectiveness. The improved focus means code is completed sooner (not faster) and that product may be delivered to production more often. This focus is frustrated if we have too many "#1 priority" items and "emergency" items thrown in on top. Controlling the flow makes teamwork possible, and actually speeds the process.

We sometimes refer to the system that filters tasks into priority order as the magic funnel. All tasks and features are metaphorically dumped into the top of the big funnel. Ideally, the first (or next) item that falls out of the bottom of the funnel is the absolute most important thing we could possibly be doing. It has won the prize. It is the task we should be working on.

Setting priority is unsurprisingly one of the hardest things a customer can do. Do you pick the story that satisfies the most "important" customer (where "important" varies daily, almost hourly, and by which person you ask), or the greatest number of customers? Do you pick tasks that you think will bring the greatest number of new customers, or please the existing customers? Do you streamline processing or go with aesthetics? What makes one story more valuable than another?

Since the problem is large, and depends on local standards, there are many ways of prioritizing stories. The method shouldn't be something a team should agonize about for weeks. The most important point is that the product owners are able to reach a consensus, and that the features they pick are truly important. Tactics can vary, provided the strategy is good.

Here are some schemes and techniques that might help build a working "magic funnel":

• A simple priority triage system (one short sentence) was reported by Mary Poppendieck in Lean Software Development as one of the simplest things that could possibly work. The rule doesn't have to be subtle or obscure. A maintenance team might adopt a three-part rule, where production crises came first, followed by bugs on the release-in-testing, followed by everything else. Features that affect all customers may come first, followed by those that primarily affect million-dollar customers, followed by those that affect half-million dollar customer, followed by whatever. It is possible that product stakeholders already have a rule of thumb that suffices 80% of the time. The other 20% of the time is settled through escalation.
• Determine "The Constraint" as in Goldratt's writings The Goal and Critical Chain. If you can define the one thing that inhibits you from being successful, then you can prioritize all outstanding bugs and backlog items in such a way that those that remove the primary obstacle become top priorities. Goldratt was speaking of process steps and production, but why not apply it to features that sell or flaws that inhibit? It may be that scalability of your web application is more important right now than enhanced functionality. It may be that a difficult feature is the primary reason you are losing sales or even customers. It may be that one issue prevents you from expanding, growing, producing. Critical Chain shows us that we need to solve this one big problem. And then the one after it.
• Covey's Quadrants from First Things First will have us divide work into quadrants, so that #1 are those things that are both important (to the future of the company or product) and urgent, #2 are those that are important but not urgent, #3 are those that are urgent but have little long-term impact, and any unimportant non-urgent work is simply eliminated from the backlog. These items are taken in quadrant order, so that important, non-urgent tasks are done before urgent, non-important tasks are considered. Covey, et al, were talking about personal importance and daily tasks, but why not apply it forward to planning other work that has potential value and obvious levels of urgency? Remember that urgency and (to a lesser degree) importance are dynamic properties, to be revisited frequently.
• Value-First is the rather like the Covey and Goal approaches, where the work that provides the greatest value (frequently represented in customer satisfaction or decreased support) goes first. This sounds easy, but of course the problem is defining "value" and dealing with "value to whom?" One could place value to the customers or to the sponsors or to the shareholders first.
• Risk-First would tell us to start on those things that need the most research and testing sooner. High-risk items have unpredictable schedules, and may not pan out at all. Such stories would be prioritized earlier because they cannot reliably be produced on a schedule. Alternatively, some will choose to use risk-last planning, so that the so-called "low hanging fruit" may be completed before moving on to less predictable work. Some organizations may want to plan a mix (70/30 or 80/20) of low-risk to high-risk features. 
• Greatest Good to the Greatest Number works like the other "value first" kinds of schemes, but adds voting to the scenario. Each customer, role, or market segment that would benefit from a change votes for it (consider dot-voting). Among items of the same importance/urgency quadrant, those with the greatest number of votes would go first. Changes with fewer stakeholders would be done later. This is a customer-focused version of "Bargain for Success."
• Fixed-length action queue would work by using a short list of only a few bug fixes or marketable features. This list would be overseen by executives, and to push an item from "backlog" to "action queue" requires high-level approval. Items in "action queue" can be retracted only by executive order, since stopping a feature-in-progress represents a financial decision to throw away development dollars. The customer works to push items into the fixed queue as other things are completed or redacted. Customers will have a list of two or three items that might be next if approved. This tight focus helps the Customer not lose focus on immediate delivery. It is a lot like the Biblical "Law of the Medes and Persians", famous for being unrevokable.
• “Only One Thing” is a little game best used when the priority rule is more intuitive than explicit. The stakeholders are given a hypothetical situation that only one feature can be reliably developed in the next iteration/month/quarter, and are tasked with picking which one feature it would be. Once they have picked the solitary feature, they are asked why it is important to them. This game can be played with "only three" or "only five" for a larger sample size. Most organizations, having argued through the exercise, will have a Simple Triage System they can use for future priority meetings.
• Bargain for success is another game-like system. Each product stakeholder is given a number of votes/dollars/tokens/chips to spend. They are allowed to buy into features in order to bring their priority up in the queue. Stakeholders can make deals with each other ("I'll give 7 tokens to your feature Y this time if you'll give 7 to my feature Z next month"). The game-like nature of the exercise may cause some to see personal "winning" as the goal, rather than the greatest good for the company, but it will drive consensus nonetheless. Ultimately, the stakeholders have decided either way.

Some of these techniques (Bargain for Success, Only One Thing, Fixed-Length Queue) may qualify this card as a "Power Card". They are not guaranteed to result in happy handshakes and back-patting among all stake holders. Some may even resent their work being treated as a game (though in my experience they usually prefer to work in a game-like system with some rules and a fair degree of bargaining authority).

The ultimate power card (not included) is "CEO/Owner Demands". Most development organizations will pitch in and do all they can to get the feature completed well and soon. As long as the demands are within reason it should not be a problem. Most executives should understand the realities of development hours and sustainable pace, and are pretty hard-working guys themselves. The CEO DEMANDS power card is rightfully used sparingly and in cases most deserving of it.

Overplayed, all power cards will result in demotivation, loss of quality, slowed pace, and/or attrition. Coaches, managers, stakeholders, and product owners should move cautiously.

Principles of Package Coupling

Principles: Robert C. Martin

Post: Tim Ottinger and Jeff Langr

These three principles, devised by Uncle Bob, describe how to structure relationships between packages (arbitrary collections of classes). A related (forthcoming) card, Principles of Package Design, talks about the forces that bear on how to optimally group classes.

There arise occasional shouts of "But that was in C++!" Yet we find that these principles can apply also to dynamic languages, where abstraction works rather differently than in interface-declaring languages.

• Acyclic-Dependencies Principle (ADP) - You stay late to finish up some work, but come in the next morning only to find that your code is broken--someone else stayed later and checked in changes that make your code very unhappy. Fixing your code breaks a third developer's code, and his fixes break yours again. This "morning-after syndrome" can be an unending nightmare of changes breaking other code, which in turn must be changed, which in turn...
  
  Cycles begin to make all packages dependent upon all other packages in the system. They dramatically increase build times to the point of pain (something one of us lives with daily on the C++ project he's working).
  
  Some languages, like C#, have facilities to prevent circular dependencies among packages. But even modern, dynamic languages like Python can suffer in surprising ways from round-trip dependencies.
  
  Dependency cycles among modules are a very serious problem, slowing development and wrecking deadlines. Note that mutual dependencies among classes are usually a problem if the classes are split across modules/packages/assemblies.
  


• Stable-Dependencies Principle (SDP) is based on the understanding that code has volatility, and that changes ripple through a system along dependency lines. From the point-of-view of volatility, dependency is transitive.
  
  If the majority of the system's classes are dependent upon a volatile bit of code, then we can expect a high frequency of system breakage and rippling changes. If, instead, a system depends on nonvolatile code, then breakage should not ripple through the system very often at all.
  
  Some modules should change frequently, and should be volatile. Other modules are essential core abstractions and should not change very often.
  
  Therefore the package dependency graph should flow from instable packages (packages that are easy to change) to stable or "responsible" packages (packages that are hard to change). In a sense, this is the dependency inversion principle applied to packages: You should depend only upon things that are unlikely to change.
  
  
• Stable-Abstractions Principle (SAP) - Per Uncle Bob, this principle sets up a relationship between stability and abstraction. A stable package should be as abstract as possible, thus ensuring that its "stability does not prevent it from being extended." In contrast, an instable (not responsible) package should contain lots of concrete classes whose details can be easily changed.
  
  This further reflects the Stable Dependencies principle, with the added observation that the reason for interfaces is to give safe, stable dependencies to clients of an abstraction. This drives us to the understanding that we should depend on the parts of the system built specifically to provide freedom from volatile bits (implementations).
  
  In dynamic languages, SAP is a harder metric to automate. Clearly the principle still applies. There will be classes whose purpose is to provide a stable interface and other classes that provide easily-changeable behaviors. The only problem is that not having declared interfaces makes it harder to automate the process of assessing abstractness.
  
  

A "benefit" of working in C++ is that poor package coupling choices are all the more noticeable. Build times of several hours are not uncommon, yet these wasteful cycles can be quickly brought down with a bit of appropriate package reorganization. Without dramatically increasing build times, developers in languages like Java might not notice as quickly the cause of their morning-after sicknesses.

So how do we figure out where the problems are and what to do about them? The flip side of the card provides metrics that you can use as the basis for graphing the relationships between packages. Such a graph can point out nonsensical packages, such as maximally abstract packages that no other packages depend upon, or maximally concrete packages that all other packages depend upon. Uncle Bob's stability article provides some detail on how to put this graph together (but there are also tools that can help!).

Software Craftsmanship Ethics

Source: Doug Bradbury, Dave Hoover, Robert Martin, cast of dozens

This is the outgrowth of a conversation among Doug Bradbury and a number of Craftsman Manifesto signers. The goal of the collaboration was to come up with a statement of the Ethics of Software Craftmanship. In the course of the conversation, one of our participants suggested the pillars shown here (the brainchild of Jason Gorman). 

 The long form is quite beautiful, and presented here in entirety as an explanation of the four points above:

• We consider it our responsibility to gain the trust of the businesses we serve; therefore, we take our customer's problems as seriously as they do and stake our reputation on the quality of the work we produc.
• We consider it our responsibility to write code that is defect-free, proven, readable, understandable and malleable; therefore, we follow our chosen practices meticulously even under pressure and practice our techniques regularly.
• We consider it our responsibility to hone our craft in pursuit of mastery; therefore, we continuously explore new technologies and read and study the work of other craftsmen.
• We consider it our responsibility to perpetuate the craft of Software; therefore, we enlist apprentices to learn it and actively engage other craftsmen in dialogue and practice.

FURPS+

Source: http://en.wikipedia.org/wiki/Furps

Nowhere does it say that you must dispense with all prior knowledge in order to be agile. The FURPS+ card is a good example of a tool I've found useful in non-agile years long past, and still useful in the context of an agile project.

In agile, we don't refer to requirements; instead we talk about stories. Stories are really reminders for customer needs (requirements, in the general sense). We will refine these needs into product through continual discussion. We demonstrate confidence in our ability to ship software product by executing tests that represent acceptance criteria for each story.

Deriving a list of stories to represent a deliverable product is something that must happen in order to plan a release cycle. This backlog of stories must of course include all the customer's functional needs, as well as their non-functional needs. Non-functional? Many "real" customers don't think about such things.

The FURPS+ acronym, devised by Robert Grady of HP, provides a bit more meat around what we mean by non-functional stories, and also provides a good way to categorize such needs. The breakdown here suggests some representative questions around potential needs.

• Functionality - What the customer wants! Note that this includes security-related needs.


• Usability - How effective is the product from the standpoint of the person who must use it? Is it aesthetically acceptable? Is the documentation accurate and complete?


• Reliability - What is the maximum acceptable system downtime? Are failures predictable? Can we demonstrate the accuracy of results? How is the system recovered?


• Performance - How fast must it be? What's the maximum response time? What's the throughput? What's the memory consumption?


• Supportability - Is it testable, extensible, serviceable, installable, and configurable? Can it be monitored?

The + reminds us of a few additional needs that a customer could have:

• Design constraints - Do things like I/O devices or DBMS constrain how the software must be built?


• Implementation requirements - Do the programmers need to adhere to standards? Is the use of TDD required? Is statistically sound testing in the context of Cleanroom required?


• Interface requirements - What downstream feeds must be created? What other systems must this one interface with? How frequent are feeds produced?


• Physical requirements - What hardware must the system be deployable on? Must we be able to deploy to a machine no larger than 12" square, to be stationed in the Iraqi desert?

Just remember, in agile, stories are negotiable (but end product is usually not)!

Value Of Index Cards

Typographical search continues. Font is Marker SD.

Source: Langr, Jeff and Ottinger, Tim

Index cards are an important part of the agile experience. We use them as tokens for user stories, for task breakdowns (sometimes), for backlog generation, for estimation, for reminders, etc. Here at Agile In A Flash, we use them as cheat sheets and motivational posters. There is a surprising amount of value to having an agile card posted in your workspace. Print one of ours out, and try it for a month.

Index cards are great. Why?

• They are Low-tech and High-touch. You can put your hands on them, hand them to other people, stick them on the wall, tape them together, sort them on the desktop, etc. Having a tactile device such as an index card can make various workspace rituals possible (like moving a card from "in progress" to "done").
  
• They can be dynamically reorganized to order them by point cost, or functional area, or assigned teams, or originators. They can have connections one did not intend originally. This puts them a step above most software programs -- they are routinely re-purposed on the fly.
  
• They can hold schema-free markup. For instance, the security or documentation team can write notes on them. Developers and product owners can mark them up with issues or orderings. They can have priorities added to them. They might have none of the above, only a two-word name. They can have pictures, and notes with arrows. If you have a card and a marker, your only constraints are space and penmanship.
  
• They are reminders for much bigger ideas. Being small, they can hold very little content. Yet they can have vast evocative powers. Take the agile flash cards on this site for instance. Once you know a principle or practice, a card bearing as little as three words can help you do your job better.
  
• They are extremely portable, being small physical items. You don't have to copy them to a USB drive or mail them to your teammates. You can use them on the bus, train, or hiking trail. They don't mind being without web access and are immune to OS and browser incompatibilities. Whereever you are, they are the same.
  
• They are inexpensive and replaceable. Of course, you never want to be without your Agile In A Flash (tm) cards but most other cards can be lost without any real cost. If they are important, they can be easily reproduced. If not, maybe you didn't really need them. I think every day we can tear up a story card or two is a good day. They are not heirlooms to be maintained for the life of a project, and that makes them even better.
  

Shu-Ha-Ri

Images from Wikimedia

Font: j.d.

References: Takamura, Yukiyoshi: Teaching and Shu-Ha-Ri

Cleanup & edits: thanks Tim!

Kata are patterns of movements practiced by martial arts students; the moves act as the basis for actual application, i.e. fighting or self-defense. The notion of executing kata is analogous to a musical student practicing scales or finger exercises. Note that many professional musicians still practice scales, and professional martial artists still use kata.

Learning music, martial arts, the game of go, new software development techniques, or anything that requires extensive practice to master, can be broken into three levels, or stages, known as shu-ha-ri. These three words are (according to Ward's wiki) roughly translated as "hold", "break", and "leave."

• shu - In first learnings, a student strictly follows the instructor, replicating each move as precisely as possible. Per the late Takamura, "the student must first resign himself and his ego to a seemingly random series of repetitious exercises." Kata at this level (known as shoden) are specifically designed to get students to focus on basic movements. Some shoden are even designed to create "physical discomfort," in order to reinforce the student's need to master focus. Ultimately, the goal of shu is to ingrain basic moves in the student, so that they do not think twice when it comes time to apply them. Wax on, wax off. Or, "Red, green, refactor."


• ha - At this secondary level, the instructor begins to grant the student some leeway to experiment and diverge from strict application (but not to the point where the kata no longer resemble the originals). The goal is to open the students' minds to begin to recognize the true usefulness of the thing being mastered. The ha level is where we expect to start seeing light bulbs click on in peoples' heads. After practicing strict TDD for a while, students start to see some of the power of being able to move their code bits around. They learn when they can not test something because it "could not possibly break." Letting students diverge also allows them to note the value in things like consistent test naming, taking smaller steps, or exploring more sophisticated forms like BDD.


• ri - At the ri level, the student no longer views the rules as constrictions, but instead as having been the stepping stones to learning and freedom. In fact, the student no longer thinks about rules. He or she has ingrained a set of techniques that can be applied at a moment's notice, but has also learned to specialize that knowledge with additional experience-based elements. In some cases, this level may find the master abandoning Red-Green-Refactor to even greater success, for short periods of time. Don't try this before you're a master!

Rules were meant to be broken, but only after you've followed them for a while, to the point of intimate or intrinsic understanding of how they benefit you. Eventually, the essence of the rule has been internalized and the rule (as a rule) has been obviated. At that point, you are able to operate as if the rule were common sense. You are then free to deal with the repercussions of not following the rules.

CRISP

Brad Appleton provided this as a reader submission. Thanks Brad!

Mike Clark gave a 2004 presentation on Pragmatic Project Automation that included a description of what he called the "CRISP" criteria for build. There is a similar description in the 2007 presentation “All Builds are Good”, and a more detailed description in this 2007 CT-SPIN presentation on project automation:

Complete:
• Build from scratch and independently without human intervention.

Repeatable:
• Must be able to create exactly the same build at a later time.
• Store build scripts in source control.

Informative:
• "Detector of unexpected changes".
• Provide information on why a build failed.

Scheduled:
• Let the builds run automatically.

Portable:
• Build should be runnable from any system (same platform), not just that of the developer.
• For cross-platform software, it should build on all platforms.

Why Put Big Ideas on Little Cards?

"... certainly it is excellent discipline for an author to feel that he must say all he has to say in the fewest possible words, or his reader is sure to skip them; and in the plainest possible words, or his reader will certainly misunderstand them..."

John Ruskin

Metaphor

The font for this agile flash card is Lavos Handy

Of the XP practices, the practice of Metaphor seems to be the most widely argued and misunderstood. Simply put, it is a shared understanding of how the system should work. It serves as the oversimplification that "everyone knows" and against which changes can be discussed intelligently.

It may literally be a metaphor, such as a description of a hive of bees or an assembly line or blackboard, but it may also be a more literal description of the basic entities of the system and their interactions (ie not metaphorical at all). The shared understanding is the thing.

The metaphor is:

• A shared theory of system operation from which additional features may be discussed intelligently. If the system is like a spider's web, then we can talk about strands being moved and the spider waiting. If it is like an assembly line, we can talk about stations near the end of the line and the constant feed of parts. If it is like an accounting system, we can talk about double-entry accounting. The importance of the metaphor is partly that it provides a back story for further development.
  
• A model of the system's primary entities and flows can be sufficient shared understanding if the solution is relatively straightforward and can be easily-enough digested. In this way, the business domain of a sufficiently transparent system may be its metaphor(!).
  
• A shared system of names is provided by the metaphor. In a beehive system, workers and drones and queens may serve as class names or database tables. In a petri net, it may be tokens and places. I personally suggest a more straightforward naming style, but a metaphor can certainly be a rich source of meaningful names.
  
• May be an actual metaphor or story that is well-known to the development team. If the system can be well-understood by comparison to a nest of termites or a storm system or trash collection then (as far as it goes) the metaphor can be at the core of all implementation decisions and requirements discussions.
  
• May be replaced or rejected when the system, through evolution of an implementation, ceases to have a compelling resemblance to the orginal metaphor. Remember this when choosing names. The metaphor may still have paid for itself by simplifying earlier requirement and design discussions, and a new metaphor may have at least as much descriptive power. There is no requirement that one cling to an inappropriate metaphor.

The point of the metaphor is that it is supposed to make it easier to think about and discuss the implementation of a system relative to its requirements. Having a metaphor can certainly aid in development of new features, especially in the early days of a software project.

Rules for Commenting

Font is Lavos Handy

Have you ever noticed that almost every popular code highlighting scheme places comments in low-contrast grey-blue-on-white or dark-blue-on-black? Commenting has long been considered an important part of programming, yet in practice comments are written to be ignored. I think it is because they are typically added to satisfy bureaucratic requirements rather than to supplement the source code with information.

I used to be a huge supporter of comments. I loved to see glorious, rich, voluminous comments in all the code I read. I loved to write them. I thought that for a program to be 60% comments was a pretty good start. But as my teams began to write cleaner code, they sometimes would brag that the code was so clean that it almost didn't need comments. Now I want all of my code to go beyond almost.

I also have come to value vertical space in a file. I want it to be filled with working code instead of boilerplate and fluff. I hate scrolling through noise to find the code.

A few simple points to consider, and we can all have code that is more clear and gloriously free from noise and distraction.

• Comments provide information that is not expressible in code. Comments are meta-commentary. Anything that we can express in the code, we are duty-bound to express as code. Comments must never repeat the code, or the version control system. They are never substitutes for expressive code. Comments may express things like copyright, sources of algorithms, and reasons one algorithm or data structure is chosen over a more obvious alternative.
  
• Comments are to be deleted when they are obviated. Any comment that provides information available in the code must be deleted. We don't need comments to tell us that x++ increments X. Any comment that does not provide value is to be deleted immediately. This applies to passages of code that are commented out. If the system runs fine without them, we don't need those lines.
  
• We should always strive to obviate comments. When we see a comment, we should take it as a challenge. If we can make the code express the content of the comment through refactorings (renames, extracting variables/methods/classes, inlining methods/variables, etc) then we should do so post-haste. If we can do it, then the code itself becomes smaller and simpler. And then the obviated comments can be deleted.
  

Inevitably someone will read this card as giving them license to stop writing comments, but that is a gross misunderstanding. If one takes a big steaming mound of poor code and removes the comments, it becomes an even more unmaintainable steaming pile of crummy code. There is a system and a balance that needs to be understood here.

This card doesn't tell us not to write comments, but rather to avoid writing code that needs them. Do I write comments? Yes, when I absolutely have to. But now I know when I have to.