Acceptance Criteria: Yes? or No?
The question of what a customer really wants has tripped up even the smartest salesperson. Even as salespeople have become more tech-savvy, the ability to please each individual customer remains one of the most difficult challenges to master. Within the software development arena, the term acceptance criteria have become the key to unlocking the mystery of really satisfying a customer’s needs. The website hosted by Agile for Growth defines acceptance criteria as, “how a particular feature could be used from an end user’s perspective.”
I have found that the format of acceptance criteria is not nearly as important as the process used to create them. The power of creating acceptance criteria with the development team and the customer is what gives software development teams the power to give the customer what they want.
Now that we know how powerful acceptance criteria can be let's define what they are. Before we do that, I think it is important to understand another concept, namely BDD or Behavior Driven Development. Scaled Agile ® on their SAFe (Scaled Agile Framework) website says,
“The BDD process moves through three phases—discovery, formulation, and automation—where the acceptance criteria are transformed into acceptance tests that are later automated.”
Therefore, BDD is a process of developing acceptance criteria. Scaled Agile ® refers to a development team along with a customer representative when using BDD and says, “Together, this group reaches alignment on exactly what to build to reduce the rework associated with building the wrong thing and to accelerate the flow of value.”
So, if we consider what Scaled Agile ® calls “discovery and formulation”. Then acceptance criteria are the results of the discovery phase or written formulation of the desired behaviors the customer would like. Or what John Ferguson, the creator of Feature Mapping calls, “executable specifications”. If acceptance criteria are written ‘executable specifications’ is there a special format they should be in? Well, many have described various formats that can be useful. Scaled Agile ® describes one method or format as the 3Cs: Card, Conversation, Confirmation. Scaled Agile ® attributes the 3Cs to Ron Jeffries who is one of the inventors of XP.
Card: Represents capturing the statement of intent of the user story on an index card, sticky note, or tool. The use of index cards provides a physical relationship between the team and the story. The card size physically limits story length and premature suggestions for the specificity of system behavior. Cards also help the team ‘feel’ upcoming scope, as there is something materially different about holding ten cards in one’s hand versus looking at ten lines on a spreadsheet.
Conversation: Represents a “promise for a conversation” about the story between the team, Customer/user, PO, and other stakeholders. The discussion is necessary to determine more detailed behavior required to implement the intent. The conversation may spawn additional specificity in the form of attachments to the user story (e.g., mock-up, prototype, spreadsheet, algorithm, timing diagram). The conversation spans all steps in the story life cycle:
- Backlog refinement
These discussions provide a shared understanding of the scope that formal documentation does not provide. Specification by example replaced overly detailed documentation of functionality. Conversations also help uncover gaps in user scenarios and Non-functional Requirements (NFRs).
Confirmation: The acceptance criteria provide the information needed to ensure that the story is implemented correctly and covers the relevant functional and NFRs. Some teams often use the confirmation section of the story card to write down what they will demo.
The Agile for Growth website gives seven tips for writing acceptance criteria:
Each product backlog item or user story should have at least one acceptance criteria. Hey, don’t take writing acceptance criteria lightly or think of skipping it.
Acceptance Criteria is written before implementation – this is obvious yet frequently missed by teams. Write acceptance criteria after the implementation and miss the benefits.
Each acceptance criterion is independently testable. Why shouldn’t it be?
Acceptance criteria must have a clear Pass / Fail result. Write complex and long sentences at your own risk.
It focuses on the end result – What. [Not the solution approach – How. ]
Include functional as well as non-functional criteria – when relevant.
Team members write acceptance criteria and the Product Owner verifies it. It confirms the PO and the team have shared understanding of the user story.
Agile for Growth also mentions some of the most common challenges and mistakes they have noticed with acceptance criteria from user stories they have seen in the real world:
Pretty Narrow: Acceptance criteria is written very specific to a particular use case, scenario or technical approach.
Keep in mind, when you already have a solution in mind while writing the acceptance criteria, you end up leading your developers in that particular direction.
[Separate acceptance criteria should address all scenarios of importance, happy path or expected behavior when not happy]
Very Broad: Acceptance criteria is written pretty broad.
Too broad acceptance criteria usually increase the risk of the user story being too bulky and may introduce fat in the features.
[Too broad acceptance criteria tend to be hard to test]
Complex: Acceptance criteria is complex, includes jargons and technical details.
Good acceptance criteria should be written in simple English and should be easy to understand. The key is to keep it simple.
[Separate out different cases, using tables or other techniques, to make it easier to understand.]
I added an extra bullet point (in brackets) to Agile for Growths statements about the challenges of writing acceptance criteria. These extra bullet points represent the challenges that the team I work with has seen over the years.
I particularly like some of these additional ideas Agile for Growth suggests:
Who writes the acceptance criteria, or who defines the acceptance criteria, is not a matter of rules, availability or convenience.
What matters is – writing acceptance criteria (AC) should help establish and communicate a shared understanding between the product owner and the development team about solving a customer’s challenge or building the product capability.
Writing the acceptance criteria clarifies the scope for the team and also allows for the Product Owner to verify if the team and the PO have a shared understanding of the feature.
Another method for identifying acceptance criteria is the detailed method authored by John Ferguson. His method is known as “Feature Mapping”. John says on his website that, “Feature Mapping is a simple collaborative practice designed to help teams write great executable specifications. Feature Mapping draws on Jeff Patton’s Story Mapping, Matt Wynne’s Example Mapping and other techniques.” He goes on to say, “The most well-known format for BDD acceptance criteria uses the “Given-When-Then” structure:”
John points out that sometimes the Given-When-Then format goes wrong. That is where Feature Mapping comes in to help. Feature Mapping is best described by the flow diagram below:
Define a feature or story, or pick one from the backlog
Understand what actors are involved in the story
Break the feature into tasks to identify the main flows
Identify examples that illustrates a principle or variant flow. Ask questions like “But what if…”, “what else could lead to this outcome”, “what other outcomes might happen”, and use the answers to create new examples. Use rules to explain and give context to your examples
Rinse and repeat for other rules and examples
Create Executable Specifications: automate the main high-level flows with pending steps
Now that we know several different methods for obtaining and writing acceptance criteria, we must ask ourselves a question. Should we always use BDD to do so?
The Department of Product offers this insight:
“BDD scenarios are best suited to specifications where you think there is a likelihood that the requirements may be misunderstood without them or that a more thorough testing approach needs to be adopted.”
While BDD could be used for the development of any feature's acceptance criteria, it is most useful for more complex features. For example, features that include varying user types and multiple steps that too vary based on other parts of the system.
However, if a feature is not complex then one of the following forms may be more suitable:
Test that <criteria>
Demonstrate that <this happens>
Verify that when a <role> does <some action> they get <this result>
Demonstrate system diagram for <role> happy path
Demonstrate system diagram for <role> unhappy path
The need for acceptance criteria in software development has rarely been debated. The questions that are asked many times are what form should the acceptance criteria be in, who writes them, and what does a development team do with them? One can conclude from the details about the acceptance criteria mentioned above that the format really “depends” on the feature being developed. All the sources point to one answer when it comes to the ultimate source of the acceptance criteria – the whole team. As far as what is done with them in the end, the answer may seem obvious. However, John Ferguson answers it best when he says:
“Each acceptance criteria is a concrete example of how a user interacts with the system to achieve some business goal.”
Therefore, the acceptance criteria are used to build solutions that assist users’ interactions with the system to achieve their (the users’) business goals.