How to Ace Project Planning with Specification by Example (SbE)

In software development, almost everyone you work with understands the importance of Quality Assurance (QA), but rarely understands the value of engaging QA engineers — especially during early project planning stages. The end result is that a lot of value QA engineers could have provided is simply wasted.

In my experience working as a part of SCRUM development teams with little to no technical specifications of the projects and communicating with a non-technical product owner, the role of my QA team has proven to be an essential part of the product development process. Here I’d like to share an effective way to bridge the gap between software development, software quality assurance, and the business with a method called Specifications by Example.

In this article, we aim to focus on the value of QA engineers, especially during a critical project stage: planning. We will also touch in detail on the SbE approach, which is more commonly referred to as specification by example.

Let’s dig deeper into how exactly this is done, but before, a few QA fundamentals.

Why Start Early with Quality Assurance

Quality Assurance (QA) is the process of identifying whether a product meets all the requirements in terms of (including, but not limited to):

• Functionality
• Security
• Compatibility
• Usability
• Performance
• Reliability

Ideally, it allows to avoid errors in late QA project delivery stages, saves time and unexpected expenses, decreases the cost of defects (bugs).

The cost of defects depends on the impact they have and when they are found. Usually, the earlier you find one, the less time and effort it takes to fix it. Any error is easier to fix and costs less when you’re still in the requirements or design stage. On the other hand, if you find a defect after the product goes live, error correction may cost much more in terms of both the fixes and the users’ reaction.

Typically, the Requirements Gathering and Analysis stage (also sometimes called Planning) doesn’t get enough credit within the software development (SDLC) process, even though it can significantly impact a QA project.

In my view, a QA specialist must be present at all stages (based on Scrum criteria), yet, the requirements for testing and detailed grooming that involve QA engineers aren’t usually handled within early spirits, keeping only to the letter of good Scrum standards but not its spirit.

Maximizing QA Impact During Requirements Gathering and Analysis

When only product owners, business analysts, and software developers work with the requirements, defects due to poor planning are easier to miss. The importance of the right QA process for software development is illustrated in the cartoon below.

During the first and most critical in terms of impact stage, quality assurance engineers can:

• Expand product knowledge. By exploring user stories, skeleton models, and papers, QA engineers can get an idea of the work process, figure out the goals of the interested parties, and prepare questions.
• Reveal logical contradictions. As soon as QA engineers dive deep into the documentation, their understanding of the project grows and they can spot logical contradictions early in the development process.
• Address unaddressed elements. They start noticing flaws in the business process, authorization, and user roles. Poorly defined things like user and input validations and text-field requirements typically become visible to QA engineers early on.
• Provide user stories feedback. User stories are brief descriptions of what a product should do for a user, for each of the functions. If the stories are too dense, complicated, or incomplete, they lead to difficulties in coding and preparing test cases. Relying on experience, QA engineers can leave their feedback and suggestions on user stories to make them more accurate
• Pose questions. To help address issues, they post questions or point out the flaws they’ve noticed.

Including QA engineers early in the software development process prevents many defects and decreases the negative impact on the timeline, budget, and risk.

QA Engineers and Their Role in Communication and Collaboration

The success of effective collaboration in a software development project lies in clear communication and can be easily achieved by focusing on meeting business goals. Even though QA engineers and software developers each have their own focus, they are still bound by a common objective: to manufacture a reliable, quality product. That’s why we can use a concept called “Three Amigos”.

In simple terms, the Three Amigos is a meeting of product owner or business analysts, software developers, and quality assurance engineers to discuss a new feature and refine the specification. From various perspectives, the parties collaborate to determine what they have to do and how they will know that it’s been done correctly (the “Definition of Done” is clear and precise).

The quality assurance engineers have the following role in this process:

• Ask questions while communicating with the business.
• Analyze the requirements and create test documentation that will prevent ambiguous interpretations during the development stage. Accurate test preparation helps developers avoid bugs and deliver a high-quality product without delays.
• Suggest that clients review block schemes before sending them to developers.

There are many approaches that aim to make planning and requirements gathering more effective for all stakeholders. Behavior-Driven Development (BDD) is one such method worth exploring.

Quality Assurance in Behavior-Driven Development

Behavior-Driven Development (BDD) implies example-based communication between developers, the QA engineers, and business analysts, or non-technical parties (usually presented by business participants). Ultimately, BDD is meant to improve communication among stakeholders.

The outcome of behavior-driven development example would demonstrate test cases written in a common technical-jargon-free language, which leads to effective collaboration between interested parties. The team may prefer the BDD approach to TDD (test-driven development) for several reasons:

• BDD provides more accurate instructions on how to better arrange conversations between stakeholders.
• BDD applies “Five Whys” — the famous technique used to discover the causal relationships in a specific problem — to each user story so that its purpose becomes definitely related to the business result.
• BDD comes with a “Given-When-Then” approach, which refers to a style of writing down test cases for user stories in simple language.
• BDD contains a variety of tools that address the automatic creation of the technical and end-user documents.

Behavior-Driven Development is inevitably linked to another technique called Specification by Example. Both of them use examples to determine how the system should behave. Next, the examples are expressed in a common language that any of the participants can understand, including non-technical members. In some cases, it comes down to Gherkin language that’s intended to help interpret the requirements from the client’s perspective.

Here’s how Specification by Example and Gherkin work.

Specification by Example and Gherkin

Specification by Example (SbE) is a collaborative technique. This methodology helps determine requirements and functional tests suitable for the needs of a business. The needs are based on recording and illustrating requirements, with explicit examples instead of abstract statements. SbE is applied in terms of Behavior-Driven Development and is especially important for large-scale projects (because it allows managing requirements and functional tests).

Meanwhile, Gherkin refers to DSL (or domain-specific language), which provides a platform for business analysis and end-user testing.

By giving an idea of how the client sees the requirements, it clarifies a user’s working process and creates explicit conditions that the software must satisfy to meet a user story.

How Does Specification by Example Work?

When diving deeper into SbE and considering it in the context of Quality Assurance, it’s important to mention the process patterns that make adjusting software products easy and guarantee that the necessary product is delivered.

Now let’s take a closer look at each of the process patterns and their goals.

Collaborative Specification

The objectives of this pattern include:

1. getting all necessary roles into the team so that everyone can share their views and contribute to the general understanding;
2. shared responsibility for the project’s features and project specifications.

Illustrating the Specification Using Examples
Here’s when a Given-When-Then formula is used for creating testable specs. Upon this pattern, the participants in the development process – including developers, QAs, and a product owner – keep considering the examples until they correspond to the expected behavior of the feature.
Refining the Specification
Based on the raw examples and collaborative specification, the product owner creates refined requirements, which are easy for all the stakeholders to understand.
Automating the Examples
Using DSL, you can automate testing by displaying clear links between inputs and outputs. While performing test automation, make sure you emphasize specification rather than a scenario. Besides that, the tests should be accurate and easy to understand, hence testable.
Validating Frequently
In the deployment pipeline, you can also validate examples every time changes are applied. Additionally, you can expect to have a great number of tools and techniques you can use to ensure product quality. By performing testing as often as possible, you have a chance to reveal weak links at the early stages.
Living Documentation
The principle of this pattern is to keep specs easy and precise, evolve them as work progresses, and allow everyone to access the documentation.

SbE Examples

The product owner writes the acceptance criteria if they have a clear understanding and experience in software development and know how to write a project paper. But in most cases, it’s a task for the development team.

For more effective communication, the developers must collaborate with business analysts, the product owner, and a QA engineer. After gathering the whole team, they start working on creating the acceptance criteria.

Let’s consider what steps specification by example process requires:

1. Discussion the acceptance criteria

Product owner, developers, and quality assurance engineers initially each see acceptance criteria differently. This can lead to miscommunication and defects further on.

2. Illustrating acceptance criteria with examples

To minimize differences in understanding, a common language format can be used during requirements analysis.

3. Discovering hidden assumptions

To avoid each team member filling in missing data with their own undocumented assumptions the team discusses edge cases, exceptions, and alternative business process and user flows.

4. Writing abstract acceptance criteria

The goal is to arrive at common acceptance criteria that elicit additional clarification questions for non-obvious scenarios.

5. Working on examples of acceptance criteria

Augment the abstract criteria with real-life examples and clear acceptance criteria that anyone can understand.

The Benefits of Acceptance Criteria

Different kinds of methodologies and approaches aim to bridge the communication gap in the project lifecycle. Using Specifications by Example, each party involved in the development process can significantly benefit from using any of them. And by adding acceptance criteria into the mix, all participants win. Here’s exactly how each party benefits from writing them:
Product Owner:

• Expands and validates the requirements
• Clarifies negative cases
• Has a clear understanding of business goals and provides the team with early feedback (resulting in a better alternative decision than the developers would have chosen)

Software Developers:

• Ensure the requirements are explicit and lack functional gaps
• Keep up with the business goal, which allows them to stay motivated to find the proper solution
• Write better code due to their clear understanding of test case dependencies, which allows them to offer better alternative architectural designs

QA Engineers:

• Build quality early on in the development process and help create a better-quality product as a result
• Provide ready test scripts for user acceptance testing
• Offer formalized testing formats that are perfectly suited for automated testing
• Prevent development errors, as long as the developers clearly understand which specific criteria their code will be tested against

Closing Thoughts on Specifications by Example

Including QA engineers from the very beginning of a project has a lot of benefits, from saved time and money to fewer bugs in the final stages. All of this leads to one major advantage — the ability to deliver a high-quality product that meets all the client’s requirements and business needs.
Besides, QA engineers play a significant role in communication and collaboration with all the stakeholders who participate in software development. It is important for them to communicate throughout the software development process, especially when many approaches and techniques are or might be used to improve communication (e.g., Three Amigos, Specification by Example, etc.).
While seeking to build a high-quality product that will meet business needs, I suggest you give Specification by Example a try. It accentuates shared understanding and allows the team to determine a single source of truth. Using this collaborative approach, the stakeholders can automate the writing of acceptance criteria and focus on preventing defects.
If you apply QA testing to the Requirements Gathering and Analysis stage, you’ll be able to prevent defects in design from being coded into the software. At this stage, QA deeply learns the documentation along with the requirements, monitors whether they are met and not skipped over by developers, ensures business problems are resolved, and reveals contradictions and unaddressed elements. Hence, QA is an essential part of requirements management.
And finally, if you’re a part of the development team, keep in mind the following:

1. A formulaic approach to planning without working through goals and aims doesn’t lead to great product results.
2. There is no magic pill — try different methods in your specific process; don’t cling on to a single one.
3. Each team has its own optimal blend of approaches. If one doesn’t work for you, go for another methodology.
4. It is important to experiment and modify the elements until they work for you.
5. All team members with different competencies should take part in the planning process to create a fuller picture of the requirements.