The art of creating a good agile backlog is something that teams often struggle with. Telling the right story is an art that teams often neglect. Too often there is a rush to get the sprint moving and there may not be enough effort invested in making a solid backlog. What is missing often is proper context. Whose story is this? Today, the standard for user stories is that they should follow the INVEST pattern:
In my experience, Scrum teams full of engineers don’t typically have problems dealing with the “EST” part of this acronym. But the “INV” part is often foreign to teams that are new to agile, so they tend to neglect this aspect and focus on the operation aspect of their backlog. These teams can turn into more order takers and don’t realize their full potential as self-managed teams that are focused on delivering value. This article is about recognizing this problem and some ways to help transition teams to the next level by making sure they get whole story!
Are you getting the whole story?
What are the warning signs that teams are not working based on a well formed backlog? Here are a few signs:
- Teams feel requirements change too often
- Teams skip or don’t have compelling sprint demos
- Teams aren’t regularly gr
- Teams release at a very slow cadence
- Teams require significant technical guidance or designs before they can proceed
- Customer solutions consistently need reworking post-delivery
Many of these symptoms are hold-overs from past development processes, organization, and development culture. But, to complete any agile transition will require decentralization of authority and more team empowerment. It’s starts with understanding how to structure the backlog, such that your stories are independently deliverable, give the teams the power to negotiate the solution, and ensure they focus on the value they are delivering (INV).
Why is Independent so hard?
Delivering independent stories can be easy if your company is a unicorn and you architected and built your system with agility in mind. But, most companies don’t have this luxury. They must live with solutions that were architected in the past using some waterfall development process. These processes focused on a horizontal layered approach to development. Each layer of the solution is built out independently and often by different teams and/or organizations. Over time, the organization adapted to fit this model. Now we are trying to introduce agile teams and we can’t easily produce stories that can independently deliver value. Instead, it takes multiple teams, multiple sprints to actually complete any value. It is the curse of Conway’s Law.
Conway’s Law –
- organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations
Changing your organization to one based on agile teams is often fighting against your past. Now you have a complex dependency across teams and it’s impossible for teams to be independent and empowered to deliver value. How do you dig out of this? The first step is to end your denial.
The diagram below is an example of horizontally organized teams:
Newly formed scrum teams are eager to launch and start building velocity. So, they will twist and contort the definition of a story in order to fit the work they need to do. You’ll see a lot of:
- As a developer I need to ….
- As a product owner, I need to …
- As a system component, I need to…
They will start arguing what is a “technical” story and how to define good ones. Stop this nonsense before it gets out of control. The goal of an agile team is to deliver value for their customer. “Technical” stories are typically never value delivered to a customer. Instead they are tasks that are necessary to deliver value. So, unless your company delivers APIs as your product, there is NO SUCH THING AS TECHNICAL STORIES. Trying to pretend that there is will only mask the issue. There is an unwritten rule that a story must be completed within a sprint. Well guess what, sometimes that is impossible. Don’t mold the definition of story to fit your organizational and architectural issues. Instead, here is the best way to move forward:
- Enforce the INVEST pattern for all stories
- If takes multiple teams to deliver a story, so be it. Link all their tasks to the story they are delivering
- If it takes multiple sprints to deliver a story, so be it.
- Measure teams output and determine where the bottlenecks are
- Reorganize or re-architect around these bottlenecks
- Repeat again and again until teams can meet the goal of delivering true stories every sprint.
Pretending this problem doesn’t exist will only cause it to remain indefinitely. Teams need to understand exactly why they can’t deliver independent value and fix it. It probably took many years to create the mess of dependencies that cause this problem so it won’t be fixed overnight. Be the best time to start is now.
What does it mean to be Negotiable?
To me, this is often the most overlooked aspect of the INVEST style of defining stories. Are the teams truly engaged in negotiating their stories?
The definition itself is up for a lot of negotiation. What is it about the story that needs to be negotiated? I’ve seen too many teams decide if the story can be completed in the sprint or not and then feel like they have negotiated enough. They focus only on the tactical aspect of completing the story not the story itself. When in reality they are only dealing with the “S” in INVEST. Is the story small enough to consume within a sprint? This is something that engineering teams are comfortable dealing with. In order to be agile, teams need to become problem solvers NOT just order takers. The goal of any good story is to start the conversation. Teams need proper context, experience, and access to information in order to start the conversation. Building out a well-defined backlog will facilitate this transition for teams.
Recognizing value delivered
Scrum teams are self-empowered and own the responsibility to deliver value. But in most organizations this has not typically been a responsibility that developers had. They, all too often, were in charge of the technical aspects alone. Now, teams must be able to understand and prioritize based on the value delivered by a story.
“With great power, comes great responsibility”
-Francois-Marie Arouet aka Voltaire (1829)
Focusing on value requires that the teams take accountability for what they produce. The power of agile comes when scrum teams are hooked directly to their customers and have very strong feedback loops. But, even in these cases, the value of stories can be lost. Conventional wisdom says that the customer is always right. But, when it comes to defining value, they can always use your help. When communicating with your stakeholders it is important to understand how to ensure that what you deliver has value. Discuss with them on why they feel the current solution will deliver value. Understand what their problems are and how this solves them. It is important to know what needles are going to move when you deliver. Are you going to increase efficiency, improve accuracy, lower costs, increase volume, lower response times, or improve any other measure that is valuable? Be wary of ever taking some one’s word that the story is important. Instead be sure you understand how to measure success and build out your requirements around delivering to those measures. Some tips are:
- Be sure your stories link to clear business objectives
- Don’t be afraid to ask “Why”?
- Clearly understand the business drivers and how they can be measured
- Don’t declare success until your objectives are met
- Iterate and evolve your value statements over time
- Take ownership of the problem. Don’t expect some higher authority to give you all the answers.
It takes as strong PO and team to build out good business objectives and ensure the work they do moves these objectives forward. But, this is a critical stepping stone to becoming a self-empowered team. Without these skills the team must forever be dependent on others to understand the value in what they are working on.
Building a better backlog
The goal of backlog definition is not to define “What” needs to be done. The goal is to clearly scope the problem that needs to be solved. Teams often focus on the “What” aspect of the backlog before understanding the problem itself. This can lead to missed requirements. If teams don’t clearly understand the context of the problem, they will often make up their own context. Each person involved in the solution could be actually working based on different assumptions about what the customer actually needs. This can often lead to overly complex solutions and confused end users. This occurs because the grooming process starts immediately by deciding implementation decisions. The content of the backlog leads a team down this path too quickly. Stories are often written from the system’s perspective and not the user. For example:
User Story Version 1: “As a library system user, I want to filter my subscribed books by author so I can find the book that I want to read. “
On this surface, this story doesn’t seem that bad. It meets all of the INVEST criteria. But, it has some weaknesses:
- It is dictating implementation details
- The problem definition is way too high level (“find the book I want to read”).
Is finding the book to read always tied to filtering authors? How was this decided? This story relies on someone other than the team deciding “What” the customer needed. This is taking the power away from the team.
User Story Version 2: “As a library system user, I want to know when the latest book is available from my favorite authors as soon as they come out, so I can start reading them.”
This story is a much better description of the actual problem from the user’s perspective. It will probably need more breakdown to fit within a sprint. But, it is a much better definition of the problem. If you look closely at the first example you can see that it is not actually written in the users’ voice. Instead it is written from someone describing what the system will provide for the user. This is a clear smell that indicates a potentially bad story. Teams need to push back and negotiate the story until it clearly defines the true user’s story. If they jump on the first story and start implementing quickly, the end result would be an under-whelmed user experience. It is NOT enough for the team to only understand what they think they need to do. They need to be engaged in the solution itself. Let’s take a look at hypothetical scenarios on how development teams deal with each of these 2 stories:
User Story Version 1:
The team would analyze the requirements and may decide to not just limit the filtering to just author. The scope could grow as the team may add multiple filter criteria. When the solution is delivered, library users would have to filter through all their favorite authors daily to see when new books are available. And, they received a bunch of filter functions that they would never use. The effort to implement the story grew and still the end user experience was not that great.
User Story Version 2:
If the team was given the second story, they may have set up a completely different solution. They could implement a way to subscribe to authors and receive notifications when their latest books are available. Now the library user can just set up a notification and wait for new books to be released. This is a much better experience for them. But, the team would never have come up with such a solution if they weren’t given the correct context to start with.
Anatomy of a well-formed backlog
In order to understand the whole story, teams need to know the following:
Why – Why is this story important?
What – What is the problem we are solving?
How – How are we going to solve the problem?
A good backlog will be structured so that all of this information is available to the team. The backlog can be refined through the grooming and planning processes of the team until it is fully defined. I prefer a hierarchical approach to capturing this information. At the highest level you can contain more Epic level stories. These stories contain the information about what high level business need is being addressed. For example: “Increasing user install base…, lowering support costs…, increasing transaction throughput…” These epics are more oriented towards a business initiative of some sort. Below the epic you can have the feature level. At this level you can define high level stories that describe the problems we need to overcome in order to solve the business objective defined in the epic. You could have one or more features required to satisfy the business need. Below the feature are user stories broken down to a level that teams can deliver incrementally. At this level it is still important to not lose sight of the problem. This is the level the team will plan their sprint work. When planning their sprint work, the team will break down their stories into the tasks necessary to complete to solve the problem defined in the user story. The overall backlog hierarchy looks something like this:
Scrum teams need proper context to design great solutions. Agile processes rely on the sprint backlog to contain the information necessary to build great software. Often, teams don’t have direct access to the customer and information doesn’t flow. In these cases, it is critical for the backlog to fully scope the problem. It is handicapping teams to try and dictate solutions. Teams can only be empowered to solve problems if they fully understand the scope of them. Building a good backlog is far more than just following acronyms and templates. Agile backlogs must contain enough context and information for teams to act intelligently and effectively. Let’s make sure to give them the whole story!