Agile teams have a clear view of how to tactically burn-down any backlog they have.  Many use Scrum or Kanban or some variation of these execution processes.  Given the nature of these processes, it is entirely possible for the team to execute through their backlogs without fully understanding and optimizing why these backlogs existed in the first place.  All companies have visions and long-term views into their products and business.  The merging of corporate plans and strategies with Agile practices does not always mesh perfectly.

• How can the company achieve long-term goals using an Agile process?
• How much planning is necessary to ensure successful execution of corporate strategies?
• How do we know if we are succeeding or not?
• How do we know what the right budgets are for development?

All of these questions pose a significant problem for many companies trying to adopt Agile.  Aligning all stakeholders requires understanding of Agile across all cross-functional areas of the company.  And, if even a single area is operating in a disconnected mode, the entire process could be at risk.  Aligning corporate plans and strategies to leverage the advantages of the Agile process can be a clear differentiator for any company.  Why is it so hard?

The Waterfall Process Gravity Well

The first obstacle companies run into is the Waterfall Process Gravity Well.  Any company that spent a long time developing software using the Waterfall process has a built-in gravity well that sucks all work towards it constantly.  Even if you are a startup, you probably still have plenty of people that brought their gravity well with them to the company.  It only takes a few to suck the life out of your Agile process.  Does any of this sound familiar?

All of these examples are a direct result of the gravity well sucking on your process. Your organization hasn’t fully adopted Agile and there are still many parts that rely on waterfall.
What are the root causes of this behavior? Why is the suck of the well so enticing?

• Familiarity – When things get tough, most people rely on processes that they are familiar with.  Without proper coaching and guidance most people will follow processes that they find familiar.  They need help to understand the impact of this and how they can align better.
• Lack of Understanding – Many corporate teams aren’t as familiar with Agile practices and have some erroneous assumptions. They may have only a cursory view and are still confused around basic concepts around Agile.  Since they can’t easily see the gates for the process, they can become uneasy and the gravity well will start pulling on them.  Without proper understanding, they will view Agile as a chaotic process and start to pull back.
• Lack of Alignment – As work flows through this system it need alignment from inception to delivery across all areas of the company. The gravity well of the waterfall process is so enticing because it allows teams to fire and forget their work.  Finance can just build a budget once a year.  PM can setup a roadmap once.  Sales can get customers to commit to a statement of work and walk away.  The problem is that ALL of these are completely resistant to change.  And, they will require management of these constraints and commitment for long periods of time.  All of which kills any opportunity for agility.
• Ambiguous Accountability – I did my part! It must be someone else’s problem.  This is a classic cultural attitude that waterfall encourages.  If accountability isn’t understood clearly, the gravity well will suck teams back into a waterfall way of thinking.  Once they finish their part, it is up to someone else to finish the rest.  This same attitude can kill agility.  Agile teams need to be accountable from inception to deliver of value.  Each team member must feel accountable to a business outcome.

The rest of this blog attempts to tackle these issues and provide some alternatives and strategies that avoid sacrificing Agility and still allows companies to deliver on plans and strategies that they need to ensure the business succeeds.

Building the Foundation for Agility

Most of the problems discussed so far occur because there was never a proper foundation for Agile implemented.  Without some sort of strong foundation, it is impossible to fight the gravity well.  There aren’t really good answers out there today around this problem.  Most solutions discuss implementation of greater process or organizational alignment.  All of which may help.  But, without a solid foundation, all implements may be at risk.  What are the attributes of a good Agile Foundation?

1. Organizations with solid agile foundations have vertical delivery streams that operate efficiently and independently. They eliminate Hand-offs and Gates.
2. Organizations with solid agile foundations learn together. Cooperative not competitive.
3. Organizations with solid agile foundations align accountability around shared business objectives.

The concepts behind creation of vertical deliver streams that optimize around delivering business value was already discussed in my blog on creating good user stories called:   What’s your Story?.  Check out that article on fighting back against Conway’s Law and the years of damage organizational misalignment can inflict on your ability to deliver value.  Also, be sure and check out how architecture can impede implementation of vertical value streams in my article:  Architecting for Agility

I also plan on a future blog around building Agile learning organizations.  Without this culture, agility can be impeded.  I already follow much of what Spotify pioneered in this area.  I write at a very slow pace.  So, in the meantime, I can definitely suggest you check out articles on how Spotify organizes.  Here is one from TechCrunch:  How Spotify Organizes.

This article is about the third item above:  Building solid Agile Business Objectives.  The following sections discusses in detail ideas on organizing corporate strategies and planning in ways that will optimize the value delivered by your Agile Processes.

Agile Business Objectives

Most private software development companies rely on profits to feed and enable corporate growth.  Product development processes are designed to create a rate of return on investment that will satisfy shareholders.  Basically, it boils down to, if I spend X what will I get?   Or conversely, how much will it cost me to get Y?  There is usually an entire organization built around managing these questions.  You have finance, program management, project management, and more.  The problem is that a lot of what these groups need flies directly contrary to what is necessary to be Agile.  Fixed scope and cost are what make projects waterfall projects.  The business groups managing the company will demand both.  There needs to be some way for a company to manage and control the execution in order to maintain profitability and, at the same time, not compromise all the benefits of Agile.  How can it be done?

What is a project?

Before we get too far, let’s first level set on what a project is.  I offer the following definition for each of the project types:

In the Waterfall world, a project is completed when all features are done.  When this occurs, teams will move on to the next project.  With Agile projects, it is very different.  Projects live more dynamically and can come and go based on business objectives and priorities.  Often, companies adopting agile, will still try and run their Agile development as waterfall projects.  These projects start based on a weak foundation for agility and usually end up in the same place most water fall projects do.  (i.e. over-budget and missing the mark).  So, how can you build a strong foundation for Agile projects?  Here are some guidelines:

It is easy to be skeptical about these and I admit there will always be situations where you need to compromise one or more of them.  But, use these guidelines to shape what your view of an ideal Agile project is and constantly retrospect to improve.

Creating good agile business objectives

Agile Business Objectives focus on results not tactics.  Waterfall projects and their associated business cases rely on if we do X we will get Y.  They require that X be fixed throughout the entire project.  These projects typically turn into very tactical exercises.  They are geared 100% towards what is necessary to deliver on commitments and can completely lose track of the targeted business results.  In these projects, development teams are only responsible for the committed deliverables, not the actual business results.  Creating good Agile Business Objectives require that this relationship completely shift.  Agile Business Objectives define the results and the Agile development teams must be accountable to deliver these results.  This is a big change and many organizations have difficulty getting the head around how to implement it.  Here are some guidelines:

1. Build product roadmaps that target results not feature delivery. Define roadmap milestones as objectives instead of delivery commitments.  For example:  Reach X% market share by Y, convert X% of current user base by Y, decrease development costs by X% by Y, Decrease support call volume X% by Y, …
2. Empower development teams to innovate and deliver business results. Give them the close customer contact they need to define and adapt their backlogs to deliver the value required to meet their objectives.  Define the objectives but leave open the path to get there.
3. Track and measure progress towards goals meticulously. This is NOT an ad-hoc process.  Teams need to be accountable to continuously delivering value.  This needs to be measurable.  Finishing story points is not necessarily value delivered.  It is not enough for teams to consider velocity as their measure of value delivered.  They need to see actual needles move that are tracking the progress towards their business objectives.  For example:  Are more people now using the product?  Have sales increased based on our feature delivered?  Are we having a positive impact on support costs?

If this sounds good, the next question is: Who do you need to get onboard to make this happen?

There are a couple of strategies, I believe the best approach is to build alliances in Product Development, Product Management, and Finance.  These are the key groups that you need to convince before going to C level executives.

Product Development Impacts

Not all development teams are ready for this level of engagement.  Do they have the right experience?  Do they have the right culture?  Are they engaged and willing to accept accountability?  Making this transition can take time.  Be sure and ease the teams through this transition and be sure they understand and are committed to objectives they own.  Committing to business objectives requires the right people and the right encouragement from management.

Product Management Impacts

Driving alignment on business objectives is the responsibility of product teams.  Each product owner needs to have the support and access to the information they need to build a backlog of work items that will drive value.  This will require a clear communications and feedback channel with customers.  Putting too many layers of management in the way of information flowing will impede the process.  It is best to flatten this organization as much as possible so information will freely flow.  If your team is too big, I recommend organization concepts as defined in the LeSS approach to scaled agile.  Here is a link for more information on LeSS Framework.

Finance Department Impacts

The Finance department can be the trickiest area to get onboard with these concepts.  They mostly still operate within the delusion promised by Waterfall planning.  (i.e. we can define all features and scope before funding a project).  It is often convenient to establish annual budgets based on this delusion.  It is a little harsh to call it delusion.  But, this sort of planning is never valuable.  And, it creates a ton of bad behavior throughout the entire project.  And yet most CFOs on down still demand it.  How can you get around it?

The first thing you need to do is to realize that they have very real concerns that you need to address.  You can’t just talk them into a plan where your funding will vary wildly and you can change budget willy-nilly whenever you want.  You need a proposal that will show them you have valuable business objectives and full transparency into exactly how the project is progressing towards them.  It is not as hard as it sounds:

1. Work with the product team to define the business objectives. Commit to achieving these objectives within a budgetary time frame.
2. Establish a baseline cost that would still justify the value. For example:  In order for this value to be worth it to the company, it can’t cost more than X.
3. Present this simple case instead of some resource intensive effort to build a plan based on features and estimates. Get buy-in from development teams, finance, architecture, product, and customers on the viability of this case.  If all agree on viability, start the project.
4. If the project is approved, start tracking both business value delivered and cost.  NOTE:  I must reiterate that business value delivered and story point velocity are 2 very different things.  If the objective is to increase sales by 50%, the project should track up to this level of revenue gain iteratively.  If the project doesn’t appear to be tracking to deliver the promised value, or the costs are starting to track higher than expected, there will need to be a change in the project.  Finance can control and manage these changes.

From a financial perspective, this is a far safer plan than a traditional waterfall project.  But, it is not the traditional method.  This can cause some friction and planning.  Start small with a few pilot projects until the entire team can gain confidence.

Conclusion

The Waterfall process has been around for decades and it doesn’t go away very easily.  If one aspect of your organization wants to be more agile and another part still operates within the waterfall, the waterfall will always win.  Driving close alignment to all organizations involved in agile product development is critical to escaping the Gravity Well of the waterfall process.  Transitioning from a feature-driven portfolio to an objective based plan can create projects that can operate with agility.  It will also drive ownership to the entire team for business results.  This is the type of focus that will allow teams to pivot and adapt so they can optimize value delivered.