3R – Refactor, Retrain, and Retool

As your agile conversion progresses you may start to feel friction from many of your past cultural and development practices.  And, they could be pushing hard and preventing you from realizing the benefits you had hoped when you started.  You might not even realize that this is happening.  Some people just give up and begin to feel that agile isn’t a good fit for them.  The problem is underestimating how deep you may have to go to truly complete your conversion.  It’s never enough to just learn the ceremonies without truly understanding and embracing the key principles that define agility.  See my article: Achieving Agility for more thoughts on agility.  In this article I am going to discuss in more details some of the obstacles I have found and some actions that could help remove these impediments.   Converting to agile will require some refactoring of your code and processes, retraining of your teams and management, and a sharpening of your toolsets.

Obstacle 1: Your code

Conway’s law states that:

organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations

Adopting agile is a significant organization change. But, according to Conway’s law, your systems will likely represent more your old organizations and processes. This can impede progress on achieving a more agile release cadence. And, it can cause significant risk to the overall quality of your product if you are not careful. See my article on Agile Quality Practices for more details on the quality gap and how to avoid it. In this section we’ll talk about how to move past some of these problems and make the quality gap truly a thing of the past.

Refactor for testability

The first problem to address is the testability of your code.  How quickly and safely can you apply changes without increasing the risk of introducing defects?  There are some great resources out there that can help you start refactoring today.  Michael Feather’s book Working Effectively with Legacy Code is a great place to start.  He covers much of the key problems we all have with legacy code.  His book includes the following chapters:

• I Don’t Have Much Time and I Have to Change It
• I Need to Make a Change, but I Don’t Know What Tests to Write
• Dependencies on Libraries Are Killing Me
• I Don’t Understand the Code Well Enough to Change It
• My Application Has No Structure
• My Test Code Is in the Way
• My Project Is Not Object Oriented. How Do I Make Safe Changes?
• This Class Is Too Big and I Don’t Want It to Get Any Bigger
• I Need to Change a Monster Method and I Can’t Write Tests for It
• How Do I Know That I’m Not Breaking Anything?

Once you understand what to do, how can you get going?  It always seems so impossible to start a major effort because you have so many other commitments.  This is where you need to become more disciplined and find time to invest in refactoring.  As a goal, every time you touch some code, leave it better than you left it.  Over time, larger scale refactoring will come out of business need and may be accomplished over longer periods of time.  But, don’t wait or expect time for these efforts to just appear.  You need to learn how to make a practice of consistently making progress.

Simplify your development stack

The next big area of concern is the complexity of your development stack.  How many technologies are involved in having a scrum team add value to your products?  Typical waterfall development slices the system horizontally into layers such as UI, service, data, and database.  In the past, development teams

were dedicated to specific layers.  These teams may have made their own technology choices in the past.  The end result is that you have an extremely complex development stack.  You might have VB on the front end using some C++ libraries talking to a Java backend that uses an Oracle data layer.  (Don’t laugh, I’ve seen worse that this).  These types of abominations occur when teams become detached from delivering value and focus on their horizontal technology slice alone.  Agile tries to right this ship, but your choices of the past may be pushing back in a big way.   Teams may be forced to work across all technologies just to be able to deliver a very simple feature.

When you are stuck in this problem, there really is no easy way to re-align your organization and system to be more compatible with agile software development.  It is a clear mismatch.  The only thing you can do is to start to make clear architectural decisions for the future.  But, this is very hard to do.  You have ingrained expertise and specialists everywhere.  It will be impossible to make hard choices without disenfranchising many in your organization.  But, you really don’t have any choice.  You need to find ways that scrum teams can own delivery of value and can support the full stack necessary to do so.

Obstacle 2: Your culture

Management style

If you’ve been a waterfall development organization for a long time, your organization must have adapted to a hierarchical structure that was responsible for moving the process through the gates necessary to complete software.  In this world, requirements were dictated to teams.  What they would do and by when was decided by management.  When converting to agile the first step is to reorganize around agile teams.  It is hard for companies to make this move in one big reorganization.  Often there are significant remnants of the old organization that will still exist.  Here are some of the people problems I’ve seen:

• Project Managers without a home

In an agile organization it is no longer necessary to have project managers driving teams with their Gantt charts and status meetings.  Agile teams are self-empowered and should be able to navigate without this level of management.  There are better fits for PMs in agile organization.  They need to adapt to more roles of coaching and mentoring teams, supporting customers’ rollouts, or becoming scrum masters.  In agile there is an emphasis on making the agile teams effective.  PMs must learn new ways to contribute.

• Development managers that can’t adjust

One of the quickest way to drive agility from teams is to micromanage them into non-productivity.  Managers often become enamored with the visibility and transparency of agile in a counterproductive way.  They view all of this new data as something that they can use to drive teams.  Instead of the team taking ownership, they become the bottleneck for all decisions.  Good management needs to encourage and mentor teams to take ownership.  Otherwise the team can never be self-empowered.

• Product Managers that don’t trust

Often in the waterfall process, product managers become the single decision point for all customer requirements.  They are also set up to be responsible for all aspects of providing the solution.  They become very accustomed to dictating requirements, implementation, and expectations.  They can have a very hard time adjusting to this new way of working.  It is a huge leap of faith for then to trust the team with managing delivery.  They still feel it is necessary to drive teams with detailed feature roadmaps, committed solutions, and fixed dates.  Often they can feel that agile is in the way of getting what they need done.  They can view teams as available bandwidth for them to drive.  This problem is a hard one because typically product management and development are separate organizations.  Without leaders on the PM side that are agile advocates, this problem can be unsolvable.  Teams need to push back when they can and try and prove their ability.  Over time there can be progress made.  But, it will take decisive leadership to make the changes necessary to open up the flow to a full agile workflow.

• Sales and Service that don’t get it

Having agile development teams doesn’t really accomplish much if your sales and support teams are still driving waterfall projects with your customers.  Promising fixed scope, long term project deliverables to customers can undo any attempts at being agile.  True agile organizations must drive out the value of their process to their customers.  And, this can’t be done without sales and service organizations that buy into the value.  Accomplishing this conversion takes time.  You can’t expect these teams to adjust based on just talk.  You need to prove out the value over time by delivering solutions of high quality, providing visibility into projects in progress, and be adaptable to change.  If your agile teams and management still act like they are a waterfall organization, the field will never understand or buy into the conversion.

• QA teams lost in the shuffle

Removing the long waterfall test cycle and replacing it with automation takes time.  But, often the QA teams that existed to support the test cycle are disbanded and spread out across teams.  This can cause a lot of loss in quality testing and cause distress to QA members that are lost in the shuffle.  Companies need to be thoughtful on how they convert their quality processes.  There are a lot of factors involved in testing and certifying product releases.  See my article on Agile Quality Practices for discussion on these complications.  During agile transformations QA resources need to retool and get more involved in automation.  There may still be a large demand for manual testing.  But, the goal of the organization is to eliminate this over time.  These team members will need to adapt.  But, this change will only happen as fast as the rest of the development can change.  This could take a long time and/or never happen completely.  So, it can be a benefit to keep QA teams that work based on a quality mindset intact and functioning during this transition.

For companies experiencing these growing pains the best advice is to not ignore it and hope it goes away.  Actively coach and mentor groups, re-organize when needed, and realize that agile may not be for everyone.  You may have to make some hard choices but do it.

QA vs Dev

Scrum teams should be responsible for the ownership of their deliverables.  But, often during transition, there is still a clear separation between QA and Dev resources.  In some cases, scrum teams are too dependent on external resources to ensure quality.  In other cases, they are running a mini waterfall each scrum.  Where part of the team is testing and the other part is developing.  Long term, either mode is not optimal.  Developers on scrum teams need to take charge of the quality of the code they are developing.  This will require a change in development practices.  But, also a change in culture.  Years of waterfall development experience has taught many developers to rely on others for testing software.  Scrum teams cannot optimize in this model.  Testing efficiency is probably the biggest factor in improving velocity for scrum teams.  If this is only the responsibility of a few members, it will be impossible to achieve a high degree of efficiency.  And, the tester s for the team are often the team members with the least development experience.  Testing and quality practices need to be a high priority for the whole team.  And developers need to be active in improving testing efficiency.

This can be a painful transition for some developers.  They have grown accustomed to not having to take this level of ownership.  And some don’t want it.  They may even feel it is “beneath them”.  This can be a toxic to team chemistry.  My recommendation is to make testing and development the same job.  Techniques like Test Driven Development (TDD) can help align developers how to build quality into their processes.  Teams need to be driven to drive out inefficiencies in their test processes and find ways to build out quality as a team.  This will require a training in test automation skills and some shakeup for teams.  Developers that can’t embrace this models, and QA experts that don’t have a desire to do more development may have problems finding a home on scrum teams long term.    Team members need to be adaptable in order to ensure that the team can be as agile as possible.

Collaborative vs Cowboy

What sort of behavior does your company reward?  Are they expecting developers to jump in and constantly save the day?  Do they consider working long hours a sign of valued employee?  Do they value individual achievement over sharing?  Do they rank and rate employees?  Many of the HR and management practices of a company can actually work against creating agile teams.  Companies need to pay close attention to if their values align with agile.  There are a number of practices that work against successful agile conversion:

• Stack ranking employees

In my opinion probably the most damaging thing you can do to a team is start pitting one member against another.  Why should teams collaborate if their incentives are based on placing higher in a stack rank to their peers.  Encourage teamwork and the team will tell you who should be rewarded.  Give them power to decide who is contributing and reward accordingly.  Having an outside authority subjectively rank team members will end all collaboration.

• Individual award and recognition

Having management award things like MVP and such also takes the power away from the team and can pit one team member against another.  Why take the chance on this type of incentive model?  Instead, drive incentive based on team achievement and reward achievements of real results.  Periodically making some arbitrary award decision can be damaging to the team.  Team members will question why the award occurred.  And trying to spread awards over time just adds more confusion.

• Forced overtime

Overtime should occur when someone has screwed up.  If this is happening all the time, then there are bigger problems.  If your corporate culture rewards overtime, then you need to question how well management is performing.  Teams know when they need to kick it in.  Empower them to own their solutions and they will do what it takes.  But, if that means worked overtime for extended periods, then there may be some issues that need to be looked at.  Teams that are working overtime constantly are rushing things and cutting corners.  They are likely creating significant technical debt.  Many failed projects follow this pattern.

• The hero mentality

Management can put too much reliance on a single individual.  They are called into solving every emergency and tough problem.  Management may only value their opinion over other team members and even other teams.  Depending on the person this could be a bad thing.  Some developers feed on this sort of attention.  They can horde knowledge and be very secretive about the efforts.  They don’t like to be questioned and fear collaboration.  And, often management can be fooled into perceived competence by these type of individuals.  This sort of expertise does not scale.    In agile, leaders lead teams.  The heroes are the team members that make everyone else better.    Don’t be fooled by the cowboys that feel their expertise is unique and you can’t live without them.

Obstacle 3: Your development/organizational processes

Bureaucracy

Long term project management requires a lot of bureaucracy.  If you have been doing the waterfall for years/decades, it may be hard to recognize the amount of bureaucracy inherent in your processes.  Leaders that are pushing organizations to adopt agile often focus on the ceremonies and see the added transparency as additional data for the management machine to consume.  But, this is a failed approach.  Agile relies on organizational process to adopt techniques of Lean software development.  Lean software development was adapted from Toyota manufacturing techniques.  The focal points of Lean software development are:

• Decreasing cycle time – Eliminate waste, deliver as fast as possible
• Delay commitment – Decide at “the last responsible moment”
• Amplify learning – Feedback loops, team retrospective, and personal development
• Build Quality in – Quality is addressed up front and not at the end
• Empower the Team – Self empowerment, management pull of new ideas
• See the whole – Think big, act small, fail fast; learn rapidly

Organizations moving from waterfall processes have a hard time adapting their culture and processes to be more lean.  Often lean principles become more slogans than actual reality.  It requires a big commitment from management to convert.  It can take many years and requires change from developers as well as management.

Developers:  Kaizen is the Japanese word for “improvement”.  Teams need to adopt a the kaizen mindset of continuous improvement.  Good enough never is and no process is perfect.  Be
aggressive about eliminating waste in your processes.  Challenge management to create a better development environment and ask for what you need.

Management:  Focus and listen to what the teams are telling you.  Understand your development processes and empower the teams to improve.  Develop confidence within the organization on the ability of your teams.  Open up feedback channels so teams can work with the latest information from customers.  Question corporate commitments and push for simplicity.  The old organization was probably set up in ways that inhibit lean process improvement.  You need to represent the teams across the rest of the organization.

A good reference for lean software development is Mary and om Poppendieck’s book:  Lean Software Development:  An Agile Toolkit

Devops

Operations, in most all waterfall organizations is a separate department from development.  There are often many departmental obstacles that create impediments between operations and development.  For example, many waterfall projects cannot even start unless an annual IT budget is approved first.  Devops is an approach designed to eliminate the barriers and allow ops teams and developers to work together to rapidly develop and deploy solutions.  This is a challenging transition because the original role of operations is to ensure stability and mitigate risk.  While development teams are tasked to introduce change at a rapid rate.  These are often diametrically opposed motivations.  Years of this have create negative cultural biases in both directions.  Developers see ops as obstructionists and ops teams see development as incompetent and ignoring risk.  And, there is some truth to both of those viewpoints.  The solution is to bring these two teams together and create shared goals for delivery.  Thus, the term Devops was created.

In order to align goals, both developers and operation need to agree on a framework for delivery.  This framework needs to:

1. Ensure stability, scalability, and quality of deliverables
2. Support rapid development and change

Both sides work together to build out this framework.  The framework needs to include:

• The ability for developers to build and test code with rapid and continuous feedback
• Automated Packaging and releasing solutions with control and auditability
• Ability to Configure and monitor production environments

Building this framework needs a cultural shift from both camps.  Developers need to be more operational focused when developing new software.  Building in quality and monitoring up front.  Not at the end.  Operations need to adjust to be more service oriented.  They need to change their mindset from that of being a control and monitoring group to one that is enabling teams to rapidly and safely deliver new software.  Control is still necessary to ensure integrity of customer deliverables.  But, it should be implemented by a framework, NOT with process overhead.  This will require a significant retooling and retraining process.   There are a ton of tools and techniques to accomplish this conversion.   Some examples are:

• Test driven development
• Behavior driven development
• Continuous Integration
• Continuous delivery
• Blue-Green Deployments
• Canary Releasing
• Cloud Computing
• API Gateways
• Network Performance and monitoring tools
• And more….

Merging your software release and operational procedures behind a framework of tools and techniques can help to ease the bottlenecks and create synergy across all teams required to ensure that your organization can deliver value at the pace demanded by your markets and customers.

Summary

Agile conversion requires more than just adopting a few ceremonies.  Because Agile is not a just new process.  It is a total make-over of how you organizationally, operationally, and developmentally address building out your software.  It will never be possible to complete this shift in rapid fashion.  Instead it is a process of continuous improvement with a clear end state in mind.  So, keep improving!!!