Traditional software design can dictate layers of abstraction that can challenge teams striving for agility. Often companies trying to adopt agile have years invested in both architecture and supporting organizational infrastructures that don’t readily lend themselves to adoption of agile. This article discusses the impediments that can exist in architectures that can impair teams searching for agility. Moving forward can be challenging and costly. But, this is a challenge that teams have no choice about. Ignoring this key aspect of agility will impede teams to the point that they will eventually revert their process back to what they originally had. Often, you hear that agile is not a good fit for some teams/companies. Problems with architecture and organization structure are often the catalyst to creating this perception. In reality the problem may lie in something more fundamental that they are either not willing or understand how to change. Adopting Agile requires an architecture that maximizes the throughput of delivery of value to your customer and minimizes the organization friction necessary to do so. Architectures will need to realign to fully support this goal.
Impediment #1: Specialization
The waterfall development process typically was optimized based around ownership of different tiers of the architecture. Th
is created a model for scale where multiple teams would independently work on the same deliverable. Typically, there were long development cycles followed by integration and testing cycles before the product released. Each team would specialize on their layer of responsibility. Teams could focus on database, service, or UI development and rarely would they move beyond their areas of expertise. And, the organization of the company would move to match the work being done. Now, as the company converts to an Agile process, these old boundaries seem unnatural. The old matrix-development model is now riddled with dependencies that create impediments for the teams. The old model assumes that specialization of skills can make each tier more effective. But, with Agile, teams need to be able to pivot rapidly. They can’t when so many teams are dependent on each other. And, bottlenecks form when multiple teams are dependent on the same team. This kind of organization creates a lot of friction that is directly opposing any change. This is counter to what is required for agility.
Impediment #2: Complexity
Software reuse is a noble goal that architects usually make one of the highest priority when designing solutions. But reuse can be a hindrance to agility. It can create organizational friction and unnecessary dependencies. The complexity and organization friction created by overuse will cause impediments to teams trying to move fast. Teams managing reusable components want to build components that will be reused by as many teams as possible. This creates scenarios where the more reuse you get the more teams are dependent and the greater the bottleneck for new features. This can create a spider web of dependencies and a lot of complexity. Reuse needs to target agility directly. The only reusable components should be infrastructure that will accelerate all teams. And, they should be as thin as possible.
Impediment #3: Alignment
Architecture needs teams to align on technology choices. If they don’t it can greatly complicate supporting infrastructure and hamper team agility. Deploying and supporting applications can be complex. Differing technical choices will require different techniques and tools to manage solutions once they hit production. So, it is critical for architecture to view the entire development lifecycle of the solution before determining the development stack. Scrum teams would love the flexibility to choose their technology based on whatever is latest and greatest. But that isn’t always the right choice. Architects need to understand and design solutions with the minimum support friction based on the current capabilities of the organization. The latest and greatest doesn’t
always fit in so well.
Another critical aspect of alignment is in resourcing teams. Agile organizations need some level adaptable resourcing model. It would be great if a scrum team can work on a small set of services
indefinitely. But that isn’t realistic. Customer and market demand will require resourcing to shift fairly often. When this occurs, teams shouldn’t have suffered significant technical ramp up time every time there is a change. If teams align better on technology choices this should be minimized. A technology stack that is too varied will drive teams to specialize creating a new impediment.
Agile Architectural Principles
Before you can have agility in your architecture you need to agree on what the basic principles that drive your architecture. Is it Performance? Scale? Response to change? Portability? Compatibility? Reuse? Adaptability? Security? Consistency? You can’t have it all so you need to decide what is the most important principles to align your architecture around. Building a definition of core principles will align teams and architects around what is important to focus first on. For example: if your business deals in a lot of financial transaction you need security and redundancy as important principles. If you provide media content for a large number of subscribers, you might value scale and performance. If you are under demand to build out a lot of customization for your clients, then you would value adaptability. Defining your architectural principles is not as easy as it sounds. Once you get 5 or more people in a room it is often very difficult to make decisions. Driving to a common set of priorities is the goal of this exercise. Try it, you will be surprised how much diversity of opinions exist today within teams guiding your architecture.
Simplification of Development Stack
Agile is about empowering teams to make their own choices. But, as your organizations grows, how can you make solid architectural designs locally that work together across organizations? Technical teams need to collaborate and make some shared decisions in order to be most efficient. 100 good decisions made in isolation will create a problematic architecture. It is important to have shared communities/guilds across multiple teams that work on common practices. One of the goals for this shared team is to simplify the development stack. Having a simplified stack will allow teams to share knowledge and tools necessary to build, test, and deploy solutions at scale. Creating a complex development stack will make it much harder to implement concepts like Continuous Delivery. So, working together as a bigger team is required to ensure that everyone can share infrastructure and tools that make the entire organization more effective.
Vertical vs Horizontal Architecture
Traditional software was designed as horizontal layers of abstraction. Each layer provided higher levels of specialization. Often companies organized around this architecture and created specialists across each layer. And then they adopt Agile. Often, they feel like agility is very hard and maybe isn’t right for their company. When in reality it is their own architecture and organization that is fighting against it. In order to create new business value requires mobilization of multiple teams across each layer. It can take massive coordination across multiple teams even when attempting to provide the smallest of features. This is remnants of past waterfall development processes still alive today within the architecture of your software. In the past, teams worked in isolation across massive feature deliverables using highly specified interfaces. This created the monolithic system that today is so hard to achieve agility with. To undo this will require significant time and investment. And, rarely is there a business case to start over with software that is currently generating value and revenue for your company. So, this problem is something that needs to be addressed in small chunks.
The first point of attack is to try and combine your assets to more closely match your new organization and process. This will require an architecture that supports vertical stack development where a scrum team can deliver end to end value without having dependencies. This surgery can be selectively applied to key areas of your software until over time, the majority of value driving software has been converted. The Micro-service architecture is a great pattern to strive for in the conversion. Micro-services can be designed, maintained, and managed by a single scrum team. So, if done right, they can greatly improve overall agility and allow teams to independently deliver value at a much higher velocity. Designing and building Micro-services is a topic far too large for just this post alone. But, it is worth understanding and investing in if you want to dig yourself out of the stranglehold your monolithic system has on your agility.