The Agile Manifesto 20 years on: agility in software delivery is still a work in progress
We've had two decades to absorb the values and principles of the Agile Manifesto. What have we learned, and what are we still learning? 'We are closer and more aware, but we are turning a tanker and it is slow and incremental.'
There is still much work to be done to deliver on the promise of agile -- this being the 20th anniversary of the creation of the Agile Manifesto, which outlined the importance of close, unfettered collaboration between software creators and business leaders. However, while "agile" is become the most-popular buzzword in the business technology lexicon, it often is still just a buzzword.
It's not that enterprises aren't aboard with the values and principles of the Agile Manifesto -- most are. However, these values and principles are not front and center in most strategies. "I'm not sure that the thinking in the Agile Manifesto is the norm in the industry, still," says Kief Morris, principal cloud technologist at ThoughtWorks. "People know they need to change how they think about technology, they talk about digital transformation, but I don't know that they really see how the values and principles of the Agile Manifesto connect to this, how you can use them to make it a reality."
The challenge arises "because many come to agile as a solution or prescription, rather than starting with the philosophy that the Agile Manifesto focused on," says Bob Ritchie, VP of Software at SAIC. "Many best practices such as automated test-driven development, automated builds, deployments, and rapid feedback loops are prevalent in the industry. However, they are frequently still unmoored from the business and mission objectives due to that failure to start with why."
Still, others feel we're still nowhere near achieving the vision of the original Agile Manifesto. "Absolutely not at a large scale across enterprises," , says Brian Dawson, DevOps evangelist with CloudBees. "We are closer and more aware, but we are turning a tanker and it is slow and incremental. In start-ups, we are seeing much more of this; that is promising because they are the enterprises of the future." Agile initiatives "all too often are rolled out from, and limited to, project planning or the project management office. To support agile and DevOps transformation, agile needs to be implemented with all stakeholders."
Some organizations turn to agile "as a panacea to increase margins by cutting cost with a better, shinier development process," Ritchie cautions. "Others go even further by weaponizing popular metrics associated with agile capacity planning such as velocity and misclassifying it as a performance metric for an individual or team. In these circumstances, the promises of the manifesto are almost certainly missed as opportunities to engage and collaborate give way to finger pointing, blame, and burnout."
What's missing from many agile initiatives is "ways to manage what you do based on value and outcomes, rather than on measuring effort and tasks," says Morris. "We've seen the rise of formulaic 'enterprise agile' frameworks that try to help you to manage teams in a top-down way, in ways that are based on everything on the right of the values of the Agile Manifesto. The manifesto says we value 'responding to change over following a plan,' but these frameworks give you a formula for managing plans that don't really encourage you to respond to change once you get going."
The challenge, is "most agile initiatives still miss the upfront work of value stream mapping and optimization," says Venky Chennapragada, DevOps architect with Capgemini North America.. "Agile projects still look at optimization of work that generates unnecessary context or reduces quality." The changes requested by the business will need to produce the intended results, with feedback from customers by releasing small changes, he explains. "Quick feedback is needed to help justify the business case to continue forward or make an investment somewhere else."
The Agile Manifesto "talks about things like valuing 'individuals and interactions over processes and tools,' but that's hard," adds Morris. "Processes and tools seem easier. So many software companies and consultancies offer you processes and tools to give you agile and DevOps and make you digital. But they're really about giving you the comfort that there's an easy way, a formula, that will make you as successful as the companies that are winning in the market."
Ritchie agrees that there's too much of a tendency to pigeonhole agile into rigid processes. "The first and most-common mistake is the interpretation of agile as simply a process, or something you can just buy and do to immediately call yourself agile," says Ritchie. "This more often than not results in process for the sake of process, frustration, and - contradictory to the intent of agile - an even further disconnect between business outcomes and the IT professionals chartered to deliver them." Related to this, he says, is there often can be a "dogmatic agile zealot approach, where everything a particular framework says must be taken as gospel, and the emulate-fallacy approach -- most easily identifiable by phrases such as 'we are following the Spotify model.'"
Conversely, there's a tendency to get away from rigid process approaches, and over-simplify agile efforts. The second most-common mistake Ritchie sees with agile efforts "is to over-correct away from the dogma and numerous frameworks in the market and attempt to shift to business agility via the philosophy of 'just be agile.' Typically, with this mistake you witness a lack of discipline, a few very fast and early successes, a period of over-commitment and over-promise, and ultimately a burnout and collapse of the technology organization. What many organizations that struggle with these pitfalls are truly missing is the analog to sports and the necessity of the role of the coach to artfully steer the organization through their agile journey. While it may present as unnecessary overhead to many IT organizations, I have found that especially in the early days of agile transformation there is nothing more paramount than an experienced agile Sherpa."
Agile practices have succeeded among organizations "that understand the limitations of their capacity, obsess over their customers and end users, prioritize efficiently and rapidly, test and prove hypotheses objectively, love the problem not the solution, and have transparent, measurable, and aspirational business objectives, however, do enjoy the many benefits of true business agility," Ritchie says.
The values spelled out in the Agile Manifesto of 2001 are still as relevant today as they were at the time they were formulated and published:
"We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools