Why enterprises struggle with agile software development, and what they can do about it

Want to be agile? Then run your technology operations like a startup or an internet company.

That's the gist of a recent podcast featuring Santiago Comella-Dorda and Gerard Speksnijder, both with McKinsey, who shared their observations with McKinsey colleague Roberta Fusaro.

They discussed the challenges of agile, in which developers and business users work closely on fast-paced, iterative software releases. "The idea around agile is to create small, cross-functional, self-contained teams that deliver technology in quick increments," said Comella-Dorda. "And because they're self-contained, they feel accountable, and that's truly the magic of agile, a team that really feels accountable for the outcomes they're producing."

So why are established enterprises having a tougher time with agile than startups and internet companies? For starters, startups don't have legacy systems and processes. "They don't have the application-architecture legacy. There are no monolith applications," says Speksnijder. "Everything typically is being defined in a pretty modular fashion, with lots of microservices, APIs, which allows you to make changes to the specific component of the application architecture. You can test it and release those features quite fast and without having lots of dependencies on other parts of your application landscape."

The problem is, agile hasn't been able to scale across larger enterprises, said Speksnijder. Agile efforts usually succeed in small teams, but it's difficult to engage as wider parts of the enterprise -- especially non-IT teams -- get involved. Agile becomes unwieldy "when it gets outside of the realm of IT, and you really have to take the whole enterprise and become an agile organization as a whole," Speksnijder says, "We find that the biggest barriers to success are around organizational structure, around how to resource, If you solve for that, then you're really able to get to agile at scale."

The McKinsey team has created a four-point program to help scale agile into today's enterprises. (More details are available within an article they published on the topic as well.)

Take a product-oriented, not project-oriented, approach to agile. Agile attempts at mainstream enterprises tend to be centered around projects, which keeps things in the IT domain, they observe. Instead, "organize IT resources around products, gathering business-unit leaders, developers, and other members of the organization in stable end-to-end teams that are focused on delivering designated business outcomes."

Improve interactions between the business and IT."Closer collaboration can be achieved by designating strong product owners from the business units to work with IT -- individuals who understand the company's products well and who have the technical knowledge and authority to prioritize feature changes in products."

Redefine managerial roles and responsibilities. About half of successful agile organizations studied by Comella-Dorda and Speksnijder "have redefined managers' roles and responsibilities to account for the distinct capabilities associated with agile." Under the old waterfall approach, for example, a manager "typically needs to coordinate a range of tasks occurring across application-development teams, database teams, and so on," they said. "Under an agile approach, however, the number of tasks (and therefore the need for coordination) is minimized. The tasks that remain are handled by a strong product owner or the agile team itself."

Redesign budgeting and planning models -- and consider a "venture capital" approach. Comella-Dorda and Speksnijder suggest that companies tear up their traditional annual budgeting and planning exercises. "Some businesses in our research base are taking a different approach," they observed. "Road maps and plans are revisited quarterly or monthly, and projects are reprioritized continually." In addition, "a handful of companies are even exploring a venture-capital-style budgeting model. Initial funding is provided for minimally viable products, which can be released quickly, refined according to customer feedback, and relaunched in the marketplace -- the hallmarks of agile development. And subsequent funding is based on how those products perform in the market."