What is Agile software development? Everything you need to know about delivering better code, faster

Agile software development is more popular than ever - here's where it came from, how it's being used in major organisations, and how it's likely to evolve in the future.

What is Agile software development?

Agile software development is a set of collaborative methods and practices for producing software code faster and more efficiently. In particular, Agile development uses an iterative approach, where teams continually revisit, inspect and adapt their development techniques to deliver applications that meet business requirements flexibly and quickly.

What are the origins of Agile software development?

A group of software developers created a series of lighter techniques in response to what they perceived as the heavyweight methods associated to waterfall development, where projects are broken down into a series of linear sequential stages.

These 17 developers met in Utah in 2001 to discuss these lightweight development methods. They subsequently published the Manifesto for Agile Software Development, which outlines a set of values for developing software in a flexible, iterative manner.

SEE: Guide to Becoming a Digital Transformation Champion (TechRepublic Premium)

These values centre on self-empowerment, collaboration, responsiveness and creating working software solutions – rather than final products – that can be honed over time. The agile manifesto has 12 principles, ranging from continuous delivery of software, trusting individual contributions, and onto reflective team processes.

Is Agile better than waterfall software development?

Just about every conference features a session with tech leaders – whether that's CIOs, IT directors or IT managers – who expound the benefits of Agile methodology. Sessions on the virtues of traditional waterfall techniques are thinner on the ground these days.

One explanation for the shift is that waterfall development require a tighter focus on stages: testing is undertaken after the build phase is complete. In Agile, testing is an iterative process, where software is developed, used and updated. Many CIOs believe this approach is a good fit for digital transformation, something we'll return to below.

Yet the rise of Agile methodology doesn't mean waterfall is dead. Some projects still have clearly defined stages and deliverables. If you know exactly what you need and when, then a waterfall methodology approach might be better. Perhaps the better question to ask isn't which methodology is best but to instead ask which is better suited to the task at hand.

Why is Agile development so popular for digital transformation projects?

Digital transformation has moved from the periphery to the core of all organisations, as tech leaders have sought to change their business models in response to fast-changing conditions.

Think, for example, of incumbent firms being challenged to respond to disruptive startups. Then think of the impact of the coronavirus pandemic and the rapid shift to remote work, e-commerce and e-learning. Experts suggest Agile methodology is a great fit for companies that want to test and develop new business models and products quickly.

How do companies use Agile software development?

Tech analyst Forrester says Agile delivery is critical to successful digital transformations, yet the best enterprises go even further. The researcher says just 47% of less successful firms have 75% or more of their development teams using Agile software development practices compared to 93% of successful companies.

Here are some examples of digital leaders who have embraced Agile development as a way to help their organisations transform:

What are the main features of Agile software development?

An agile team is defined by its collaborative approach and some of the main frameworks associated to the methodology include, but are not limited to:

  • Lean – Empowered teams that work quickly to eliminate waste. Lean itself originated from lean manufacturing processes, which were pioneered by Toyota and the 'just-in-time' production cycles of the twentieth century.
  • Kanban – A lean method for managing work where items, such as features, user stories and deliverables, are visualised on a board.
  • Scrum – Agile framework for small teams of 10 or fewer, who break work into time-constrained chunks, known as sprints, where progress is reviewed in scrum sessions.

Notable Agile development practices include: backlog, which is a breakdown of work that needs to be completed; standup, which is a daily meeting to communicate issues; and retrospective, which is held at the end of each iteration to consider lessons learnt.

What's the difference between DevOps and Agile software development?

DevOps is a combination of software development and IT operations. By using continuous delivery and constant iterations, DevOps aims to create higher quality software. Many of the key practices of DevOps can be traced to Agile software development.

While the Agile method focuses on sprints that can last weeks or months, DevOps is centred on hyper-quick releases that take days or even hours. Both DevOps and Agile can be used in tandem as they complement each other.

What about the rise of Agile leadership techniques?

The coronavirus pandemic has in many ways been a beta test for the widescale rollout of Agile development. Self-empowered employees simply had to work in a socially distanced manner to complete projects and develop products as quickly as possible.

Many CIOs report that Agile has been a great fit for the new working normal – and they've adopted leadership approaches to support this shift. This flexible form of leadership – known as either agile project management or agile leadership – involves the application of the principles of Agile software development to management tasks, relies on decentralised decision-making.

Agile management produces benefits in two key ways: it gives workers the empowerment that research suggests they crave, and it frees up leaders to focus on higher-level tasks, such as refining strategy and developing new business models.

What are the downsides of Agile development?

Amazon's Jeff Bezos believes every internal team should be small enough that it can be fed with two pizzas. Various other experts suggest Agile works best in small groups, yet there's a danger that, as Agile shifts from the IT department to the wider business, it becomes applied too broadly and its benefits are watered down.

Executives, who are fed up with the process-driven lethargy that undermines many big enterprises, often like the sound of self-empowered teams. But what they can get – if they don't apply Agile carefully – is chaos. Analyst Gartner refers to the danger of corruption, where the core values of agile are debased by misunderstanding.

Even in the IT department, not everything is rosy. The collaborative nature of the methodology means face-to-face interaction is often essential, something that has been impossible recently. In many ways, we're not seeing true Agile in full effect.

"The thing I miss the most is creativity; being able to stand in a room with a brown bit of paper and a load of sticky notes and argue and debate until you've got the solution to a problem," says Boots CIO Richard Corbridge, who is a big advocate of Agile techniques.

What are the long-term prospects for Agile development?

As companies emerge into the post-COVID age, it will be interesting to see what business leaders think of the products that the Agile software development team has created.

Will they reflect on the rapid digital transformation process of 2020 and conclude that Agile simply helped the business work effectively at a very complex time? Or will the shift to Agile – both inside the IT department and out across the wider business – become a permanent transition?

Evidence so far suggests CIOs and their teams have gained a lot through the shift to Agile and that the technique will be well-suited to the hybrid mix of office and home working that is likely to define the post-COVID age.

Johnson Matthey CIO Paul Coby says CIOs have been talking about the importance of Agile methodologies for the best part of 15 years. But in a competitive post-COVID age, he says agility will be crucial to supporting the business' almost-continual transformation: "They need agile IT, in the best sense of the word, to support that."