Buy vs build: The pendulum swings
special report So, your back-end systems are showing signs of age? Put down the chequebook; David Braue finds that when it comes to building business apps, many companies are putting themselves back in the driver's seat.
Contents  The pendulum returns ...and in this corner... The vendors' response, and yours The resurgence of "build" Sidebar: Finding a healthy balance Sidebar: Get vertical Executive summary | ||||
Developed over several years and piloted last year at 43 Tasmanian branches serving 250,000 customers, the new system will be the culmination of hundreds of thousands of man-hours' work. Ultimately, the .NET-based system will have around 21,000 users in over 1000 locations, a scale made even more remarkable by the fact that CBA built the entire system itself from the ground up.
For such a major investment, and such a critical information system, the genesis of CommSee is a considerable break with tradition. In-house development used to be the only way for a company to get the applications it needed, but with a diversifying and maturing software market offering many rich off-the-shelf CRM systems, few companies can still muster the determination to build such a major system themselves.
There are other reasons for the flight from in-house development: particularly since Y2K remediation made the world collectively realise the potential consequences of poor development, risk-averse large organisations have shifted towards buying major systems from established vendors -- if only so CIOs might have someone else to drag into the swamp if things went pear-shaped.
Clearly, the CBA believes its team of more than 150 developers is up to the task of development, which has derailed both smaller and larger efforts in the past. And with the system having scaled past 30,000 users during stress testing last year, it's possible that the potentially career-ending decision by CEO David Murray could actually pay off as expected rather than going down in flames like so many projects before it.
It might seem brave to some and foolish to others, but CBA's in-house renaissance reflects what some see as a growing trend towards building increasingly complex applications in-house. Noting research that has consistently found 25 percent to 30 percent of companies would prefer to build their own applications, research firm Gartner recently projected that maturing development tools and growing dissatisfaction with packaged solutions would see fully 20 percent of companies building their core application components by 2008.
That's by no means the majority, but it represents a significant change after years during which packaged application vendors had to work to carve out their niches and to build attractive value propositions. Thanks to ever-growing functionality, ERP systems' gradual expansion across other related areas of the business, and Y2K marketing campaigns that convinced business executives the old way of building systems was fundamentally flawed, packaged application vendors successfully turned off-the-shelf systems into a big business worth billions annually.
Consequently, internal development teams moved from positions of central responsibility, to building quick-and-dirty applications and implementing the often painful integration of new packaged solutions with existing systems.
While Y2K fears may have tipped the scales of the buy vs build question in vendors' favour, they also raised corporate awareness of the need for risk management -- and this meant improving IT project management, which is essential to risk minimisation. Development became less an exercise in verdant creativity than a carefully managed exercise in which code is built to precise requirements and projects are, ideally at least, managed to tight budgets with clear deliverables.
This discipline seems to have helped developers get better at delivering on their promises. Standish Group International, a US-based company that since 1994 has tracked the results of more than 40,000 IT implementations, reported last year that fully 34 percent of all projects were judged successful -- more than twice the 16 percent rate in 1994. Just 15 percent of all projects were deemed failures, compared to 31 percent in 1994. These figures included both implementations of packaged software and in-house development projects.
Although there's still a lot of room for improvement, the recent upswing in project completion suggests that a gradually growing body of expertise is helping implementers avoid many of the mistakes of the past. With that increasing competence, many companies are gaining the courage to question whether buying core business systems off the shelf is actually the right thing to do. And while the "buy" decision still dominates, the resurging "build" philosophy -- which will get a big shot in the arm in Australia if CommSee works -- is forcing IT planners to give serious consideration to the in-house approach.
That is not to say that building in-house is always the best solution, either; as any risk-aware executive knows, each case must be considered separately.
Gartner, for one, paints the buy vs build decision in terms of four basic issues: differentiation (how can internal resources be best used?); new structures and directions (can internal skill sets continue to keep pace with technology?); opportunity management (how will strategy address future opportunities?); and market forces (how will staff dilution affect the continuity of in-house projects?).
Countless white papers on the subject paint these four issues in many different contexts. Interestingly, return on investment (ROI) is not so much of an issue -- but try telling that to vendors. In white papers published by the ream, many vendors paint the buy vs build equation as a simple ROI issue. Paying a team of developers to build a big application, they argue, will incur both developers' salaries and licensing fees for a wealth of tools such as core application servers, integrated development environments, and so on. By buying a commercial system instead, they argue, much of that expense can be avoided -- particularly as systems get more complex and the perceived difficulty of building them in-house increases.
The wealth of options in the market lends credence to the "buy" argument. Natural competition has pushed down prices while feature sets improve, so much so that it no longer makes sense to build many types of applications that were once typically brewed in-house. For example, portals, content management systems, business reporting, and even customer relationship management were often handled in-house but are now well addressed by both commercial and open source solutions.
Underlying vendors' statements about ROI is another often fallacious assumption: that costs will blow out because systems built inhouse are likely to run over budget and take longer to implement than packaged solutions. In the past, this was certainly true: the 1994 CHAOS survey found average cost overruns of 180 percent. But by the 2004 survey, Standish Group figures pegged this at just 43 percent. This is still a large figure, but reflects growing project management competency within inhouse development organisations.
Vendors' ROI-based figures also ignore one significant issue: because they've been built to suit all sorts of commercial environments, packaged applications often include far more functionality than most customers will ever require -- yet still pay for. When building such systems in-house, the scope of the project can be tailored far more closely to actual business needs, resulting in a smaller system that can probably be built faster and more cheaply than the purchased system.
From this perspective, the buy vs build comparison is one of apples versus oranges -- and building suddenly seems a lot more attractive. However, actually getting the system running is only one part of the whole equation; in the long term, the issue that invariably comes back to bite any company is maintenance -- patches, upgrading, and so on.
Simply keeping a system up to date can be a Herculean effort, and can be a veritable show-stopper if internal development teams don't have adequate resources. Maintenance becomes particularly problematic if customisations to packaged applications interfere with future updates from the vendors. To avoid this problem, it may be desirable to build enhancements to commercial systems as add-ons, rather than actual modifications to the application core.
"During the last three years I have worked only 18 months, the strain on my partner, my wallet and myself have been immense. Try explaining to a bank that you're not a high risk for a home loan (they're not interested).
As someone who has been in the industry for over ten years I appreciate the need to keep skills up to date BUT, there's a big difference between ongoing professional development and spending all your spare time studying. I for one cannot afford thousands in industry courses while not working. "
-- Charles Boorman
Rather, consider the incentives driving each type of development. Vendors held to a fixed period contract will move heaven and earth to finish a project as quickly as possible to avoid non-compliance penalties, and to move onto the next project. Internal developers may feel more freedom to take their time -- particularly since specifications may vary by executives ordering additional features that a vendor might refuse as being out of scope. Scope creep can be minimised or avoided altogether by strict adherence to project management discipline and formal development frameworks, but many companies lack this discipline whereas outside vendors typically have many suitably trained project managers.
Software development is no longer child's play, and internal development teams need to follow the same sorts of project management and risk minimisation strategies as third-party integrators. Ultimately, the question is not whether internal development teams can deliver a working system, but whether they can do so to produce a working, documented, supported, flexible system that fulfils the business needs and the requirements of corporate governance.
The issue is not coding sophistication, but management sophistication -- and in this area, many companies are still coming up short. For this reason, it is for expertise, as much as the capabilities of the packaged solutions, that many companies elect for the "buy" option.
"There's certainly a preference to buy rather than build, especially with larger customers or customers who are risk averse," says Jeremy Horey, senior consultant with systems integrator Dimension Data.
"There are exceptions, but you don't see nearly as much [internal development] as you might have five years ago. People have become more confident with products as they've matured, and we've gotten better and better tools for development. But we still haven't got the right tools to extract the needs and wants of users so we can turn them into code. You spend a lot of time finding out what people actually want."
Given the intense competition of the enterprise applications market, constant pressure to innovate has driven vendors to find new ways of meeting customer requirements. A few years ago, this meant providing a broad set of tools that could meet any possible need. These days, however, smarter customers are demanding customisation and more appropriate solutions for their needs -- driving many vendors to offer templates and entire consulting organisations geared at specific vertical industries.
In short, vendors are getting better at tailoring systems to customer needs -- potentially taking away some of the perceived advantage of in-house systems. Vendors, of course, can never have the intimate organisational knowledge of an internal development team, but they will do their best to work around this fact. This is particularly the case given many companies' growing determination to build many components in-house.
One way that vendors justify and expand their role within customer environments is to construct broad technological frameworks on which their products can build. Standards compliance ensures interoperability between solutions, while vendor-specific innovation ensures some degree of customer lock-in.
"Vendors are responding to users' requests for making sense of their IT environments by offering an IT ecosystem with a comprehensive software, hardware, business app, infrastructure, and service offering to essentially solve your broad IT requirements," says Gartner analyst Betsy Burton. "We're starting to see more integrated environments coming from vendors. The fascinating thing about an IT ecosystem is that you're actually getting in much deeper than you ever have in the past."
Lock-in can be a major issue for many companies. Just how much you commit to any one vendor depends both on how much their technology vision matches your business plan, and on your perception of the risk any particular choice presents. Rather than simply juxtaposing "buy" and "build" options, for example, Gartner identifies six different approaches to system implementation. These include buying the technology stack (application server) and applications from a single vendor; buying applications from a single vendor to run on top of a separate technology stack; linking one vendor's application backbone to many best-of-breed applications; and buying separate best-of-breed applications and technology stacks. In-house development and hosted applications are the remaining choices.
Each of these choices carries its own risk profile and trade-offs. A broad suite of prods, for example, may integrate well with underlying infrastructure in fundamentally useful ways. But it can also be limiting, for example tying future development to a specific technology framework that cannot be easily changed down the road. SAP, for example, is delivering tight and effective legacy application integration with its NetWeaver framework, but just try to switch to Oracle Business Suite after a few years' investment in NetWeaver. Rampant customisation and tight integration between systems will make it nigh unto impossible.
Disturbed by the thought of handing over the reins to one vendor, some IT executives may pull development inhouse or intentionally select a variety of best-of-breed applications to keep all the vendors honest. Others, of course, will focus more on bottom-line questions of business availability rather than philosophical ideals. For such pragmatists, the choice to build or buy systems comes down to a question of risk -- not return on investment, which is usually a best guess and can conceal many unknown costs that spring up down the track.
If you have a small or over-extended internal development team, the question of whether to build anything more than simple applications may be academic. But if your team has the resources, from a cost perspective it may be immediately better to build many types of systems, then manage them throughout their lives using any of the many project management and software development lifecycle methodologies on the market.
Strong arguments in both directions make it difficult to fall on one side of the build-or-buy equation. Partnerships with providers may lead to easier implementations and long-term management, but it's important to be able to build your own components where desirable.
Working both sides of the fence has also become easier with the increasing competence in J2EE and .NET frameworks, which both provide easier application development and integration with commercial application environments. This way, in-house developers might tackle smaller and clearly defined projects, while paying vendors for the application frameworks and associated skills that the vendors have in spades. Consulting partners also typically have strong expertise in areas such as quality assurance, legislative requirements, corporate governance guidelines, and other matters where internal teams could likely use some refresher courses.
Relationships with vendors, however, must be continually managed, whether a company has elected to build or to buy its systems. Whatever choice your organisation makes, its success will be determined by how well you balance between competing interests.
Just remember: vendors are generating enormous profits from selling you software and tailoring it to your environment. That money also buys peace of mind by giving you access to particular expertise or technologies that can help your business. If you can buy what you need but build -- and maintain -- what makes sense, you will strike the right balance.
"If organisations are attentive, they can absolutely make it work," says Burton.
"But the problem with the word 'partnership' is that they tend to think vendors have skin in the game. Most vendor relationships aren't really partnerships; they are buy-and-sell relationships. This means you have a responsibility to manage your vendors much more aggressively than in the past. They have an ability to get much deeper into the roots of your company, so the IT manager has to be a trusted guide that makes sure that risks and rewards are managed within their organisation," he adds.
Like every IT executive, Mary Wollmering, CIO with Melbourne Health Shared Services (MHSS), knows how hard the build vs buy decision can be. After years making such decisions, however, she has developed a pretty good sense of when it's better to go one way or the other.
MHSS, which manages IT infrastructure and applications for thousands of employees across several Melbourne hospitals and healthcare sites, has a team of six mainframe developers and five .NET and Web development specialists. Together, they've delivered a number of systems providing critical functions such as patient transport bookings, staff scheduling and the like. They're also involved in day-to-day maintenance of the organisation's complex applications environment.
However, when it comes time for major new applications like a massive forthcoming digital imaging environment, tenders are the way to go, both because public sector regulations demand it and because it makes the most sense. The organisation must also deal with outside pressure: the many stakeholders in a shared services environment want to maximise their value-for-money and may see building applications as a way of doing that, without appreciating the finer issues Wollmering knows so well.
"Sometimes it's so much quicker and less expensive to do quick things in-house, and developers have the corporate knowledge that saves you time," she says. "But it's risky to do too much in-house because you become very vulnerable. My main issue with development in-house is the maintenance of it all. Patches, updates and so on become more and more difficult when you've modified a system. You need vendor support, service level agreements, and so on -- but be careful. Vendors aren't always responsive to what you want, so you have to work hard to get what you want."
Knowledge transfer is also a critical issue: deciding whether to build an application inhouse may ultimately come down to the question of whether the skills to maintain that application are readily available in the market. To ensure that in-house projects run smoothly, Wollmering's team has gained considerable discipline when it comes to development. All applications have to be documented, for example, and the team recently revisited all of its disaster recovery plans to ensure that knowledge could be easily transferred.
"You really don't want to be in the position where you've developed [a critical application] in-house and somebody leaves," says Wollmering. "It all goes back to the argument about risk, and balancing it effectively."
The Federal Government wants to get to know the Australian software industry and map out a strategy to make it "get vertical".
The Department of Communications, Information Technology and the Arts (DCITA) has released a tender seeking consultants to help analyse the Australian software industry, with particular emphasis on the vertical applications market and its growth potential for Australian information and communications technology (ICT) companies.
A previous study by analysts McKinseys -- undertaken as part of the government's "Framework for the Future (F3)" -- concluded that "the development of specialised software applications and the provision of specialist skills and services to multinational companies represent the greatest opportunities for Australian firms to participate in the global ICT industry."
The new study will focus on applications for vertical markets, which McKinsey identified as offering "the best opportunity for globally oriented specialist firms in Australia".
Vertical applications are typically focused on a specific industry and are closely associated with the work done in that industry. The McKinsey study singled out the resource, transport, healthcare, retail, and energy sectors as "having potential for vertical application developers in Australia". With the proposed consultancy, DCITA wants to investigate the Australian software industry and the potential for Australian firms to succeed in vertical applications markets.
DCITA also seeks to develop "a map of the Australian software industry, including the concentration of software firms and capabilities, significant strengths and weaknesses, and relationships between companies, key customers and the innovation base."
DCITA also wants an assessment of selected key applications markets and the scope of these markets "as a basis for sustainable growth for the Australian software industry".
The consultancy is expected to identify the markets the Australian software industry is involved with, the broad levels of activity and key competencies.
"The objective is to provide a clear picture of the relationships between the major players or groups of firms and other organisations and drivers in the value chain, including significant suppliers; customers; multinational corporations and the research training sector".
The chosen consultant will start work upon execution of the contract with DCITA and conclude by June 10, 2005.
10 questions to ask during the buy vs build decision.
- Are you really that good at writing code?
Your developers may be good, but do they really have what it takes to build, test, deliver, and maintain a complex business app? Say what you want about vendors, but they have a history of making it work. - Do you need all that functionality?
Commercial applications may have loads of bells and whistles, but most companies rarely, if ever, use them. Don't pay for features you don't need; if your requirements are limited, it may make more sense to build a smaller system that satisfies your current and near-future needs. - Can you maintain the systems?
Vendors upgrade and patch their systems all the time. If you're building in-house, you'll need to build a similar regime for identifying bugs and updating deployed applications. This requires specific expertise so make sure you have it. - Are your skills too concentrated?
Flight of expertise has been the death of many a project. If you can't depend on your developers to stick around, make sure everything they write is well documented so that knowledge can be passed on to future staffers; otherwise, in-house applications will become difficult if not impossible to update. - Can a vendor provide additional skills you don't have?
When you buy an application, you can typically get access to much more than just the software. Your company may benefit from project management skills, consulting about legislative or corporate governance requirements, or even continual education about technology changes. - ROI is about more than implementation.
Many companies may shape their buy vs build decision around the relative cost of each option, but the real cost comes in the long term. Don't forget to factor in these expenses during your decision making. - Frameworks can be useful.
Seeking to own a bigger piece of the enterprise pie, vendors are building out application frameworks for developers and other applications to work with. Paying for such frameworks -- then building on top of the -- can ease standards adherence, improve integration between application components, and retain the flexibility needed to match future business growth. - Keep managers under rein.
One of the biggest difficulties with in-house projects is scope creep, particularly when it comes from variations ordered by other managers with little sense of how much technical challenge their orders require. Effective business-IT communication will retain a necessary sense of perspective around in-house development projects. - It's not a black or white decision.
Even when you buy software, you're likely to end up building interfaces, customisations or related applications to fill in functional gaps. Ensure you have adequate internal development expertise, or access to a company that can provide it for you. - You can do it.
If you really see value in building your own solution -- whether as a competitive weapon, or out of desire to own your core business systems -- you can probably make it happen. But it's not easy. Make sure you're doing it for the right reasons, you have clear expectations, and that you have enough controls and guidance in place to ensure you get what you need.
This article was first published in Technology & Business magazine.
Click here for subscription information.