ie8 fix
madison

The bigger the system, the greater the chance of failure

By | November 3, 2010, 7:00pm PDT

Summary: IT projects will see more success as smaller, bite-size chunks, especially since software has reached a state of complexity far beyond the ability of any individual, team of individuals, or vendor to manage its development and upkeep.

Roger Sessions, CTO of ObjectWatch, provides some controversial views on what he sees as a looming meltdown of IT systems, brought on by out-of-control complexity. In an interview with NetworkWorld’s John  Dix, he issues a warning, and even discusses the role of service oriented architecture.

IT projects will see more success as smaller, bite-size chunks

Perhaps we should call it “Session’s Law”: “the larger and more expensive an IT project is, the more likely it is to fail. A system that costs less than $750,000 “has a good chance of succeeding,” but a system or project exceeding $2 million “has less than a 50 percent chance of succeeding… by the time it gets much larger than that, the chances of success drop to near zero.”

For a wealth of insights on classic examples, as well as how to avoid IT project failures, check out the site of my colleague Michael Krigsman.

“Most failures are in the $2 million to $4 million range,” says Sessions. In the interview, he plugs a methodology he and his company developed, but the gist of his argument is that systems projects will see a higher chance of success if they’re kept in smaller pieces.

Sessions makes sense, and major projects and processes are better off served up as bite-sized chunks. There’s nothing new about the “chunking” concept, of course — Henry Ford broke automobile assembly down into chunks when he mastered the assembly line.  That was the beginning of chunking of production processes.

We saw the chunking of operational or office-based processes with the advent of the computer. When I was director with the Administrative Management Society in the late 1980s and 1990s, the chunking of white-collar and administrative tasks into more standardized and automated tasks (word processing, reporting, database management) was sweeping through organizations.

Tom Peters, co-author of In Search of Excellence, and author of Thriving on Chaos, and my favorite, Liberation Management, talked about chunking as a strategy for breaking up complex tasks and projects into manageable components. Now, we are talking about the chunking of software itself.  This seems to be part of a very inevitable and natural revolution, since software has reached a state of complexity far beyond  the ability of any individual, team of individuals, or vendor to manage its development and upkeep.

And, service-oriented architecture is the process of breaking applications into bite-size chunks to be delivered when and where they are needed. The question is — can we achieve simplicity by breaking down complex software operations into manageable, bite-size chunks? Or will things end up as brittle as the CORBA (Component Object Request Broker Architecture) efforts of the past decade?

Interestingly, in the interview, Sessions talks about the role of SOA in IT complexity, observing that companies have tended to build SOAs though arbitrary “decompositional design,” versus a more predictable mathematical approach. It’s this randomness that sinks many SOA efforts, he says.

The bottom line is big systems have become too unwieldy.

Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.

Topics

Joe McKendrick is an author, consultant and speaker specializing in trends and developments shaping the technology industry.

Disclosure

Joe McKendrick

Joe McKendrick is an independent consultant, editor and speaker.

Joe has performed project work (white papers, articles, blogs, research and presentations) for the following companies in the IT marketspace:

  • CBS Interactive/CNET/ZDNet (this blog)
  • ebizQ
  • Evans Data
  • Gartner
  • IBM
  • Informatica
  • IDC
  • Microsoft
  • Systinet/HP
  • Teradata
  • Unisphere Reseach, a division of Information Today, Inc.
  • WebLayers

Joe has also performed research work for the following sponsoring organizations in partnership with Unisphere Research, a division of Information Today, Inc.

  • IBM
  • Luminex
  • Noetix
  • Oracle Corp.
  • Teradata
  • Informatica
  • International Oracle Users Group
  • Oracle Applications Users Group
  • Professional Association for SQL Server
  • International DB2 Users Group
  • International Sybase Users Group
  • SHARE (IBM large systems users group)

Biography

Joe McKendrick

Joe McKendrick is an author and independent analyst who tracks the impact of information technology on management and markets. Joe is co-author, along with 16 leading industry leaders and thinkers, of the SOA Manifesto, which outlines the values and guiding principles of service orientation. He also speaks frequently on Enterprise 2.0 and SOA topics at industry events and Webcasts, and serves on the program committee for this year's SOA & Cloud Symposium in London. As an independent analyst, he has also authored numerous research reports in partnership with Unisphere Research, a division of Information Today, Inc. for user groups such as SHARE, Oracle Applications Users Group, and International DB2 Users Group. In a previous life, Joe served as director of the Administrative Management Society (AMS), an international professional association dedicated to advancing knowledge within the IT and business management fields. He is a graduate of Temple University.

Related Discussions on TechRepublic

Did you know you can take part in these discussions with your ZDNet membership?
2
Comments

Join the conversation!

Just In

Conceptual integrity
jorwell 6th Nov 2010
Frederick Brooks (author of "The Mythical Man Month") expressed the view that the key to conquering complexity was by designing with conceptual integrity in mind.

This means designing a complex system as a whole. Obviously it has to be built in stages and some parts can be implemented before others, but the basic design concept has to be there and should be adhered to throughout the system.

I think the idea that you can build a complex system by building lots of individual pieces and then sticking them together somehow at the end is unlikely to work in practice. Nobody would think of building a complex system like a plane or a ship like that, so why a computer system?
Be care what you ask for. Will a 100 small projects be more effective than 10 large projects for enterprise class solutions? If both project sizes use the same technology base (SOA, BPEL, etc.), it really comes down to management and system engineering capability not project size. If the 100 small projects must be "integratable" into an enterprise class solution, then significant up front system and interface engineering is required prior to start of development of each project. In a large project, approach much of this engineering can be allocated to the development project. It varies by company which approach is better, based on the management and engineering skills of that company.
0 Votes
+ -
Conceptual integrity
jorwell 6th Nov 2010
Frederick Brooks (author of "The Mythical Man Month") expressed the view that the key to conquering complexity was by designing with conceptual integrity in mind.

This means designing a complex system as a whole. Obviously it has to be built in stages and some parts can be implemented before others, but the basic design concept has to be there and should be adhered to throughout the system.

I think the idea that you can build a complex system by building lots of individual pieces and then sticking them together somehow at the end is unlikely to work in practice. Nobody would think of building a complex system like a plane or a ship like that, so why a computer system?

Join the conversation!

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]
ie8 fix
Click Here
ie8 fix

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources
ie8 fix
ie8 fix