Tim Bray's Enterprise 2.0 conundrum

Tim Bray's Enterprise 2.0 conundrum

Summary: How does one create small systems that address similar requirements as large ones? It's a basic matter we need to consider.

SHARE:

Yesterday, I reprinted an excellent and thought-provoking blog post by Tim Bray, one of the inventors of XML.

Tim's post represents a reality check for traditional enterprise software, suggesting we could avoid massive IT failures by following a more web-oriented approach to IT and software development.

Tim summarizes his point this way:

The community of developers whose work you see on the Web, who probably don’t know what ADO or UML or JPA even stand for, deploy better systems at less cost in less time at lower risk than we see in the Enterprise.

He goes on to compare the culture of web development ("starting small, gathering user experience before it seems reasonable") to that of traditional enterprise IT, where he says failure rates "remain distressing."

Tim then argues against building large-scale custom software unless absolutely necessary (a premise I wholeheartedly support) and finally concludes that "we need to do it better." In this case, improvement means philosophically adopting an end user, consumer-oriented approach to software creation.

In Tim's words:

Suppose you asked one of the blue-suit solution providers to quote you on building Ravelry or Twitter or Basecamp. What would the costs be like? And how much confidence would you have in a good result?...

The point is that that kind of thing simply cannot be built if you start with large formal specifications and fixed-price contracts and change-control procedures and so on. So if your enterprise wants the sort of outcomes we’re seeing on the Web (and a lot more should), you’re going to have to adopt some of the cultures and technologies that got them built.

THE PROJECT FAILURES ANALYSIS

Tim correctly characterizes traditional IT projects as big, expensive, cumbersome, and failure-prone. However, he almost (but perhaps not quite) strays into the realm of utopianism by suggesting that organizations can successfully build large systems as they might construct a relatively simple consumer service such as Twitter.

Blogger,

A year or two ago someone "discovered" that a very common, important record in one of our internal systems had 44 (or something like that) required fields, and decided that this was ridiculous. A team was formed to streamline the processes associated with this record by reducing the number of fields. A detailed process audit was conducted. It turned out that every single one of them played a critical role in a downstream process. All the fields remained, and some old timers smiled knowingly.

We should also remember that teams upgrading old systems or building replacement applications are saddled with costs related to old, messy infrastructure. The folks building Twitter, to use an example, didn't face that hurdle.

HP Fellow and Enterprise Irregular colleague, Charlie Bess, summarized this point:

Part of the problem is the legacy burden. Not only are [projects] burdened by previous failure, but also software for on-going concerns is burdened by previous success. That's why an organization that starts over from scratch can take advantage of their steep learning curve -- "innovators dilemma." They don't have that installed base and previous success holding them back. That's why so much of IT budgets are locked up in "fixed costs".

This all leads to a clear conclusion: simply cutting features from software is not the solution to IT failure because organizations often retain the need for that functionality.

In theory, perhaps we could solve the problem by a wholesale redesigning of IT into federations of small systems that somehow all work together like magic. But that just ain't gonna happen anytime soon.

Well-known enterprise architect (and fellow Enterprise Irregular), Dion Hinchcliffe, suggests that service-oriented architecture (SOA) offers a step in the right direction:

Allowing interoperability and data sharing between IT systems ultimately leads to seamless functioning together as a unified whole. It's not a panacea, but can reduce the need for endless, large-scale integration projects that always seem to go over-budget.

However, for most organizations, the barrier to SOA adoption lies in the human realm; it's not about technology.

Technology is not the problem. These problems are not technical matters, but arise from preconceived notions, deeply embedded work processes, and heavily invested expectation mismatches between IT and the lines of business they serve.

Helpstream CEO (and yet another Enterprise Irregular), Bob Warfield, amplifies the point that these failure problems are not rooted in technology:

Most of what's being said doesn't require Agile, Scrum, Object Oriented Programming, Interpreted Languages, or any of the rest of the buzzwords. Most of it can be understood by anyone who understood the Mythical Man Month. Big IT starts out anything with Second System Effect run rampant. It's a BAD idea, but it is institutionalized.

So, we combine a hopelessly flawed boil the ocean strategy with talent that isn't really up to a proper methodology and a compensation plan guaranteed to lead us further astray and we should not be surprised that disasters await.

Individual organizations can make strides to improve their own situation by investing the time and resources needed to improve the culture and processes around IT. As an industry, however, full-scale change is many years away.

My take. Without using the term, Tim's presents an Enterprise 2.0 perspective that links IT success to simplicity and improved collaboration. In doing so, he raises seemingly-intractable problems that run deep into an organization's technology, culture, and economics.

At heart, Tim asks a fundamental Enterprise 2.0 question: how does one create small systems that address similar requirements as large ones? It's a basic matter that henceforth I will call "Tim Bray's Enterprise 2.0 conundrum."

[Photo from iStockphoto]

Topics: Enterprise Software, Browser, CXO, Software, Software Development

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

5 comments
Log in or register to join the discussion
  • Very thoughtful assessment

    I totally agree with the assessment, the conundrum will continue to be with us and I don't see how it could possibly go away.

    There will always be complexity in large organizations, and IT representing that complexity will also be complex at least on the semantic level. The underlying IT architecture is another matter of course, there the complexity is driven more by security, reliability, and performance aspects.

    Some organizations manage to streamline their processes, and thereby are able to get rid of complexity in their IT systems as well. However, let's not forget that automation and user friendliness in IT systems necessarily adds internal complexity, so I would argue that in any project the right balance needs to be struck between simple systems that don't do a lot, and highly optimized systems that are expensive to build and maintain but offer a lot of automation.

    My experience is that most IT projects start small. There are good and bad reasons why some of them become much bigger over time: Because users like the applications and want to do more (can be good if the right balance is struck, see above), or because the people involved don't agree on the goals, don't work together well, don't know how to build software efficiently, etc.

    Would be nice if no IT ever became big for bad reasons, but it ain't gonna happen. However, and that is my point (and I think Michael's as well), there is a place for big, complex IT systems. The desire to live in a simple world that can be supported by Twitter-like IT is understandable but delusional as well. This really becomes a problem if complex systems are designed based on such a delusion.
    mmondo
    • I support Tim's view

      Maybe the post isn't sufficiently clear -- I think Tim Bray identified the problem. The question is, how can we achieve these apparently conflicting goals.

      Thanks for commenting.
      mkrigsman@...
  • Loose coupling: needed but often ignored

    I've been at both web startups and large enterprises,
    steering technology, and I have a similar but differently
    phrased take on the drive for simplicity. Yes, it's a people
    problem much more than a technology problem, but it's the
    way that people typically approach system construction. I'll
    need to blog about this at great length to explain it more
    fully, but a key part of the solution to the dilemma boils
    down to this: ensure, all along the line, that systems are as
    loosely coupled as possible, both internally and
    interapplication-wise.

    I know: easy to say, much harder to do. The natural
    tendency in the way most people approach systems design,
    unless pushed hard by maximally annoying (and necessarily
    influential) people, appears to be to tightly couple things:
    the new whizbang shipping module links directly to the
    finance module, because that was both fast and the
    seemingly easiest, simplest approach at the time. Changes
    to one require changes to the other. Card follows card, and
    soon you have a delicately teetering house, where
    maintenance is tricky and implementation of anything new
    almost has to, well, follow suit. [end of overly stretched
    metaphor].

    Believe me, with all due respect to Tim Bray, internet
    entities that evolved from 2-person startups are anything
    but immune to this syndrome. Yes, they often get rolling
    fast, with great functionality and innovation. But as they
    grow, they're prone to hairball-ism: they almost inevitably
    hit a wall of performance, scalability, maintainability, etc.,
    due to this tight coupling. Examples? Go to Tim's own list.

    Beating this natural entropy takes strong leadership,
    annoying and persistent and undaunted by internal and
    external forces pushing to just "get it done." There's no
    special magic in it; it's just hard, hard, often unpopular
    work. I agree with Hinchcliffe that technologies such as
    SOA can help spread the philosophy and approach of
    maximizing independence, but they alone won't solve the
    problem, anymore than acquiring an expensive PPM tool
    magically transforms an organization into efficiency.

    So yes, I agree that wholesale change in all this is many
    years away, despite the lessons we've learned as an industry
    for the last four decades or more. I'm writing about this
    now on my "CTO/CIO Perspectives" blog, by discussing key
    "human factors in IT" books over that period of time:
    http://www.peterkretzman.com

    Post is at http://bit.ly/8YyNgw

    --Peter Kretzman
    Peter Kretzman
  • Will Simplifying Make the Problem Go Away?

    First, I'm a firm believer in the concept of "divide and conquer" --- i.e. take a big problem and see if it can be broken into a bunch of smaller problems. Usually, this makes the problem more understandable, and solvable. However, I'm also sharply aware that by doing so, I've transformed one type of complexity into another type.

    Am I better off? Who knows? For some, this transformation either pushes the problem into the future, or makes it someone else's problem.
    For me, it seems to be a matter of "choose your poison;" because the transformation didn't transfer the complexity problem onto someone else.

    Complexity can be a function of problem, business, a work process problem, any type of problem. For some situations, the complexity of solutions may be well-warranted because the reality is complex. A process for one organization may be more complex than for another because X, Y, and Z was instituted in response to a serious problem. Let's say the conditions which necessitated X,Y, and Z is reflective of the business environment. Adopting some other organization's simpler process, which reflects the realities of some other business will result in serious problems.

    Sometimes when we simplify or even if the task is "simple", we forget to consider the people and things that affect or get affected. So "the job" might be successful, but the organization gets disrupted.
    elizab
    • RE: Tim Bray's Enterprise 2.0 conundrum

      @elizab

      How simple is E = mc2?

      "Simple Solutions" to "Complex Problems" is what we need and this is not going to came by more reductionism and analysis but by synthesis, this is the simplicity on the other side of the complexity, and this kind of thinking is totally lacking on enterprise IT.

      Take a look at http://changethis.com/22.SimplicityCycle

      Cheers,
      Juan.
      juan.bernabo@...