Ever Wonder Why Upgrades Are So Damn Hard?

Ever Wonder Why Upgrades Are So Damn Hard?

Summary: It’s no secret that every vendor needs its customers to upgrade early and often. While much of this requirement is centered around support costs – vendors have trouble banking all those big maintenance bucks they’re earning if they have to support too many older versions of the software – long-term strategy is also at play.

SHARE:
TOPICS: Oracle
11

It’s no secret that every vendor needs its customers to upgrade early and often. While much of this requirement is centered around support costs – vendors have trouble banking all those big maintenance bucks they’re earning if they have to support too many older versions of the software – long-term strategy is also at play. If a vendor can’t convince its customers to upgrade to the latest and greatest stuff that has emerged out of the R&D labs, then how can it justify those expenses – and the marketing bucks that have to be spent to flack the new stuff – to their shareholders.

Meanwhile, upgrading is hard. And potentially costly. And complicated. How complicated was just driven home by a study recently posted by Helene Abrams of eprentise on the company’s website. eprentise is a software vendor that is commercializing tools to help companies navigate the upgrade process. As such they have an interest in exposing the complexity of the problem – but I can assure you her data is no exaggeration, nor is it unique to the Oracle Applications she specializes in.

What Helene has done is build a hand-dandy little table that shows the growing complexity of Oracle Applications over the last five revs, across four basic parameters: the number of modules, objects, columns in the database, and the number of constraints. These numbers are revealing of a problem that dogs the entire market and its collective user base. Oracle 10.7 had 35 modules, while, four revs later, rev 11.5.10.2 has 258. Meanwhile, the number of objects went from 18,072 to 319,715. And the number of columns in the database went from 156,607 to 2,161,603. (The full chart is reproduced here.)

These are order-of-magnitude changes in complexity that are further exacerbated by the tendency of users to significantly upgrade the number of users in their systems – it’s not uncommon to hear of a 50 percent increase in total number of users during an upgrade. And, with most upgrades – Oracle, SAP, and others – involving a shift to service-oriented architectures, the complexity of linking up external services to the internal enterprise suite adds yet another level of complexity.

I could go on and on about this problem. Helene is especially interested in what impact M&A and other business dislocations add to the problem. Other problems crop up to bedevil the upgrade process: Companies like IntelliCorp sell software tools that help manage the upgrade (or sunsetting) of customizations, as well as provide an understanding of what processes are actually being used or not used, and therefore ripe for sunsetting – in this case in an SAP system. And the vendors themselves keep throwing new tools and services at the problem as well.

But, as Helene’s data show, the tools that vendors are bringing to market are focusing on a very fast moving target. Too fast moving by some measures. But that’s the price of progress. And if you don’t like it, you can always opt to stay right where you are. These data show all too well why so many do.

Topic: Oracle

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

11 comments
Log in or register to join the discussion
  • or...

    you could take an entirely different approach...
    dahowlett@...
  • Looking in the wrong place for the problem

    It is interesting how easy it is to produce these statistics from a relational database.

    It is interesting to note that no statistics are stated for what is actually going on the application (I take it the object referred to in the article are database objects - tables, views, procedures and so on).

    Generating the delta between tables, columns and views in versions is a trivial exercise in an RDBMS. Generating what order you should load data into a new version to avoid violating foreign key constraints is also trivial.

    Furthermore generating queries that test existing data for conformity to new constraints can also be generated from the database metadata.

    How many of the application issues are that straightforward? A large number of the constraints in Oracle applications are in the applications and not in the database. Evaluating the impact of these contraints is much harder than evaluating the impact of the changes in database constraints.
    jorwell
  • Undocumented Complexity is the Problem

    You are right, More than 70% of constraints in Oracle Applications are not in the database, and many that are documented in the FND tables or the technical reference manuals are wrong. The eprentise software systematically and properly identifies the ?undocumented? constraints and their impact. My table and Josh?s blog were relying on the commonly known data objects to demonstrate the ever-increasing complexity of these systems. Your comments reinforce Josh?s point rather than undercutting it. As Josh says, this complexity is what makes upgrading (and changing) hard and costly, and complicated. If those of us who are experts in identifying and validating the constraints (in the apps) admit that evaluating the impact of these constraints is difficult, then aren't we agreeing with Josh in saying that change is even more difficult for the business user, for the database administrator, and for the programmer who is trying to interface to other applications? How many customers going through an upgrade or change actually go through the ?trivial? exercises of generating the delta between tables, columns and views in different versions or generating the data load sequence to avoid violating foreign key constraints? We would hope that Oracle would do that in their development. They do not. The point of the graph is not in the numbers, but in the growing complexity of the apps and the difficulty of anyone (functional or technical) to keep up with the changes, and to make changes. This problem is exacerbated by the lack of an extensible database design and data modeling skills in those who develop these complex systems (Oracle, SAP, Peoplesoft or any other).
    It is not the physical upgrade that is difficult - there are scripts for that and most of them actually run pretty well. Instead, the difficulties are in accommodating the business changes that force the upgrade and understanding the impact, not only of the data constraints, but also the impact of changing a configuration parameter, and in testing the changes throughout the applications to verify that the changes support the business processes.
    helene@...
    • Agreed

      While generating the delta is a fairly straightforward exercise for someone skilled in SQL and familiar with the DBMS system tables I agree that the vendor ought to supply the code to do this.

      I also agree that the other big problem is understanding the business implications of the changes.

      There are certainly some questionable database designs in these big systems and these tend to exacerbate the complexity of upgrades (and of running the system generally).

      Also putting constraints in the application is a fundamental mistake, what is disturbing about these big applications is not how many constraints there are in the database but how few there are in relation to the number of tables. If the application has a verification after the fact facility then this is sign that something has gone very wrong with the database design.

      My point about the numbers is how easy it is to generate these numbers from the database and how hard it would be to generate anything comparable from the application (not to mention keeping track of all the dependencies).
      jorwell
  • difficult to upgrade?

    I can not/ would not/ try to speak for everyone. but me personally,;
    my issue with upgrades is, the fact that some vendors have an already bad rep, and upgrades are pointless. the public is not interested. and I am not naming names, of software. while others appear semi new to the public and immediate upgrades are scary if the original work is untested by the users. again no names here.
    but...BUT! those familiar venders we use a lot,..some have weak upgrades and some have mediocre upgrades.
    run an update and nothing looks different to the user.
    it would be better at times if vendors added a new bell or whistle so the user feels he/she are getting something newer and better.

    this could-be a hot-topic but I doubt anyone wants it to be.

    just my views on it.
    varick
    • Old bugaboos

      In the general overview of upgrading any software, being a layman computer enthusiast, I have seen many changes in architectures, complete systems and speed in chip design and through all that, there just seems to be a loss of elegance in software design. Can you say "bloatware"?
      I understand the immense complexity of organizing data to produce a coherent result but the end user pays the price for sloppiness in design, execution and lack of imagination. Why attempt to fill up the upgrades with spurious "features" to match burgeoning chip and memory capacities? Do we really need to dramatically alter the appearance of our favorite software app or tools to confirm that we really "got better"? There have been quite a few times when the so-called upgrade has gone south
      of our expected result. You don't think Microsoft knows about this, do you?
      Maybe I'm a DOS grandpa who longs for the old days when machine-code progs and apps saved a lot of time and processing space. Except for online play and CD-ROM games by others in my family, nearly all of the stuff I really like to use and play with takes 'way less processor time, is almost real-time in improvement at upgrade and is easier on the learning curve for new features. Most times, they even include easy-to-understand documentation (GASP!) so you can maybe figure out what the changes were supposed to correct or enhance. And they can do that without sacrificing computer time at whatever end you want to look at. Why are MS, Adobe, Symantec and quite a few others supposedly written to consume so many of our computer resources simply because they "have the room"? It seems we pay more for them like they're getting paid by the word instead of producing a more elegant result.
      ronw1@...
  • Error on Line One

    "It?s no secret that every vendor needs its customers to upgrade early and often." Actually, that's debatable, even though you've presented it as a truism.

    Vendors may like you to upgrade "often", but that doesn't mean there's any general need to do so. Software that is trivial enough to work properly the first time, and/or that is not edge-facing, need never be "upgraded" or "updated".

    The expectation is; upgrades are for sale, updates are free. Updates are expected to add little new functionality and are generally required either to fix software defects (the "soft" equivalent of product recall) or are inherent in the nature of what the software does (e.g. antivirus updates).

    The problem for vendors is that with the exception of antivirus feeware, current software licensing mechanisms do not provide an income stream for updates.

    The problems for users are that the need for updates forces a dependency on the vendor that can be elevated to "rental slavery", and that data continuity can be broken when software handling that data is re-versioned.

    Software as a "service" (rather than a manufactured product) would suit vendors very well, as they are assured an income to cover the costs of updates, irrespective of whether users are attracted to new-version upgrade sales.

    But this represents a re-writing of user-vendor relations, and users' interests are poorly represented in this process. Already, EUL"A"s are unilaterally designed by vendors to remove most or all user rights; moving from "product" to "service" is likely to be a shark-fest.
    cquirke
  • PC consumers are starting to get educated

    The PC users today have spent a lot of cash on programs and hardware that they have either barely used, found they had no need for or just decided they never needed it in the first place. The want ads and used PC hardware, software and such have countless web
    pages loaded with stuff for sale by buyers who thought they needed something because they were duped into believing it was what they needed by the vendors. The greatest vendor out there proving this point is no doubt, Microsoft or are there any brave souls wanting to stand in on their behalf on Windows ME ? This was a cash cow devised, marketed and did nothing more than exploit consumers, so you need to back off with this vendors needing consumers to upgrade often bunk. Truth is, nobody, repeat NOBODY needs Vista. If anyone can tell me why you NEED Vista, not want, not like but NEED, I'm all ears. Please don't get on this directx 10 angle as it was already used by MS for Vista and there isn't much out there needing it anyway. In so far as security, there is no security in any OS that is bullet proof, not now, not before and not likely ever will be.
    intrepi@...
    • Linux guy, right ?

      You are a Linux guy, right. This article is mostly about database applications, not about Vista. Speaking of Vista, it is needed because new Windows applications will require more and more features included in Vista, and not included in other Microsoft products. If one needs those features, and wants to run Windows, then she/he needs Vista. If not, one can run FreeBSD, Solaris, Mac or Linux, for example.
      trenchsol
  • aplications do get more complicated over time

    Those issues mentioned might come from growing customers requirements, as well. Customers business model is changing over time in application reflects that changes. On the other hand customers want more and more elements of their business process automated and implemented in application. I am not an Oracle expert, but I do web and GUI applications that work with Postgresql and MS SQL. Applications always start as more simple in the beginning and grow complex over time.

    Does vendor push unrequested features, or not, that is the other question.

    All in all, I generally disagree with the article.
    trenchsol
    • The question is who requests the features

      If wealthy customer X is prepared to pay for a whole set of features then vendor Y will happily include these.

      As vendor Y wishes to "reuse" these features then they will be included in the next version of the package.

      Whether these "features" will be appropriate for customers W and Z is another question altogether.

      So the feature set is driven by those customers most prepared to pay for them.

      This is leaving aside the whole question of whether everyone understands the implications of these changes.

      For a database to be used effectively everyone in the user community has to agree on the interpretation of what the tables and columns mean in respect to the real world. On this basis, one might ask the question whether it is in anyway realistic to expect that the same database design will work for a large number of users all working in different organisations (in which case the whole ERP approach becomes a highly questionable enterprise).
      jorwell