Servicing the longtail of data warehousing

Enterprise applications are large, difficult to set up, and needs lots of custom programming to integrate with the rest of the company's tools.   What's more they're very expensive.  When the State of Utah switched to SAPs Payroll package, we had Cedar onsite for nearly a year doing the set-up and integration.  That's great for big organizations who might see some ROI from the project, but what about the little organizations? 

Mike Farmer, an innovative engineer I've worked with on a few projects, poses this question on his blog in a post called Data Warehousing for the Rest of Us.   Mike doesn't have many answers, yet, but he's asking the right questions.  Mike says:

I am sick of reading about "the ideal" data warehousing solution. I have been reading data warehousing and data management articles for about two years now and I firmly believe that these authors live in a dream world that only exists where there is an infinite amount of money and resources to go around.

I do give them credit, however, for promoting technology and practices that can, at least, get you off in the right direction; but in the same breath I blame them for the unbelievably high failure rates of data warehousing projects.

Data warehousing is one of those terms that makes the eyes of business people glaze over. They want answers to questions, not a warehouse.  This is a common problem in IT--we fail to use terminology that others can get excited about and then we wonder why they don't care.

Date warehousing is also something that's difficult to buy by the drink, as a service, because of the shear volume of data that has to be loaded into the warehouse to keep it useful.  Another problem is that small businesses (I'm talking about companies with less than $50M in gross revenue) don't usually have the IT staff to make even a simple data warehouse truly effective, let alone a CIO to help the business side of the organization see it's value. Most small businesses rely on spreadsheets as their business analysis tool.

Even so, Mike's right: there's a longtail of companies that need near-time answers to business questions, but don't have the bucks to buy commercial products or the sophistication to keep them going.   The company that cracks this one will make a lot of money.