Like Jon Reed and David Dobrin (who are both referenced), I was there. I filmed the whole session. I deliberately did not edit or put into the public domain the actual shoot out because in my opinion it was unfair. Instead, I restricted myself to showing the analysis provided by Vinnie Mirchandani, Ray Wang and Helmuth Guembel (reproduced above.) For the sake of transparency, SAP asked me to send over my recording and I declined. If others wish to do so or if any other filmed recordings come out then that is for others to decide.
In my opinion, it was unfair at several levels and despite Mike's assumption about the outcome it is difficult to see how anyone can realistically discuss the hyperbole issue without discussing the veracity of the shoot out. Here's why:
Netsuite was one of the event sponsors (not just the shoot out as Mike asserts.)
They were able to put Mini Pieres, an accomplished NetSuite demonstrator (I have more video including Mini to substantiate the point), on stage with the person demonstrating the NetSuite solution. That meant Mini was able to add color to the demonstration that would inevitably impact the view of those in the audience.
The person attempting to demonstrate ByDesign was trying to access a public test system that was being hammered by some 5,000 other people at the same time. That meant at least part of the ByD demonstration failed because the demonstrator could not 'hold' the required data at various steps. It was clear to me that the person demonstrating ByD did not know about optimizing performance on networks of the kind used in the shootout. This is something that was discussed in an open SAP forum in London last year but which is a configuration detail that is part of a ByD implementation. Its a vital detail.
Neither Vinnie nor Ray were given advance warning of the shoot out, nor was it part of their briefing materials for the sessions they conducted. To their credit, both of them used their many years' analyst experience to step back and look at both services objectively. In the first post that Dennis Byron published, Vinnie commented:
For the record, Helmuth did not tell Ray or I who was doing the demo till the day. It was not on his site or in our check in materials. If you watch the video that Dennis posted both Ray and I went to lengths to point out SAP was at a disadvantage given the network, not having their own demo team etc.
In the piece Mike references, i.e Dennis Byron's second piece, Dennis takes the view that SAP was ambushed and that NetSuite has overblown the result. I would not go so far as to say 'ambushed' although I am familiar with Helmuth's distaste for what has happened with ByD.
The software industry is awash with politics. The whole on-premise v SaaS argument is principally a philosophical discussion rooted in the politics of market share power. SAP has become a target for that as it wrestles (sometimes very badly) with what it wants to be and how it morphs for the 21st century. It is great sport for people like Mike and I. I now believe we should rise above that and be more analytic in both our thinking and assessment of what fits where.
Now to agendas. We've all got them. Anyone who says otherwise is kidding themselves. I hope that in my writings I remember to state what side I am taking if it isn't clear.
In another conversation at the SAP Influencer Summit the day before, I talked with David Dobrin about comparisons between ByD and NetSuite. He made what at first glance seems to be the perfectly reasonable assessment that ByD is much more intensive to implement than Netsuite. He went on to say that even though he might hate NetSuite at times, the fact he can get most of what he needs to get done outweighs problems he might experience. Is that unreasonable? For many businesses the answer would be: no. But given David's positioning, is it any surprise he believes NetSuite came out on top? I would too.
I on the other hand I take a very different perspective. Any assessment of applications or services has to be a lot more than a simple shoot out. At the most extreme end, I can make QuickBooks look good against SAP Business Suite. But then I would be comparing apples and oranges. That's why we have RFPs, analyst reports, user experience reports, partner assessments and a hundred and one other data points before customers make final choices. And even then those choices may well be irrational. Focusing on a shoot out of this kind may be appropriate at the VSB end of the spectrum where 30-day 'try before you buy' is the selection method du jour. It most certainly isn't at the mid-tier and up. Even NetSuite will tell you they'd like to see a 90-day or less sales cycle. I'm sure SAP would be delighted if they could get from RFP to check in the same period.
It concerns me that in the general anti-SAP group think I see swirling around, little thought is given to what ByD is really about. It is much more than financials plus CRM. Can it be readily compared to NetSuite? Sure - in some scenarios. Whether that makes it a credible SaaS offering will not be judged by people with agendas of whatever persuasion but by the market.
Mike's comments about NetSuite's approach to marketing might be well intentioned but they're inappropriate. None of us who comment have the right to make marketing strategy statements for vendors unless specifically asked to opine. Even then, we should recognize that each of us represents one voice of many.
It is up to NetSuite to decide how it goes to market. Even though I may not necessarily like all that they do, they are honest in bringing to the fore what those of us who are in the trenches know is often a dirty street fight in many a sales situation. In other conversations I've had with NetSuite marketers, I've made it clear I am more than happy to talk up their case material but I want a lot more before opining on the company in the public domain. They tell me they're working on that. The same goes for any vendor. Independent analysis that is meaningful to buyers depends on a multi-faceted view and so while shoot outs have their place, it's only a tiny fraction of what matters.
As to Mike's 'advice' for SAP - save it. They're just as capable of street fighting as the next vendor. Instead look at what they're doing. Leveraging a well respected brand to achieve 100 referenceable customers for ByD. That's NOT the same as the 100 touted by so many who discuss this topic as if that's all SAP has achieved. To suggest they get down and dirty in the public domain is to misunderstand the SAP mindset and to devalue the brand.
Disclosure: I am an SAP Mentor and Top 10 SAP Community Network Contributor for 2010. I have no current commercial relationship with SAP. Along with Brian Sommer, I authored a 24 page independent assessment of ByD in 2008. It is due for an overhaul.