Making the right assumptions

As I was writing about professionalism last week, there were some interesting thoughts circulating about our industry's love of numbers and some questions about the accuracy of such numbers.

Coming from a background where accuracy was something to be proud of and having a sister who continues to teach me about how to think correctly about numbers and their interpretation, this is a fascinating subject.

While we acknowledge that sourcing, like all management disciplines, is a combination of science and art, when we consider the importance of the above-listed topics, we need to be rigorous in our handling of numbers and their interpretation given the importance of topics such as pricing and financial business cases, benchmarking and market share statistics.

For pricing and business cases, we tend to validate assumptions, cross check multiple scenarios and present data in multiple ways to ensure no misunderstandings--all good signs of rigorous attention. When a service provider or an internal group repeatedly passes such rigorous attention, we gain confidence and find it easier to trust them. Unfortunately, we have also all seen groups that include 'strange' assumptions to justify a desired outcome--something that erodes our trust very quickly.

When it comes to benchmarking, we tend to check that the data is current, is representative of whatever it is we are trying to benchmark and confirm the margin of error in the data (don't let anyone tell you there is a 0 percent margin of error). When a benchmarking firm repeatedly and consistently addresses such issues, we gain confidence in their professionalism and service providers and clients alike have an increased level of trust. Unfortunately, there are also situations where firms 'make up' data to satisfy customer demands and when this is discovered, it damages the reputation of the entire industry (not too dissimilar to some of the quality issues coming out of China at the moment).

Market share statistics are perhaps the most difficult area so we have to pay particular attention to the funding of such studies, the exact data collection, scoring and ranking methods, the cross checking against other data sources and the statistical validity of any conclusions that are drawn. Firms that are not willing to disclose such methodological details are naturally more susceptible to doubt.

In fact, I am reminded of a great comment I heard recently. Think about the last time you read a newspaper article that was about a subject you knew very well and when you finished reading the article you thought--wow, that was not very accurate. Then you read the next article in the same publication, on a subject that you do not know very well and what do you often think--wow, isn't that incredible, I wonder why that is.

Good food for thought the next time someone gives you some market rankings.