I was at the Salesforce Analyst Summit a few weeks ago and John Taschek, who is the SVP in charge of Market Strategy there, and one of the best human beings in the entire industry, said something that was really funny and, pretty much, true. He said that internally, he often says "the job of analysts is to tell the vendors that their baby is ugly." Now even though I can be a New Yorker - and thus blunt, I tend to take an approach more like "hey, don't you think his cheekbones are a little uneven, maybe a little bit of work here and here and they will be even" which I hope is more useful than "your baby is uuuuuh-GLY." But then...
In a way, the CRM Watchlist is both that and its opposite. I try find those companies who fixed those uneven cheekbones, and transformed their flawed skin into something luminescent. Lest you think I'm being shallow talking about physical beauty here, one thing:
Just a brief side comment, he says, smiling at his beautiful audience.
In any case, for the purposes of this post, what that means is that the Watchlist is designed to find companies who have not only had a significant impact in the market the year before (hence the 2017 Watchlist looks at 2016) but also are built to continue having that impact in the following approximately three years. That bears a lot of explanation - especially this year because some of the companies that have won in the past are going to be disappointed, others who have never won, delighted. So this is going to be a lengthy leadup to the winners of the CRM Watchlist 2017 and the rules for 2018. The one bonus is that, even though I extended the final submission date to December 31, 2016, which was two weeks longer than last year, I'm delivering the winners today - almost a month earlier than last year! I'm just so dedicated to you guys. I read a lot of pages (see stats below) just so I can do this faster. I must really love you. (Pats himself on the back).
So, of course, before we get to the winners, in part, to establish context for this year's selections and drop offs, and changes, and to make the suspense unbearable, here's how its gonna go.
First, I'm going to tell you about the CRM Watchlist 2017 - the kind of contest it is and how I thought about it. Then I'm going to reiterate the changes I wrote about a couple of weeks ago.
Then, I'm going to give you some stats and market trends that come from the 2017 submissions.
Then I'm going to tell you about the changes for 2018 and announce the opening date for registration and if you're smart you'll read this VERY carefully before you even register. I ain't playing on that.
Finally, the winners in each of these levels: Winner - Elite; Winner - With Distinction and Winner. (winner chicken dinner?)
Okay let's get to it.
I need to clarify this every year and this year is no different. The CRM Watchlist is an impact award. It's an award that identifies the strength, mindshare, and market share a company has in the markets it addresses - or tries to address. I'm not doing my analyst thing - and evaluating whether or not, for example, they have the right mission and vision or whether their message is exactly on point. What I am evaluating is their company as a whole and how well that company executed during the year by seeing actual results and how well prepared it is to continue to execute. For example, I might completely disagree with the specific messaging but I'm primarily looking at what your marketing strategy and programs are, how aligned with the market the message is, not whether it's wrong in my eyes or right; and how well you execute against that strategy. Look, I don't want to say that if I think the message is just so far off that I see utter failure in the future, it doesn't impact the score, but the actual nature/structure of the program and strategy are far more important - in this case. While, as an analyst I may publicly disagree with you on something, the Watchlist is focused on one idea - who you are and what kind of company you have. In order for me to see the impact you had in the year previous, and the impact you will continue to have in a three year period following - you have to prove to me you have a well-rounded qualified company - not just great tech.
Of course, impact has a lot of possible definitions. Since this is my award, let's look at mine. :-)
If your company is top of mind without prompting for a customer when they are making a vendor selection, that means you are having an impact. If your competitors set their compass by you, that's impact. If you are seen by pundits, press, and practitioners as a leading solution or provider in your field, that's impact. If you are the subject of brand buzz in external channels that's impact. If prospects routinely think of you as on the short list for vendor selection, that's impact. If you are the go-to subject matter experts in your domain, that's impact. And that's only a part of what makes impact up. As you know the Watchlist questionnaire has 12 areas and multiple questions within each area (see below for details). The answers (plus several other "they-will-remain-secret" things I am looking for not related to the questions, commonly called by those with no literary skills, a secret sauce) provides me with an indication of how much impact you've had and what kind of impact I think you are going to have. The present and the future have to be, uh, present, for you to even be considered as a possible winner of the Watchlist.
Impact doesn't have to be global to win. It can be geographical, for example EMEA, Latin America, or strictly North America. It can be the domination of a category of software or services if the category is more than a small niche. For example, dominating the sales force automation area, or customer analytics would qualify. It can be domination of a vertical industry - e.g. health services, non-profit, public sector. But one way or the other, the impact has to be obvious, both in the prior year and in the anticipated next two or three years. that there is no doubt at all that your company is making a major impression on a market and actually changing or strengthening that market by its presence - and your submission has to be good enough to prove it to me.
To have an impact, the company has to be pretty much a complete company who has been doing this long enough to have established a rhythm. The company has to be well rounded - it has financial stability, solid management, excellent products and services, superb culture, and a strong partner ecosystem to help sustain its efforts. It has to have a clear vision and mission and also clear-cut strategies for outreach to get external forces - customers, analysts, journalists, prospects, influencers, etc. - engaged. That takes a complete (and complex) set of tools and activities which could include marketing, analyst relations and public relations programs, the subject matter expertise via the content produced and distributed for consumption, and the "theatrical" activities that establish the corporate identity necessary to stay top of mind, as well as capture share of wallet.
That is not an easy task. What does it take? That's the purpose of the CRM Watchlist and has been since it was created ten years ago as the Steppin' Out Awards for one of the editions of CRM at the Speed of Light.
That's impact. That's what the CRM Watchlist pays attention to. So check that box. Now let's start looking at this year's Watchlist results with some interesting stats.
These stats are being made available just so that you can see the scope of the award and understand how hard it is to actually win. Last year, I said it was harder than ever. This year it was magnitudes harder as you will see by one stat that I'm going to point out.
Interesting, huh? #10 tells you how tough it was this year.
That's the quantitative stuff. Check. Now what about the qualitative stuff? The trends that were manifest in the submissions all in all. One disclaimer here. There is NO confidential information being disclosed even though the Watchlist submissions have a lot of confidential information presented. I just wanna be clear.
I have to qualify these trends - i.e. place them in context, so that you don't think that they are more than they are. These are trends that I've noticed among the submitting companies - not trends "in the market" per se. What they MAY reflect is what the high tech market is thinking and doing, given that all of the participants in the Watchlist are from that market and they are a decent representative sample. Also, for clarity's sake, even though this is called the "CRM Watchlist" a lot of the companies submitting are really customer facing tech companies so they are analytics providers, sales acceleration/optimization/enablement, customer engagement solutions, knowledge management, and customer experience (their name for it, not mine) systems providers, etc. In other words, all things customer facing. They also might represent a part of the more traditional CRM spectrum - marketing automation, customer service, or sales force automation. So they aren't just CRM pure plays. Most aren't in fact.
All that said, what did I see? (He rubs his hands gleefully):
Artificial Intelligence rules the airwaves - I suppose this is the most obvious. There was a very significant number of companies who claimed that they were incorporating artificial intelligence capabilities either available now or in the road map for the near future. This was as ubiquitous a trend as I saw - a recognition of AI, not as new hype, but an important part of the future of technology - and one of the most requested "asks" by practitioners -especially at the enterprise level. Even though somewhat different than AI, a corollary that made an appearance in more than one place was companies offering robotic process automation in one form or another.
Account Based Marketing? Say What? - Account Based Marketing (ABM) showed up in a surprisingly large number of submissions - either as a product offering or as something that was a new part of the marketing strategy of multiple companies. One of the companies who built an ABM product had never built an operational solution before - has an entirely different product set but saw such a proliferation of interest, they built an ABM product that played to their strengths but a real stretch when it came to the existing offering. The ABM mentions were sizable. This is actually notable for me. Important.
Lack of Outcomes-based Marketing/Sales - This was a negative trend - more an indicator of a broad problem, really. BTW, this is me with my analyst hat on. To explain: customers aren't looking for technology - they are looking to technology that will enable their efforts to achieve outcomes that lead to a result they want to get as self-interested individuals. So, when they research evaluate, buy, whatever, they are looking to solve both their company's needs and their own. In other words the value of the technology is in the outcomes that it helps the customer achieve. Marketing and sales efforts that are focused around use cases, case studies etc. tend to reap the greatest harvests because they are appealing to the strongest possible part of a person - her/his self-interest and how to achieve it. Yet, the majority of the companies, even some of those who had a good marketing strategy all in all, were not thinking this way - and in fact, were still trumpeting the more traditional product-driven approach, regardless of their commitment to "customer-centricity."
Platform wars are becoming more intense - While there is no doubt that Salesforce is still the dominant platform, there is a noticeably large number of people either integrating with Microsoft Dynamics or, building on the Microsoft Dynamics platform. This is something currently underway or in near term road maps. The bulk of those are companies that have built something (either an integration or product) on Salesforce already, but a few are starting out with Dynamics as a platform. Very interesting, indeed.
Big Four dominance continues but in a broader sense - A few months ago, I wrote a post about the escalation of the Big 4's efforts to be world dominant. The submissions this year were far more expansive than in years' past when it came to how Salesforce, Oracle, SAP and Microsoft were viewing their company. I won't say more than that - since I'm keeping what I have to as confidential and the other things that aren't will be saved for their reviews. Suffice to say, they are all in battle gear and going all out in all aspects of their conduct and efforts as companies - and each in its own way. But their submissions confirm my "Seeking Optimus Prime" hypothesis. Big time.
Uncertainty rules, I think? Does it? - There were a remarkable number of companies who had done rethinking, reimagining, rebranding, had massive turnover in management, changed their messaging significantly, were expanding to new markets, etc. Meaning changing dramatically. It was a visibly hefty number of the participants. Sometimes it was for good reasons, recognition of opportunity, other times for bad reasons, a needed change in direction to jumpstart the company, other times I couldn't ascertain why but something significant changed. In the broadest sense, I think the reasons were due to the uncertainties triggered throughout last year by the economy, by everything leading up to the elections, by the instabilities globally, and in some cases, on a more positive note, by identified opportunity to expand to new markets. Sometimes it was a matter of trying to pinpoint the identity of the company. These kind of changes, for good or bad reasons, have profound scoring impact in the Watchlist as you will see below. And to be candid, as far as the score goes, its not good - even if the change is. But for now, just note that it permeated a significant number of the companies.
Integrations everywhere but.... - While it is noticeable (kind of goes along with the platforms above) that there are significant integration efforts and partnerships accordingly, the bulk of them gear toward technical integration of marketing and sales applications and/or value add sales tools, or sales and customer service integration. But what lacks more than I expected at this point is integration or even development of customer engagement related tools with the CRM related systems out there. There is lip service to it, there are also vendors who are exclusively devoted to customer engagement but the sales, marketing, customer service, vendors and CRM suite vendors have largely given lip service to that effort or when they have done them, have relegated it to the back of the line. To be entire candid, and an analyst here - that is beyond stupid. More on that in a future post, though. This is a "you have a wonderful baby" post.
So stats and trends. Check.
Now the 2017 Watchlist, observations on winning and losing this year, the actual process and its changes. Then 2018 rules and then the announcements
First, I want to thank all the companies who submitted and those who while they registered, weren't able to submit and informed me in a timely way that they couldn't. I am honored that you care enough to make the time and effort to submit what in many cases were incredibly complete looks at your companies and in one case, a genuine work of production art. (Check out the winners lists for a special note on that). That is the biggest reason, I don't charge anything for this and take the time to read every single page of every submission and score each submission completely. The time and effort is fee enough.
That said, I did read roughly 6100 pages and did nothing else really (one blog post and a trip to the Salesforce Analyst Summit) from January 1 to now - for many hours a day. The scoring process I use is a weighted score per category of question (e.g. the outreach question has about 7 other questions within it but there is one score, though you are required to answer all questions). In addition there are some 30 plus criteria ranging from things I'm looking for you to say to things that you do and don't do (including not following the Watchlist rules to the letter) that get fractional points added or subtracted. They can be the difference between winning and losing too. There are specific thresholds I work from that define whether you win, win with distinction, are an elite winner or, this year, just plain lose. One thing I changed this year (again) is, once and for all, to drop honorable mentions who typically score highly but not enough to meet my threshold. This year that no longer gets anything. Also, and an even bigger deal, because of the sophistication of the market place in the tech world now, and the brilliant levels of engineering that are everywhere to be seen (the highest quality of products and provision of services I've ever run across in all my years doing this), I did something that I've only done once before (other than initially). I significantly adjusted the thresholds to make it far more difficult to win and that in turn made it harder to win with distinction and to be an Elite. So to give you an example, of how much more difficult - roughly 20% (give or take) of the winners last year would have lost this year - and that impacted a few companies this year who had won in the past.
One of the biggest impacts on scoring was the changes that I referenced above. Here's the thing. Significant changes of varying kinds, whether for good or bad reasons, create uncertainty as to their success. There is no way to see what the results of those changes are going to be. They may be the right changes. I'm not disputing that at all. They may have been done for the right reasons. No dispute there either. But this is an impact award - and to win - two conditions must be met - one I have to be able to see the impact you made in the year of the award (again, 2017 Watchlist = 2016 impact) and to be able to see the conditions that show me how you will be able to continue impacting the market for three years out. Big changes take away my ability to ascertain the latter. Hell, I know the company who made the changes is hoping for the best and thinks they've done the right thing or they wouldn't have done it - and who's to say it wasn't right - but who's to say it was -I've seen things like that fail often enough - and that's exactly my point. I can't say. Without that clarity, the future impact is almost impossible to figure out. To be entirely candid, you get penalized for it, even if it seems to be the right move - because of the uncertainty it creates. It doesn't mean you can't win. One or two did regardless. But it makes it all the more difficult. Thus, a few previous winners in the throes of change - in combination with the threshold change - didn't win this year. A few years ago, a company I know that had won, took the advice I had given them in their review seriously and expanded their market to a new one - and eventually became the category leader. But that takes time. Their submission the year they made that shift transformation, they lost - because they couldn't prove impact in the market they had just entered. There was nothing to say the market itself had legs - or that they had impact in it. The year after, they could prove their case, did a strong submission that gave me what I needed to make that judgment (along with the corollary research supporting their case) and they were back to being winners. But effectively, they lost that first year of change because they took my advice. Uncertainty can obscure possibility and I'm totally ruthless about nailing companies for it. I am really sorry. There is no quarter in this award. But there is consistency.
Another thing that hurts companies badly is not answering the questions I asked. I don't think that I can be any clearer. If you don't file a complete submission that answers everything I ask within the submission, then your score is going to be hurt significantly for not paying attention to the rules. For example, In question #4 on mission and vision, I state explicitly
"Give me at least one SPECIFIC example of how the mission and vision are reflected in a real life activity or event of yours. Tell me a story. (NOTE: This is an area that was skipped by a lot of respondents last year. Don't skip it)"
More than 40% skipped the story!! And every one of them got dinged. To be even clearer, here is part of Rule #4:
"To repeat, if you don't answer EVERY SINGLE question as I asked it in this questionnaire, you're going to lose points. If you don't answer all the questions within the questionnaire (not via links to the answers) then you will be penalized heavily. So please be specific and address the exact question."
I don't think I can be any more obvious or clear. I should be able to have all the information that I need to come to a conclusion within the submission. But there are a lot of companies that don't seem to believe me when I say that. Every single year I have multiple companies who for whatever reasons submit questionnaires that either woefully incomplete or just simple ignore some of the questions at their convenience. Sigh. Don't do it. I really do mean what I say. This year, one company should have won because they have an impact in their industry. But their submission chose to ignore questions I asked and half answered others. So, with the information I had and the subtractions in the score for the unanswered questions they lost.Additionally, and ironically, a company that had less impact (but enough) in the same market won. Tough luck.
The process itself is straightforward. I put out the call for registration. I leave registration open until about a month before the submission is due. Then there is a submission date that has no individual extensions allowed. There is a single submission allowed, no updates, so it's wise to submit as close to the final date and time as the submitting company feels comfortable with.
As soon as a company registers, I start to track them. The reason for that is there is a lot of information I need to gather (it varies by company and how much I know them) prior to the submission that will help me corroborate their statements and supplement their submission when the time comes. Over the year, I spend hours with each registrant's company, gathering info. But to be honest, that's not hard, because I need to track some of them anyway (they are my clients, or part of my coverage as an analyst) and I like getting to know new companies this way. I started using a tool a few years ago called RivalIQ, which tracks a comprehensive digital footprint of any companies that have them. They provide detailed reports that give me some insight into the use of digital communications tools. For example, Oracle's social presence continues to be greater than any other company I tracked though the gap has been closing a bit from year to year.
Once I get the submission, I go over every single word. The submission is 80-85 percent of the information I need to score any company. The other 15-20 percent is the external work I do. That tells you the submission had better be convincing if you want to win. You'd be surprised at the number of people who give half answers to a question and they are dinged for it. I said it earlier so I won't say any more again.
The scoring is done with weighted criteria. Each year, depending on the market, the scoring weights change. For example, in years of financial uncertainty (think 2008), the financial stability of a company is weighted higher than normal, though, of course, it is always important. Some years, the weights are remarkably flat. This year, not only was there a change in threshold but the weights shifted in 3 different areas. It always depends.
Once the scoring is done, the result is "automated", meaning I have a spreadsheet that sees the score and color codes the number. The resulting color code shows me whether the company was an elite, a winner with distinction, winner, or didn't place, based on the year's rules and weights and thresholds. Winner with distinction is a category for companies that just missed (and I mean just missed) elite.
Following the scores, there is the "and the winners of the 201X CRM Watchlist are..." announcement. That same post has the criteria used for this and next year in it, so I would carefully read it (this post in other words) if you have ANY interest in the CRM Watchlist any year. Over the next few months, I will write the reviews of winners at all levels (see this year's calendar below though) - For my own and your convenience I lump them into categories that suggest themselves - like AI or Marketing Tech or whatever. But to reiterate something that I seem to have to reiterate every year, there are no categorical winners. The categories are conveniences that allow me to group winners who do similar things. That is all. A couple of years ago, I had a company who claimed that they were the "winners in the customer engagement category." They were a winner of the Watchlist, but they obviously couldn't have won that category because there isn't any winner by category!! Get it? NONE. Anyone who states they came in first or that they win a category are making a false claim. I alphabetize and I group. There is no #1 except the highest scorer overall who I announce.
There were a few. Here are they briefly.
Okay, 2017 Watchlist stuff. Check.
There are two ways things are scored: The primary scoring is an individual score (to the 1/10 of a point) for each of the 12 categories in the questionnaire. The secondary scoring is done based on roughly 30 specific things that I'm looking for that aren't related strictly to the answers to the questions. Let's start with the 12 categories (not the questions, per se, the areas of interest). The other criteria are for me only so I won't disclose anything here.
Here are the 12 categories with descriptions:
These are merely paraphrased summaries of what each category means, and not examples of the many questions you must answer for each. Each of these is weighted differentially each year based on real-life market forces. There are also multiple components that drive either positive or negative intangible scores that can have a serious impact on the final result, which aren't necessarily a direct answer to a specific question. Nor will you ever know more than one or two of them. AND they can change from year to year.
The questionnaire is sent to you once you register (see above for process) and the rules for 2018 and dates for submission and withdrawal and whatever else are in that questionnaire.
Please bear something in mind when it comes to all of this: Just because you didn't win doesn't mean you are a bad company or had little impact. As I mentioned, the uncertainty of how a transition or change is going to work out, can have a serious negative effect on your score Remember this is an impact award, and if I can't see both the impact you had in the prior year or easily foresee the likely impact you will have in the next three years, your chances of winning are diminished. Transitions make that line of sight foggier. There are an enormous number of factors at play in winning the CRM Watchlist.
Another thing, just to be entirely candid and perfectly clear, this is geared to established companies, so the larger ones have a better chance (though no guarantee) of winning the award. They have the resources that a lot of smaller companies don't have to execute on the things they need to do to have an effect on a market. They have a longer-standing history typically, and they have a lot more money. They have a much greater reach and market recognition. They are mature and can invest not only the money, but also the manpower in the specialized activities that can be required to impact a market. In smaller companies, the resources just often aren't there, but that by no means negates smaller companies from having an impact. There are several smaller companies that win each year- and it happened again this year.
Because I know how hard companies work on their submissions (for the most part), if you are a company that submitted but didn't win or dropped a notch or whatever, you have the right to ask me for time to explain why you didn't do as well as you thought you would. We'll figure out a 30-minute time slot that is amenable to both of us and we'll have that discussion. However, there are big caveats. First, this isn't an analyst consult. It is strictly to discuss your submission and why it didn't make it. Though it has elements of a consult. Second, you'll only get the answers I can give you without exposing some of the criteria I keep to myself. And to be entirely candid, while the time will be amenable to both parties, it will be more at my discretion than yours. Sorry for the rules, but I'm one person and I want to accommodate as many as I can who request it. So there have to be limits. Each year about 10 companies take me up on it - give or take a few. I wish that more did.
The other thing is that as always (except last year) I will be writing reviews. But its going to be a long drawn out process. I have to complete my book and am committed to doing it by June 30 so that is my priority when it comes to writing. Consequently, I will write the review for the Lifetime Achievement, the Watchlist Elite and the Watchlist Winners with Distinction pre-June 30. All other reviews will be done by December 31, 2017 but after June 30.
See below for a post-2017, pre and during 2018 calendar.
On to 2018.
As of today, the 2018 CRM Watchlist is now open for registration. In order to register, please send me a request for a CRM Watchlist registration form. The email to send me the request is firstname.lastname@example.org. No other email or channel. I will send you the form, you then send it back to me at the same address after you fill it out completely. Then when the questionnaire for 2018 is ready, I'll send you the Watchlist questionnaire with instructions including the 2018 due date.
Please note the basic criteria for even asking for a registration form.
If you are a small company with no revenue or just a beta product with no active customers, really, don't bother to submit. You won't win. This is an IMPACT award that means you need to have presence in a market. Several companies this year were missing multiple FUNDAMENTAL components that are needed to have an impact - and of course - had little or nothing to show.
If you register, please register with the idea that you are going to submit a questionnaire. Don't register to just get on my radar and then not submit. I am a human being. Thus, I am limited by time in what I do with my life. Consequently, I don't want to waste time. If you just want to get on my radar, there are other ways to do it than register for the Watchlist. So let me reiterate because every year, companies that register don't submit. If you don't expect to submit a questionnaire don't send me the registration form back. I don't want to spend nearly a year tracking you for the Watchlist and then find out that you didn't submit the questionnaire. It would not endear you in my eyes, to say the least.
Okay before the good stuff, my disclaimer - same one as always.
There will be NO exceptions to these dates at all. None. Zip. Nada. Zero. Nunca.
Now that all that procedural stuff is done....we get to the good stuff.
(By level and alphabetically)
Watchlist Elite (highest scoring company in the entire Watchlist is first, rest alphabetical)
1. Salesforce - HIGHEST SCORE
2. Ernst & Young Advisory - (Special note: Brilliantly produced, not just written submission)
Watchlist Winner with Distinction
15. Pitney Bowes
Likely to Win 2018 (companies that just missed)
2. Solvis Consulting,
3. The Pedowitz Group
Up and Coming (newcomer on good trajectory):
Congratulations to all the 2017 CRM Watchlist Award winners!!
On to 2018. Remember, registration is open. Email me at email@example.com. I'll be waiting.