I wonder how many readers have ploughed through a report from an industry or financial analyst? There was a time when I would grind my way through long, highly detailed reports as a way of better understanding a particular technology or to get a nuanced perspective. That’s changed. Today I, along with some of my peers create our own analysis based on real world experience, what we hear from buyers and of course what the vendors tell us. Even so, the analyst community seems to be merrily jogging along, though some have felt the cold wind of recession. Unfortunately, their model is mostly outdated and ceasing to be relevant, especially in the saas/on-demand/cloud (SOC) world where for example one company I spoke with recently claimed to have sent out 400 releases last year. In the on-premise world, you’re lucky if you get one release a year.
From the analyst side that has led to something Ben Kepes at CloudAve noticed in a particular report – bias in favour of the on-premise providers that gives the impression of inferiority where none exists. Phil Wainewright subsequently picked up on it, noting that:
SaaS vendors, of course, spare their customers the burden of installing the software at all, and therefore it’s a total absurdity to mark them down for not being available on a range of different operating systems. But that’s precisely what Forrester has done.
The story is worth bearing in mind next time someone tells you that a SaaS or cloud option is clearly inferior. Ask yourself what criteria they’re judging it on before relying on their verdict.
PS – A reader points out that the publishing timetable and two-year lifetime of analyst product comparison reports is itself a built-in bias against SaaS vendors. It’s designed to jibe with the on-premise product delivery cycle, where a new version is designed to last at least a couple of years and is made available for analyst review long before customers have deployed it. Whereas for a SaaS vendor’s product, the report is already a version or two behind by the time it comes out, and there will be as many as eight version updates during the lifetime of the report.
I agree. Some reports I have seen seem wildly out of date based on what I have seen elsewhere. The analysts will argue process and peer review but quite frankly that doesn’t cut it in today’s high speed world. I’d go further and argue that in the case Ben/Phil discuss, the question raised which led to markdown is redundant and shows a lack of appreciation about the SOC market as a whole. I have seen something similar elsewhere when a client complained that a particular analyst had to be led by the nose through the solution.There are exceptions. Saugatuck for instance had an analysis out on CODA’s tie up with Salesforce.com and now called FinancialForce.com within 24 hours of the news. They like I and others were pre-briefed and it is not the same thing to which Ben/Phil refer but nonetheless, that’s pretty speedy work by analyst standards.
In a soon to be announced new venture, I along with colleagues hope to address the problem of stagnation by being nimble, remaining relevant and using modern technology to ensure that clients have access to the latest thinking on a topic as quickly as we can make it available. In that sense, some of our material will almost be like news – fresh every day. Others less so. We can do that because the areas of our interest are grounded in territory where we’ve been active for many years and for which new material comes available all the time which we can rapidly synthesize. There is no real excuse for not knowing what’s happening other than the pressure of keeping up to speed and ensuring that you know which sources are the best for your particular needs. But I think it is more insidious than Ben/Phil believe.
In the material I am currently preparing around that venture, I say: “The analyst/consulting community is often riddled with conflicts mostly driven by the large sell side financial and technical analyst groups. That disadvantages the buy side and serves to maintain an improper relationship that is unsustainable in the long term.” As a committed buy side guy, that knowledge leaves a sour taste. But then I appreciate it is a matter of the economics with which they’re working. That doesn’t necessarily mean that all analysis is skewed, many of the analysts I know are honourable people with good intentions but it is worth remembering who the real paymasters are in this equation. And it ain’t you.
It might be worth remembering that next time you fork over hundreds or even thousands of dollars/pounds/euros from a brand leading analyst group.