Why Clear Books research fails

The mini-storm around Clear Books research is an object lesson in failure. In the end it doesn’t matter whether Clear Books had involvement in the construction of the survey or whether its hands are clean. What matters is the perception it creates among those who are interested in these kinds of thing. It doesn’t even matter whether I was right or wrong on each and every point I raised. Here is how they have failed:

1. Vendor sponsored research is always tainted. The piper calls the tune and all that. In this case there is confusion whether Clear Books is a ‘client’ as stated in the conventional sense or not. The research says so and that automatically puts people on alert. Nothing that Clear Books has said in response makes the relationship clear in terms all would accept.
2. Setting one set of objectives along with another that was not transparently explained. They told vendors they were sizing the market etc and said other stuff would be happening but that it would not be relevant and would not be part of the final outcome. They were not clear about what that other research might involve. In the minds of those who participated from the sell side, they have unanimously told me they felt they were duped. It doesn’t therefore matter what the original intentions were, the outcomes were different and based upon flawed communications. Since it is clear that Clear Books had control over the outcomes – i.e. what was published, then Clear Books and/or JBS should have ensured participants were aware of the change in direction. Failing to do that makes people naturally suspicious.
3. Let’s assume Clear Books had no control over the construction of the report. They still had control over the outcomes. They must have known that the survey which produced a spike was bound to have an aberrant effect on outcomes. Since they saw the results prior to publication (how else did they decide what was in and what was out?) they must have known what was to be included. Also, since they claim some SEO expertise they must also have known that the survey impacted the research outcome in a way that was outside of normal tolerances. The people who run Clear Books are Chartered Accountants, they have at least a rudimentary understanding of stats and standard deviation impact. It didn’t take me more than a few seconds to go: ‘hang on.’ Clear books must equally have known that someone would look at this aspect.  Or they are naive.
4. ClearBooks clearly had influence over the published results. In the minds of those who see this – that’s all that matters. So to declare they had no input to the research requires some detailed explaining. That is absent.
5. Liberty Accounts perceived the research as a facade for a marketing exercise. When it was pitched to me via David Terrar I understood it to be an independent market sizing survey to be undertaken by an academic institution. The fact two different people see this in diametrically opposed ways should be enough to alert respondents that all was not as it seemed. In other words, there was insufficient transparency about objectives and outcomes that were in turn poorly communicated. When vendors are involved with research and want that same research to be seen as credible, they’d better be certain what’s going on. Anything else discredits outcomes.
6. This is acknowledged as an academic exercise undertaken by students. As someone who has done something similar I know the value of supervision. Given what I readily found and the fact it was part of a term paper, I am not surprised it was flawed. The fact it is tied to a Cambridge institute calls into question that institute’s ability to oversee projects and protect its students from publishing fundamental errors. Given what we see, that was not part of Clear Books agenda. If they had any sense of propriety, they would know this is an important factor in establishing gravitas.
7. The fact Clear Books is continuing to defend and cherry pick arguments without consideration of the bigger picture is shameful. It is typical of software vendors and they always get caught out. There are reasons why certain software ‘analysts’ are treated as paid shills. It is unfortunate that JBS now finds itself in that class – even though that might be a wholly unfair assessment. In their defense, I still think the students were duped.
8. Instead of coming clean and providing proofs or offering clarity, ClearBooks tries to wriggle out of its problems with lame reasons that only serve to create further doubt and raise more questions. This is typical behaviour. Instead of sitting back and thinking: ‘where did we go wrong?’ they respond with half baked and self immolating answers.  Where those don’t work, they nit pick about irrelevant detail.

There is an easy ‘Get Out of Jail Free’ card here. I am not going to spell it out. Clear Books doesn’t deserve that information.

I feel sorry for the students. The more I think about it the more I think that their inexperience has put them in an impossible situation. Experienced researchers would have pushed back against such a fundamental change so late in the day and which was bound to have such a dramatic impact on the outcome. That’s why I say that Clear Books had control which they deny.

Regardless, there are lessons for all involved in this sordid story. The real loser is Clear Books. Regardless of how good or otherwise its solution might be, it has been exposed as a manipulator. No-one likes that. They have not learned any lessons from what the online world can inflict upon those who do not shape up to the standards of transparency and honesty that are expected. They have a lot of bridge building to do. I suspect this will be a tough time for them.