Crowds are dopes. So are individuals. At times.

by admin on May 27, 2008

in General

Sometimes I read something that’s so wide of the mark it requires a robust response. This post entitled Sometimes Crowds Aren’t That Wise is one such. Written as an apologia for the popularity system masquerading as a Wisdom of Crowds rating system aka Digg, the author seeks to define the way ‘crowds’ should behave in order for systems like Digg to work. That’s just not feasible and fails to make the most obvious point. Rating systems that rely on a popularity based methodology are fundamentally flawed and should at best be taken with a large grain of salt.

It seems that so many have become enamoured of ‘popularity’ that just about anything is capable of widespread belief. The example quoted in the story about something that turns out to be untrue is one such. The author uses sociological terms to describe the effects in play and to a large extent I think he has it right. But what we are also seeing is this form of behaviour being repeated with the inevitable effect that otherwise serious commenters are being duped.

There will always be those who write spurious stories looking for any method by which to grab a headline. You could argue that in doing so – and getting exposed – such stories do us all a favour. But what does it say about our ability to take the very tools that are supposed to be democratizing both the creation and dissemination of information and upon which people are starting to rely? Some will argue that the truth will win out and therefore these examples represent aberrations to which we should pay attention and be reminded of the need to ‘keep it clean.’ I’m not so sure it is that simple.

Right now I see way too many instances of instant forgiveness in the use of tools that are not delivering on the promise they first espoused and yet which seem to march on regardless. Digg is one example. Twitter, with its continuing failures is another. Take this example from Mike Krigsman writing about Twitter. Having detailed some of the reasons he believes Twitter is failing, Mike says:

My take. Twitter is a great service and I love it when it works. In addition, the Twitter folks are friendly and accessible, so it feels somewhat mean-spirited to apply the usual IT failures expectations to them.

Mike then provides no analysis of following statements made by the co-founder when it is already evident the company has little clue what the heck it is doing although he does conclude:

Despite the obvious good will, Twitter now has an $80 million valuation and provides a communications infrastructure upon which many people depend. From that perspective, users are completely justified in expecting a robust, reliable service with no explanations and no excuses.

Let’s be clear. Twitter is a free service and in this life you get what you pay for. Yet it seems that otherwise rational people are, in my opinion, coming to irrational or internally inconsistent conclusions.

Over arching these examples is the notion that the Wisdom of Crowds ultimately holds true. There is no wisdom in exposing fraud or lies. Where is the evidence that you can rely upon WOC validitated decision making inside any business? I don’t see it. Instead, Nitink is much closer to the mark when he talks about Search and the Dumbness of Crowds, detailing how Digg and other systems like del.icio.us fail in this regard.

I’m not so sure about del.icio.us, a place I regularly use to bookmark stories I find interesting. In doing so, I express an opinion about the story and it is then up to the reader to decide whether the comment makes sense. The point is I am not claiming any wisdom or insight, merely an opinon that may or may not be reasonable.

Next month I will be addressing around 130 marketers who are looking at these new tools and who are concerned to find the right way to engage in their use. One thing I will be saying loudly. Don’t believe blindly in the Wisdom of Crowds or those who are pimping this ‘stuff’ we loosely call social media unless you’ve got a social scientist in tow who can make sense of it all. My reasoning is straightforward: All the social experiments I have seen that put groups of people under pressure to conform nearly always demonstrate poor results, usually in the form of human behaviour that to the outside seems irrational. Each set of experiments has its roots in theories around cognitive dissonance and how authority is exercised. Two extremes are Bystander Apathy and the Standford prison experiment.

I can already hear the howls of derision at my making this connection but consider this. In a recent post weighing the relative merits of Twitter and FriendFeed as story spreading mechanisms, Dan Farber posited that:

Steve Gillmor makes the claim that Twitter is being strangled by FriendFeed and that his pal Robert Scoble is hijacking the conversation away from the unreliable Twitter site to FriendFeed. It’s much ado about nothing. Users have the freedom to head to their communications medium of choice. The Twitter conversation stream isn’t locked into a walled garden–tweets can flow like water into applications such as FriendFeed, Summize, and Facebook.

It’s not clear precisely where this latest twist on instant messaging and feed aggregation is heading, but just let it evolve without the prejudice in its own Darwinian way. That doesn’t mean to back away from criticism or debate, but to do so in the context of open networks that provide ways for individual users and groups to shape their online experience.

The implicit assumption is that we are all rational people who will stand back and objectively come to appropriate conclusions. But that fails to take into account the influence and popularity of individuals who are capable of shaping many opinions. In other words, they have power that cannot be ignored. Taking that further:

Robert Scoble is widely regarded as influential. Whether that is true or not outside the Silicon Valley bubblesphere is debatable. But in this post analyzed by Joe McKendrick, we learn that Scoble thinks:

I like the [Twitter etc] noise. Why? Because I can see patterns before anyone else. I saw the Chinese earthquake happening 45 minutes before Google News reported it. Why? Because I was watching the noise, not the news.

What if one day he is wrong? What if, because these tools are flawed, he is ‘seeing’ something that isn’t there? The theory runs that the self-correcting mechanism of the blogosphere will set it straight. Maybe it will but right now I’m seeing something that is little short of a form of collective madness in the acceptance of the unacceptable. Software that doesn’t work and software that’s deeply flawed. Combine those with doubts about the real Wisdom of Crowds and it is easy to see why I am both cautious and suspicious about accepting these tools as appropriate for business thinkers.

In the past, we have put up with flawed or incomplete software. Some of my colleagues will go so far as to say that we still do and that business decision makers acquire software in an irrational manner, often following herd instincts. That may also be true, but is far less so in the curmudgeonly world I occupy.

It is for that reason I prefer the ideas contained within the alogrithms of services like GroupThink, where implicit hierarchies can emerge that reflect the needs of business and which do not rely on the explicit application of popularity based systems to exert an appropriate level of influence upon decision makers.

UPDATE: Steve Gillmor thinks I am labeling. Certainly not the intention though I can see how that conclusion could be reached. Neither does my argument take account of what Steve discusses when referring to ‘the attention economy.’ And just for total clarity, I’m not immune from getting it spectacularly wrong as I know my friends will remind me.

Comments on this entry are closed.

Brad King May 27, 2008 at 7:00 am

Hey Dennis:

The flaw in the major premise is that "popularity" is the Wisdom of Crowds. In fact, there are very specific rules that James Surowiecki and others laid out. Popularity isn't one of them.

Crowds require some incentive to participate outside of simply voting — even when it's simply individuals (as Gladwell pointed out in Blink).

My guess is that the functional reality of how these systems should best be used — after all, they aren't a catch all magic bullet — goes lost of people who only understand the vague notion of what Wisdom of Crowds means.

Which, of course, is the heart of much innovation.

Dennis Howlett May 27, 2008 at 7:14 am

Hi Brad – I know what James was prosletizing in WOC and the 'rules' under which he saw the working of the concept. However, I don't see evidence of it working in the real world in the way described and I am not saying that popularity = WOC. I am doubtful for other reasons, not least of which seems the irrational way in which conclusions are being reached.

Brad King May 27, 2008 at 7:33 am

Hey Dennis:

I can say when we ran the Innovation Futures at Technology Review, our readers did a pretty decent job at figuring out when tipping points would be reached.

But I don't think I did a good job overseeing that particular project for a variety of reasons: the main one being we didn't have the money to hire a proper community manager.

The Iowa predictive market, in fact, has done well by politics: http://www.biz.uiowa.edu/iem/

Hopefully we can agree that none of the examples from your post are actually WOC type systems. They are simply rated or searched systems.

That is the fundamental issue. Nobody is truly using that type of system.

I got a note today about Hubdub, a news predictive market, of which I have spent little time on — but I plan to poke around.

Dennis Howlett May 27, 2008 at 7:45 am

What's not coming across as well as I'd like is the stuff I see swirling around and getting a lot of attention which to me seems irrational.

I'm a fan of prediction markets but right now I'm not sure they're as well understood as they might be. The relative simplicity of GroupSwim is something that I can see has genuine appeal.

Brad King May 27, 2008 at 7:48 am

I agree with you on the point that what is out there is mostly irrational, particularly if people believe Digg and such are predictive in any way.

I've not heard of GroupSwim, but I'll check it out.

It would be worthwhile to find some predictive markets that we think are worthwhile.

Dennis Howlett May 27, 2008 at 7:52 am

Spigit offers a decent system but it is very much in the 'big league'

Brad King May 27, 2008 at 7:53 am

Here's why I like Iowa, other than it's run by a university with researchers:

We gather national polls for the 1988 through 2004 U.S. Presidential elections and ask whether either the poll or a contemporaneous Iowa Electronic Markets vote-share market prediction is closer to the eventual outcome for the two-major-party vote split. We compare market predictions to 964 polls over the five Presidential elections since 1988. The market is closer to the eventual outcome 74% of the time. Further, the market significantly outperforms the polls in every election when forecasting more than 100 days in advance.

Jon Husband May 28, 2008 at 6:03 am

I like, appreciate and agree with the points you have made. There are imo relatively few people exploring the less-shiny-bits of what is called social media (as you have here), or the probability that some of its leading lights aren't necessarily all that smart … just loud, prolific and lucky to have been in the mix when things got going.

It's just a pity that you couldn't convey this analysis and conclusion in 140 characters or less ;-)

(Disclosure: I am not someone who thinks every blog posts needs to be three paragraphs or less to read the whole thing … I actually read this one twice).

Jasmin Tragas May 28, 2008 at 6:17 am

I enjoyed this post. Chewing on it.
I also wonder if I get as much value out of Twitter now that I follow more people? It's a lucky dip some days.

Previous post:

Next post: