I noted a few posts ago that I write this journal for my own benefit, a way to note things of interest and my thoughts about them, so that I can refer to them again at a later date when something new comes up that is related, or simply as a memory aid. Usually this means searching my archives to find an earlier reference, but I also pay attention to what other people search for here, and check out what they are reading.

This post from almost 5 years ago is getting hits lately.

There’s been a lot of fuzzy thinking about what we mean when we talk about collective intelligence, network, and interaction. I want to parse these distinctions.

In The Wisdom of Crowds, I wrote about the power of groups under certain circumstances to be remarkably intelligent. A model of collective intelligence: a large group of people reflecting diverse opinions offering judgments independently with some mechanism to aggregate the judgments, collectively ending up with an intelligent outcome. . .

The wisdom of crowds works well when there is a true answer, and as long as some choices are better than others. The key is that people are mostly working on their private information, which may not be good, may be fragmented, but it is diverse. Collective wisdom does not emerge out of consensus. The goal is not to get everyone to agree – it’s to tap into people who disagree, into the diverse information everybody has. It works best when people are not paying too much attention to what everyone else is doing. They have some sense – like feedback in the form of odds at the racetrack – but there isn’t a lot of personal interaction. . .

Human beings are not ants.

That's an excerpt of a transcription of a talk by James Surowiecki given at an O'Reilly Emerging Technologies conference. At the time he was concerned about his book The Wisdom Of Crowds: Why The Many Are Smarter Than The Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations being misunderstood and misused. It's full of thoughts that are apposite to our current situation 5 years later. More:

If there is too much interaction among human beings, groups end up being less intelligent than they would otherwise be. The more we talk to each other the dumber it is possible for us to become. The book has quite a bit about small groups. Put a bunch of smart people into a room and they emerge dumber than when they went in. . . The question for all of us is, how can you have interaction without information cascades, without losing the independence that’s such a key factor in group intelligence? I’m not going to come to a final answer. But there are a few things worth thinking about. First one: the best thing to do is to keep your ties loose. You’re better off, and the group is better off, if the ties are looser, because loose ties minimize the influence of those around you. I don’t think Duncan Watts’ model of the information cascade is quite true. I don’t think people are as subject to the influences around them as Duncan thinks. But we are clearly shaped by those influences. One way around that: limit the power of the influences.

That old post began with contrasting quotes, one from Keynes about reputation and one from Emerson about independence. If I was writing today I might contrast Hayek with Keynes as has been done so well lately. As Obama's paleo-politics and paleo-ideology fails ever more spectacularly Surowieki's words seem ever more apt: "Put a bunch of smart people into a room and they emerge dumber than when they went in." They don't know how to think or how to solve problems, no matter how smart they are, or think that they are. It's the same problem we see with the UN, especially its hysterical focus on climate, but that's just the current failure in the spotlight and should surprise no one since it has ever been so on every other issue.

Second, keep yourself exposed to as much information as possible. Injecting some level of randomness into the system is a good thing. Diversity is a good thing. In computer science experiments at the University of Michigan, a researcher, Scott Page, had his agents compete until they differentiated into three groups, Dumb, Intelligent, and Random. Then he had them solve problem as groups. The Intelligent group outperforms the Dumb group, but not by very much. But the Random group almost always outperforms the Intelligent group. Page’s theory is that the reason for this is that even if the less intelligent groups know less, what they know is different.

This has important implications for the way decision making works inside organizations. Make groups that range across hierarchies. The conclusion is that you actually can be too connected, if the connections are of the wrong kind and if they’re reinforcing your existing prejudices rather than altering them. You can pay to much attention to those around you, even if they’re really smart. The flip side of Pascal’s isolation is the cacophony you find on the net; it bombards you with many voices. Isolation and cacophony, interestingly, allow you to arrive at the same place: independence.

What struck me in re-reading this today was how well it demolishes Obama's recent call to Democrats to narrow their inputs, to stop listening to cable news broadcasts, reading blogs or in any other way clouding their minds with independent thoughts. Obama is not interested in problem solving, good solutions or even accurate situational analyses. He wants a dumbed down consensus such as you will find in universities and much of the mainstream media. His goal isn't to do good work, it's to sell shoddy work and somehow convince society to like it anyway.

This is why I like the Tea Party thing. It's a confused cacophony that drives the rigid minded pundits crazy. They can't easily label it, denigrate it, refute it or even understand it since it is such a diverse thing. The Tea People don't agree with each other or anyone else, they are thinking for themselves to the best of their abilities. They can perhaps be better understood as one of Page's Random groups that almost always outperforms the Intelligent group, but even if they are one of his Dumb groups they are all but a match for the Intelligent group and easily pull ahead when you also consider their intimate knowledge of their own values and preferences.

The trick is to find some mechanism to aggregate the independent judgments. It isn't consensus, and it isn't an average or a median. And it isn't that this will find The Answer, since there isn't a single true answer. What is clear is that it is essential that we dispense with the old fashioned ideas of consensus and narrow minded large scale visions. Though we can't know what is right, we can know that this is wrong.