The paper Groups of diverse problem solvers can outperform groups of high-ability problem solvers, referenced in the previous post Independent Cacophony - found by a circuitous route based on a glancing reference in James Surowiecki's talk given at the O'Reilly Emerging Technologies conference in San Diego called Independent Individuals and Wise Crowds, or Is It Possible to Be Too Connected? - notes some interesting conditions and exceptions.

An ideal group would contain high-ability problem solvers who are diverse. But, as we see in the proof of the result, as the pool of problem solvers grows larger, the very best problem solvers must become similar. In the limit, the highest-ability problem solvers cannot be diverse. The result also relies on the size of the random group becoming large. If not, the individual members of the random group may still have substantial overlap in their local optima and not perform well. At the same time, the group size cannot be so large as to prevent the group of the best problem solvers from becoming similar. This effect can also be seen by comparing Table 1. As the group size becomes larger, the group of the best problem solvers becomes more diverse and, not surprisingly, the group performs relatively better.

The insight only applies to groups that are good but not too good, large but not too large.

A further implication of our result is that, in a problem-solving context, a person’s value depends on her ability to improve the collective decision (8). A person’s expected contribution is contextual, depending on the perspectives and heuristics of others who work on the problem. The diversity of an agent’s problem-solving approach, as embedded in her perspective heuristic pair, relative to the other problem solvers is an important predictor of her value and may be more relevant than her ability to solve the problem on her own. Thus, even if we were to accept the claim that IQ tests, Scholastic Aptitude Test scores, and college grades predict individual problem-solving ability, they may not be as important in determining a person’s potential contribution as a problem solver as would be measures of how differently that person thinks.

I find this to be perfectly sensible and obvious; your worth to a group depends as much or more on the composition of the group as it does your absolute ability. If you have nothing new to offer then even high ability doesn't enhance the group.

The current model ignores several important features, including communication and learning. Our perspective heuristic framework could be used to provide microfoundations for communication costs. Problem solvers with nearly identical perspectives but diverse heuristics should communicate with one another easily. But problem solvers with diverse perspectives may have trouble understanding solutions identified by other agents.

Being different in the sense of having diverse heuristics is only of value to the group if you can communicate with the rest of the members. This isn't an issue of language so much as heuristic frameworks.

. . . our model also does not allow problem solvers to learn. Learning could be modeled as the acquisition of new perspectives and heuristics. Clearly, in a learning model, problem solvers would have incentives to acquire diverse perspectives and heuristics.

The benefit of introducing new heuristics is a one off boost in performance. Once the members of the group learn the new heuristics they do not continue to improve unless there are continuing injections of new views.

Though this work was referenced by Surowiecki and supports his contentions, it does so for different reasons than those Surowiecki cited in his original work which were based on an aggregation of independent, diverse and decentralized decisions. What Scott Page and Lu Hong show is that a diverse group working together excels. Each relatively weak member advances the problem solution from where another member got stuck. In this case the group is dependent and centralized though still diverse, and excels on different kinds of problems than those cited by Surowiecki. Page and Lu Hong speak mostly of problem solving within firms, offering insight into useful team composition and perhaps hiring practices, but the ideas seem to apply well to the emerging opportunities of ICT for assembling ad-hoc, task oriented groups.

The practices of the protagonist of Ken MacLeod's Engines of Light series come to mind. He described himself as an artist who would assemble an ad-hoc group of people and software tools (AIs) to tackle a problem, and then ride herd on them. Though he lacked the ability and interest to solve problems himself he could orchestrate a solution in which each of the diverse members contributed special skills. It wasn't simple specialization and division of labor, though that happened too, since it also included joint efforts by members with differing heuristics who could advance the solution of a single, undivided, perhaps indivisible problem in a manner much as Page and Lu Hong suggest - one picking up where another gets stuck by offering a new approach based on a different heuristic for the stage of the problem reached thus far by the heuristics of others. A group with weak but diverse heuristics could accomplish things beyond the capabilities of even groups with strong but similar heuristics.

The art was in choosing the team members. This isn't a skill that Page and Lu Hong offer much guidance for. How do we measure the heuristics of team candidates to select for diversity while still maintaining the necessary ability to communicate? Different but not too different. It would be comforting to see others develop this work further to develop a science of team composition rather than leaving it at the art stage.

I suspect that those who have spent some time in an R&D skunkworks may recognize these insights and perhaps protest that these things have been done for decades, that there is nothing truly new in either the work of Surowiecki or Page and Lu Hong. Itinerant research project leaders skilled in the art of team selection and management have long understood these things. Putting together a team of relatively weak (ie. cheap) members who together accomplish things that amaze themselves as well as others has long been an art practiced in R&D. That's true but still there is a benefit to turning it into a science that can explain why these things work and so perhaps increase the use of the techniques in less research oriented environments.

» SDMM from Crumb Trail
A number of posts here and there have dealt with group decision making. Every social group, from families to nations, makes group decisions. Though they are not always explicit, and seldom universally adhered to by group members, they are......[read more]
Tracked: April 19, 2005 10:51 AM