Does Hindsight Devalue Science?

Cullen Murphy, editor of The Atlantic, said that the social sciences turn up "no ideas or conclusions that can't be found in [any] encyclopedia of quotations... Day after day social scientists go out into the world. Day after day they discover that people's behavior is pretty much what you'd expect." . .

Daphna Baratz exposed college students to pairs of supposed findings, one true ("In prosperous times people spend a larger portion of their income than during a recession") and one the truth's opposite. In both sides of the pair, students rated the supposed finding as what they "would have predicted". Perfectly standard hindsight bias.

Which leads people to think they have no need for science, because they "could have predicted" that.

(Just as you would expect, right?)

Hindsight will lead us to systematically undervalue the surprisingness of scientific findings, especially the discoveries we understand - the ones that seem real to us, the ones we can retrofit into our models of the world.

What was found was that students - and perhaps more other members of society than we could wish - readily accept supposed findings. Why is this? That was not found, it is left to interpretation and so will vary with the biases of the interpreter.

I don't interpret this as providing any information about hindsight. Instead, it provides information about the docility of modern authoritarians.

If IQ really correlates with the ability to flourish in an industrial society (and I'm quite prepared to believe that), then it is, as I said last time, a measurement of the ability to navigate paper-pushing bureaucracies — to learn to manipulate arbitrary abstract explicit rules, and to do so on command. Presuming that people who don't manage to pull off at least some minimum level of this make very unattractive mating partners, and so have below-average reproductive success, then those of us in developed countries have spent the last one or two centuries breeding for docility, in both senses of the word.

Murphy may have a point that social scientists "discover that people's behavior is pretty much what you'd expect", but these hindisght experiments don't seem to shed any light on that assertion, or support the claim that hindsight devalues science.

The science part of the experiment tells us what. It provides no information about why, and that is the part that most often seems to me to be unadulterated bunk. Worse, once someone emits this bunk their previously more nuanced views harden to support the bunk, and others cite the bunk as if it was factual.

Sometimes I claim that the problem could be reduced by better experimental design, or that the nebulous data provided is insufficient to draw conclusions and needs to be followed up with more experiments that sharpen focus. The data has value, even though slight in many cases, so it isn't that science is undervalued so much as that many experiments have small value.

I'd say that hindsight bias mainly affects forecasting. We have an insufficiently robust appreciation for risk and the unexpected. We place far too much faith in forecasts due to the seeming ease of doing hindcasts. When we are punked by reality, as happens so very often, we scream about incompetence. Usually this is some idea or person - a politician in many cases - that we opposed all along, and the reality glitch is seen as confirmation of our existing biases. This is not just mistaken attribution in most cases, it is also hindsight bias.