In the comments of the earlier post Compared to What? there was some discussion of the futility of attempting to draw conclusions about world events, such as the war in Iraq, from press reports. In a new post Timothy Burke, a key participant in that earlier discussion, has more to say. He develops an analogy using the firing of a sports team manager to demonstrate the Knowledge Problem, and then attempts to apply it to more serious issues.

I’m going on at length about this because it seems to me this is how a lot of what we know comes into being. There is really very little we know from direct or eyewitness experience. Nor is it clear that being a direct participant yields information or knowledge that absolutely trumps all other kinds of knowledge. We know very well from recent research, for example, that witnesses to crimes frequently get some very basic details of their experience wrong. Eyewitnessing is important, and there are things you can’t know if you’re not directly there. We have to make a lot of judgments every day, some of them of critical importance, based on indirect, reported information and intuition...

Iraq is one of those judgments. I keep being struck in many conversations online and off not by the selectivity that different reasonable individuals exhibit in the information they gather about Iraq—we’re all selective, we have to be—but by the global statements about the nature of information about Iraq that they subsequently make, and how they use these global statements to categorically disregard other arguments or representations of the situation. I’ve seen a number of defenders of the war attack its critics for relying on press reports, or attack the press reports themselves for exhibiting false selectivity, sometimes both. But where are the defenders of the war getting their information, then?

It's true, we don't know and can't know what's going on with certainty. The farther we get from events the worse it gets as each node filters and augments the sparse information available. We know that press reports are unreliable both due to information fog and the venality of the press corps, something that is exposed with increasing regularity as the increasing depth of the corruption is revealed. Someone who attempts to make any sort of intellectual argument based on information provided by the press is in trouble. That's one of the reasons why I dismissed the Juan Cole post Timothy pointed to and it was the heart of this Kerry criticism:

"With all due respect to the president, has he turned on the evening news lately? Does he read the newspapers?" Kerry said. "Does he really know what's happening? Is he talking about the same war that the rest of us are talking about?"

I think this is what worries thoughtful people about Kerry and what makes political spin like Cole's seem quaint. They don't seem to grasp that reality isn't what you hear on the evening news or read in newspapers and it can't be used to formulate policy. Cole, a historian, well used to considering documents such as news stories as primary historical sources since there is often nothing else available, can perhaps be excused for having some difficulty with this concept but it seems a grave failing in someone who wants to be commander in chief.

This is an important thing to grasp. Broadcast media is entertainment and propaganda, fiction and politics. It can't be fixed, somehow reformed to be honest, because it is a structural problem, a basic defect of the method. Yes, journalists are venal and deceitful, publishers are grasping and amoral, but even if they were all saints the Knowledge Problem would persist.

Timothy continues:

Defenders of the war have to find channels of information that are entirely free of what they claim is a global taint with both the information and with the use of that information, or they have to deal with the total plurality of the infosphere that surrounds the war—and not just shrug it off as if that information is self-evidently untrustworthy by the very fact of it being reportage.

Well, no, they don't. Those who are criticizing the efforts in Afghanistan and Iraq base their attacks on spurious press reports of failure, quagmire, a story that is actually wished for by the partisan press. They have been shown to be wrong repeatedly over the past 3 years as military objectives they claimed were impossible were achieved almost before the ink dried on their predictions of defeat, making it quite clear that the press doesn't have useful information, can't understand the sparse information it obtains, and has a predetermined story line like the script of a drama that they are selling. War supporters, and even war critics, are simply being sensible when they dismiss criticism based on press reports.

But it's complicated. Knowing that the press is deceitful and that their reports are fiction doesn't mean that we know the truth, that it is simply the opposite of what the press says. Statements by the government are also suspect since they have political considerations and military needs that skew their public statements. There are things they don't want to talk about and things they can't talk about without endangering people in the field.

The intellectually honest view is that we don't really know. While no one can make a defensible intellectual argument based on press reports it is possible to make dinner party conversation and have political discussions since they have little or nothing to do with truthful representation, they have to do with rhetoric and reputation and are about social and political dominance. In that sense it is legitimate for war supporters to use the corruption of the press as a reason to dismiss them and argue the opposite. Indeed, they must as a counterbalance to the huge impact such false reports have on society.

Journalists ignore or deny all of this, and even those who admit that there is a problem maintain that it is somehow fixable.

I'm waiting for journalists to apply the same level of scrutiny to themselves that they apply to other industries. At what point do we declare a systemic failure and begin looking for the weak links in a profession that is critical to the survival of democracy? When do we stop treating the revelations as unrelated episodes, instead of symptoms of a larger cancer?

The recent Project for Excellence in Journalism report is a good first step. The State of the News Media 2004 takes a long hard look at the state of American journalism. The authors point out eight major trends and detail the forces affecting the news, including changing technology, pressure on profit margins, and convergence. It's available online for anyone to see, and everyone should see this report. But even this massive effort does little to explain why so many journalists have been exposed as liars and thieves. And it doesn't carry the weight of a news story.

When journalists think something is important, we put it on the air, or in the paper. Except for the stories written by the NY Times and USA Today exposing their own failings, the reporting on our own scandals has been episodic, not investigative.

It's taken a lot less for us to sound the alarms when it comes to other cultures or industries. By the time Arthur Andersen collapsed in the wake of the Enron scandal, journalists were doing everything they could to reveal corporate culture to the masses. Reporters profiled the personality types that become CEOs, they interviewed people who had left corporate culture in disgust or disillusionment, they visited the local MBA courses asking about the ethics curriculum.

When The Boston Globe exposed the culpability of clergy abuse at the highest levels of the Catholic Church, journalists everywhere took a magnifying glass to the faith, asking about seminaries, admittance procedures, and oversight of priests.

We are still looking at Jayson and Jack and all the other incidents as if they have nothing to do with one another. Reporters aren't poring over the J-school curriculum, asking if it could be taught differently. No one is writing page one Sunday stories about the type of personality that goes into journalism and the accountability measures that should keep journalists honest.

None of this can happen. On the industry side all owners and practitioners from top to bottom are implicated as well as the journalism schools that produced them. All of the personnel have "the personality types" that assure that these things will happen. I think it's important that we face the problem squarely, admit that we have a dysfunctional national media that is at best entertainment and at worst a propaganda machine. People want to trust institutions such as the press, and are often misled by the current corrupt media. Though the vast majority admit that they don't trust journalists and hold them in low esteem, the alternatives are not yet mature. They should at least be fully informed about the true nature of the media so that politicians can't so easily profit from the propaganda.

There is also a larger issue and superior tools to grapple with it. When Timothy says "it seems to me this is how a lot of what we know comes into being", referring to press reports and hearsay, he puts his finger on the importance of recognizing the Knowledge Problem. Once we face the fact that we can't know many things we are foolish to build institutions or engage in behaviors that require such knowledge. They are bound to fail. This is a hard limit for the useful scale of governments. A key reason that socialism failed was the inability to know what was needed though the power to act was centralized. The Knowledge Problem was a key part of the criticism of socialism made at the time by thoughtful economists such as Hayek but based on even earlier work. This isn't news.

We don't know what we think we know. We know myths, stories we tell one another to make it through the day and night. The thoughtful among us know that too and discount all such knowledge as far as possible. It can't be ignored since it IS much of what we have, but we can maintain our sense of humor about the paradox and hedge our bets by spreading risk and avoiding overcommitment.

Knowing that we don't know, and taking that fact seriously, is leading to better tools and methods. Failures, such as information cascade and the SNAFU principle discussed in other posts, in addition to the problems of precision, completeness and latency noted here, can't be cured but they can be worked around some. Simply admitting that we don't and can't know much is useful since it can dissuade us from formulating policies and institutions dependent on timely and accurate information. We are coming to understand that hierarchies and collectives are faulty organizational methods and that decisions are best made by those closest to the task, those responsible for implementation. Their work can be improved by having broader and faster information flows though they also have Knowledge Problems and will sometimes fail. Keeping decisions local, with limited scope and scale, avoids large impacts, allows others to learn from these small mistakes, and so improves future efforts.

Update:

Oddly perhaps, this statement resonates.

Reader and frequent Insta-critic Jonathan Miller accuses me of painting a sunny picture of Iraq... He says I don't link a lot of stories involving bad news. True enough -- I figure since that stuff is plastered all over the TV networks and newspaper front pages at the least provocation, I don't add a lot of value by repeating it. The good news, and stuff that bears on the strategic background, on the other hand, somehow seems to get a lot less attention. When bad news matters and is undercovered -- which sometimes happens, if it requires actual understanding to appreciate -- I do try to mention it, as with the CERP program, discussed here and here, among many other posts, or with regard to Zeyad's war crimes reports -- go here for a roundup and follow the links for earlier posts.

But, as I've said before, InstaPundit isn't a news service. I'm not sure any blog is, with the possible exception of The Command Post. And even there, I think they see their mission as supplementary to the larger media world, and don't regard themselves as a free-standing source of information. My sense regarding Iraq is that things are -- despite the problems endlessly documented in the Big Media -- moving along, and that it's no more a hopeless quagmire than Afghanistan has turned out to be. I could be wrong, of course -- I often am, about all sorts of things -- but I think that anyone who wants to assess what's going on in Iraq needs to do more than just "look at what's on television," as John Kerry suggests. I try to help provide a fuller picture than the TV folks, whose chief goal, it sometimes seems, is to help Kerry get elected.

There's a bit of internal contradiction here. InstaPundit is a news service as much as any of the MSM. The fact that his news is selective does not distinguish his from their news which is equally selective if not more so though using different filters. As net news continues to grow and mature, providing both more timely and more diverse views, it will be more difficult for MSM to conduct such flagrantly political campaigns since they are denied cover. Their markets will continue to shrink and so pressure them to provide better products just to survive though this will only delay their inevitable demise.

» THINK'N from chiasm.blog-city.com
Robin Burke leads and sets off some interesting discussions on postmodernism and politics, prompted by Derrida's death (on Winds of Change of all places!), in posts here, here, and here.  All the material and the comments are worth reading if you're...[read more]
Tracked: October 13, 2004 12:49 PM
» Just Ducky from Crumb Trail
A new M&M rant, Quack Science, whinges about the way misinformation from environmental advocacy distorts society and leads to poor decision making. A main factor contributing to doubt about dire predictions of ecological doom is that the claims are so......[read more]
Tracked: October 21, 2004 02:58 PM