I seldom find much of worth in the views of any expert, yet I do find worth in a diverse group of experts. The key bit is diversity - heuristic diversity - since they aren't all simply clones of one another with the same blind spots and irrational tics. I seek out dissenting views almost in self defense since it is an article of faith that consensus always is wrong. The dissenters may well be wrong too, so you just have to give up on the idea of accepting authority, and when the situation requires action just take your best shot with the information you have.

Said another, better way:

"I grant that you've seen a lot of evidence that I haven't. But here's my question for you: If I saw and read everything that you've seen and read, what would I conclude?"

Of course, the other guy could respond, "You'd agree with me," but he rarely does. When you frame the question as I have, it's often pretty clear that even though the disputants are not on a level playing field, that isn't the real reason why they hold different views.

One nice feature of my heuristic: It actually makes disagreement more informative. Suppose you have a well-informed friend who heavily disagrees with you. Nine times out of ten, he admits that if you knew what he knows, you'd still disagree. But one time out of ten, he insists that if you knew what he knows, you'd change your mind. As long as you trust your friend, such a statement makes it reasonable to immediately adjust your belief.

It's not that you need to reject expert advice in all cases, but you must always make the decision yourself. Sometimes the expert advice will be compelling and so you can make it your own. Seldom isn't the same as never.

Update: Toothless

I'd like to say Bryan bites a bullet here, but alas he just gums it, as he doesn't engage the hard questions: what exactly is his better-origin scenario/story, and what evidence supports that story over less-flattering stories? That is, how could Caplan tell the difference between a situation where his prior was good and mine bad, vs. a situation where his prior was bad and mine good? If he grants that a reasonable person, long before our births, would have thought these two situations equally likely, what later evidence could have convinced this reasonable person that Bryan's prior turned out better?

He can't. But that's the problem with expertise. It is insufficient. In most subjects, for most issues, knowing everything that is known is not enough to make a certain judgement. There are no reliable authorities, it is always a gamble. The best you can do is to eliminate some of the bad bets. What Caplan's system does is to help eliminate some bad bets without having comprehensive knowledge. Asking good questions is often as useful as getting good answers. Cutting through clouds of inky obfuscation is a neat trick.