Peer Review and Appeals to Authority

Suppose you write a blog post arguing a point. Some readers will reply (or, worse, think it without saying it): “If that was true, you would get it published in a peer reviewed journal. If you don’t do that, it’s because you’re wrong.” The assumption here is that peer reviewed journals do very accurate gatekeeping about which ideas are good or bad. The assumption here is that almost all the good ideas get published and few of the bad ones.

A rational criticism points out a (decisive) flaw in an idea. That means a reason it fails at its purpose. If you’re trying to answer a question about philosophy, art, cooking, math, sports, science or whatever, peer reviewed journal publication isn’t a necessary part of success. The answer could be correct even if it’s not peer reviewed. So lack of peer review isn’t a valid criticism. It’s not pointing out a reason the idea fails.

Many people look at ideas in terms of authority. They use appeals to authority instead of arguments, and they dismiss ideas for lacking authority. That’s an invalid, irrational way of thinking. Also, a lot of intellectual authority is actually about social status, which isn’t very intellectual. The status of an idea passing a peer review is a type of authority.

The rational approach is to focus on debate and criticism of ideas. To rationally reject an idea, you must either point out a flaw or find someone else who pointed out a flaw in a way you find satisfactory (then give a link or cite to their criticism). Also, you shouldn’t have a goal of rejecting an idea then find a way to do so; instead, you should investigate ideas in a neutral way, without assuming you’ll reach a specific conclusion.

But in the world today, tons of ideas are rejected, by tons of people, without anyone writing down a refutation. People reject ideas without claiming that anyone – themselves or anyone else – has written down a refutation. Or they claim that refutations have been written but they won’t name, cite or link any specific refutation.

And tons of ideas are accepted despite refutations. People say the criticism was probably refuted somewhere but won’t be specific. Or they say if the criticism was true, the idea wouldn’t have made it through the peer review process to be published (that’s an appeal to authority). Or they say if the criticism was true, it’d be published in a peer reviewed journal (most of them focus on publishing new ideas, not criticisms of ideas they already published). Or they say they’re sure the experts already considered that or know a rebuttal, even though they don’t know how to answer it. Or they say the idea has a lot in favor of it, so even if that criticism is pretty good, the idea still seems pretty strong despite being weakened by that criticism. (Weighted arguments being an irrational approach is one of Critical Fallibilism’s most important philosophical points.)