Induction and Critical Rationalism

What problem is the theory of induction intended to solve? What’s it for? What’s the goal or purpose?

Induction tries to solve one of the core problems in epistemology: how is knowledge created (and improved)? Another core problem is what knowledge is.

Induction is not intended to be a full solution to the problem of knowledge creation. A traditional solution is that we create knowledge using a mix of deduction and induction. Many people, including Karl Popper, accept deduction; the induction part of the answer has been debated more.

Another proposal is that knowledge is created through deduction alone. This can be called "rationalism" because it tries to create knowledge through pure reason without using observations. Many people including Popper believe that deduction alone is incomplete and inadequate. A common view is that you can get some knowledge using only deduction (e.g. about math), but most knowledge can’t be created that way. Adding induction to our toolkit, along with deduction, is an attempt to get a complete method that’s powerful enough to create all types of knowledge including science.

Another proposal is that knowledge is created using a mix of deduction, induction and abduction. Arguments have been made that deduction and induction together are still incomplete and another tool (abduction) is needed.

Another proposal is that the problem is insoluble: actually, knowledge can't be created. This is called "skepticism". A hard challenge for skeptics is to explain the success science has had (or at least appears to have had).

Popper proposed a different answer: knowledge is created by evolution. Popper says people also use deduction but not induction nor abduction. Biological evolution of genes creates knowledge without using deduction, so while deduction is very useful it’s not strictly required for knowledge creation.

One of Popper’s motivations was that criticisms of induction had already been known for thousands of years. Popper particularly liked David Hume’s criticisms of induction 200 years earlier. Popper developed additional criticisms including about the underlying concept of justification. Since people had been trying to figure out a way to make induction work for thousands of years, and had never succeeded, Popper thought it’d be productive to look for alternatives.

So defending induction against known logical criticisms was a very hard problem that people had failed at century after century. Meanwhile, there was another very hard problem, in another field, that people also failed at for many centuries: the problem of the origin of the species. Some people knew for a long time that “God did it” wasn’t a good answer to explain wolves and bees. But it was very hard to come up with a good answer.

Charles Darwin solved the origin of species problem (and others refined his solution). Then Popper recognized that the origin of species problem was actually relevant to epistemology. One hard problem was actually closely related to another hard problem. In each case, people had a bad answer (induction or God) and struggled with the problem for thousands of years.

As an aside, Darwin didn't personally figure everything out from scratch by himself. Similarly, Popper built on ideas from others. I'm sharing a simplified narrative.

Popper recognized that the creation of the species involves knowledge creation. The species contain knowledge. The hard part of the origin of species problem is actually where the knowledge in the species comes from. And the theory of evolution answered that. So Popper tried applying evolution as the solution to his problem of how knowledge can be created. He found it works for all types of knowledge, not just genetic knowledge. It turned out that both problems (origin of species and creation or origin of knowledge) were very hard because they were basically the same problem.

After modern refinements, the theory of evolution says that knowledge is created by replication with variation and selection, and genes are a replicator. I don’t remember being taught this in high school. I think most people are unfamiliar with it.

Popper, like Richard Dawkins, says that genes aren’t the only replicator. Ideas can replicate too. Dawkins coined the term “meme” for an idea that is a replicator.

Not all of this is super clear in Popper’s writing and he wrote a lot of other stuff too. I think it's a reasonable interpretation of what Popper was getting at, and I attribute that to Popper's skill and knowledge, not to luck. But it's understandable for people to read Popper and not figure all this out or disagree with some of it. David Deutsch refined some of Popper’s ideas and clarified the parts about evolution. I’ve made some additional refinements about how I think about it and explain it, in addition to making other improvements to Popper like about decisive arguments (which was a big enough change that my philosophy is no longer Critical Rationalism and needed a new name).

Most criticism of Popper’s work has focused on inessential issues or misunderstandings of his position. By inessential issues, I mean that I could concede that the criticism is correct and still be a big fan of Popper and still not change my mind about the Popperian ideas used in Critical Fallibilism.

An example of a widespread misunderstanding relates to the topic Popper titled a book after: The Logic of Scientific Discovery. People write rebuttals arguing about the sociology, psychology and common practice of scientific discovery. They seem to have misunderstood what topic Popper’s book discussed. It’s silly to say that Popper’s emphasis on critical discussion is wrong because many scientists don’t listen to criticism. Popper talked about what methods can and can’t work, as a matter of logic, for making scientific progress. Sociology and psychology aren’t relevant. Popper wasn’t trying to observe and document the behaviors of scientists to find patterns that way; that’s a different topic than logical analysis.

Empirical counter-examples to Popper’s logical claims would be relevant, but those would require difficult, detailed analysis showing exactly what was discovered by what method. It’d be naive and inadequate to just ask a scientist what method he used and assume that whatever he says is what he did. People often make mistakes about introspection even when they’re trying to be honest; understanding everything that goes on in your mind, in detail, is extremely hard. And there’s no way to directly observe what method someone used inside their mind; you can only observe their methods when it comes to stuff outside their mind like what chemicals and glassware they used and how they cleaned the glass. So while there can be productive discussion about potential empirical counter-examples to Popper’s logical claims, that’s hard, and a lot of criticism isn’t focused on meeting that challenge, but instead strays off topic away from logical issues.

A basic Popperian point of view is that Popper's evolutionary epistemology is not refuted: there is no known logical argument showing it cannot work. Meanwhile, induction has long had logical arguments showing it cannot possibly work at all. So if we care about logic then we should accept evolutionary epistemology instead of inductivist epistemology.

People generally try to criticize the degree of effectiveness or desirability of Popper's evolutionary epistemology, but that's only a secondary issue. If it works at all, even poorly, then that's better than alternatives that don't work. In a related way, people often try to defend induction by arguing in favor of its desirability, talking about how much they think we need it, saying how hard the situation would be without it, and otherwise making arguments that are irrelevant to whether it actually works or not. People often treat it like they already know induction works and they accept that evolutionary epistemology works too, so their goal is to figure out which one is better and more useful. But that's missing the point. The issue of whether each method works or doesn't work has logical priority.

A lot of people really want induction to work, so they keep trying to fix and defend it, and they're biased against alternatives. Their goal is to avoid skepticism and rationalism (to avoid having either no knowledge or only abstract knowledge). I, like Popper, agree with that goal. However, for many centuries, most people saw induction as the only way to avoid skepticism and rationalism, so they came to equate defending induction with avoiding skepticism or rationalism. However, to avoid skepticism and rationalism doesn't logically require induction; it requires any method of knowledge creation that uses observations. Popper's evolutionary epistemology creates knowledge and uses observation, so people should be eager to consider it given all the difficulties they've had with induction.