The Scientific Method
Saturday, July 13th, 2019 | Uncategorized
A lightning talk I gave at SciPy 2019 in Austin, Texas,
A common view of the scientific method is that it consists of:
- formulating a hypothesis, and then
- conducting experiments to falsify or confirm this hypothesis.
This view has a few problems:
- It suggests a black and white picture where hypotheses are either right or wrong.
- It paints a picture of science as a large body of disconnected statements.
- It presents a narrow view of science where experiments are of primary importance, and
- I don’t think it matches how science actually happens.
These problems are far from academic. Science encounters a significant amount of skepticism & societal views are often polarized. I don’t think the common view of the scientific method is helping. When people are bombarded with many short disconnected statements and not given any tools to connect or reason about these themselves, it’s not surprising that many of them become confused.
I think a more useful view of the scientific method is as building and characterizing models
Let’s take vaccines as an example. I’m fairly sure that the measles vaccine doesn’t cause autism, and if someone came to me with evidence that it did, I’d be fairly skeptical. This isn’t because I’ve done a lot of experiments of my own though, or read a lot of papers by experimenters I trust. It’s because I have a simple model in my head of what the measles vaccine is — an attenuated form of the measles virus. Thus it seems unlikely that the vaccine would do something the virus didn’t. On the other hand The Cutter Incident in the 1950s where Cutter Laboratories distributed vaccines accidentally containing live polio virus seems very plausible in terms of the model.
If we’re going to win over science skeptics, we’re going to have to explain our models. This is a harder kind of educating than stating facts & hoping to be believed & hoping that we were right in the first place — but no one said this was going to be easy.
Science is about model characterization. We build mental models and characterize them. We expand their consequences. We understand when they’re valid and when they’re not and how accurate they are. We simplify and improve them.
Models come in many shapes and sizes. They might be as simple as a drawing, or as complex as the standard model of quantum mechanics.
Experiments still have a unique value — their measurements are independent of our models. But how we decide which experiments to do and how we interpret the results is very model dependent.
As an added bonus, this model-based view looks Bayesian and solves the “all ravens are black” paradox.
But more importantly, a common set of models allows us to have a conversation — even if we disagree about the details. They allow us to come to shared conclusions and have constructive disagreements — to have a shared mindset. Models that are well characterized can be depended upon in their domain of validity. They allow us to have the “what if …” conversations that are the
basis of policy.
For many important issues we can’t perform experiments. We only have one planet and one life to live.