I wrote today’s episode of SciShow! This one is about Bayes’s Theorum, and it was BUTT-HARD TO WRITE. Hard as a friggin butt.
With all the other scripts I’ve turned in for SciShow, it went really smoothly, the content editor asked for a few tweaks and then they were good to go. But I had to re-write this one OVER and OVER and OVER again, because there was just NO WAY to fit everything I wanted (I NEEDED) to say into the space that I had. Bayesian analysis is INCREDIBLY IMPORTANT and REALLY QUITE COMPLICATED, and in the end, I totally failed to explain why in the space of 600 words. What you got instead was a Bayesian thought experiment, which is cool and everything, but NOT ENOUGH IT IS NEVER ENOUGH.
So since Tumblr has no character limit, I am going to say everything here that I wanted to say in the video.
Bayesian analysis is THE CODIFICATION OF THE SCIENTIFIC METHOD FOR MAKING PREDICTIONS. You want to tell the future? YOU CAN DO IT WITH SCIENCE. So sit the hell down because this is the realest shit you’ll read all day. Bayesian probability gives you the tools you need to make probabilistic predictions involving unknown variables that will inevitably trend towards the true outcome. NOTHING ELSE DOES THIS. NOTHING. N-O-T-H-I-N-G.
The other method of statistical analysis, that you’re probably much more familiar with, is called ‘frequentism,’ and all it does is take all the information you have and put it into a big pile and then try to draw correlations between data points that look like they might be significant. IT SUCKS. It’s why you have such a low opinion of pollsters. It’s why one grey-faced white dude in a blue suit on Fox will tell us that we’re in a recession economy, and another grey-faced white dude on MSNBC will tell us that we’re in the midst of a brisk recovery, even though they have access to all of the same friggin information. It’s how we end up with awful social effluvia like the anti-vaccination movement, who have clanged the information involving vaccinations and the information about autism together hard enough for long enough that they’ve knocked loose a few correlations.
Frequentism is useful for analyzing information that is PURELY FACTUAL and involves NO HUMAN JUDGMENT when it comes to picking and choosing which facts to examine. IT CANNOT MAKE PREDICTIONS. Because the future is inherently uncertain. Uncertainty means you need to make judgment calls. Judgment calls invite bias.
Frequentist analysis has no built-in mechanism to counteract bias, and so its predictions are overwhelmingly wildly wrong.
So why is it the only method of statistical analysis that you’ve ever seen in action (whether you knew it by its proper name or not)? Because wildly wrong predictions play better for a TV audience. Because CEOs and politicians want a statistician who is certain. No one wants to hear that there is a 70% chance that we are in a bear market. They want to hear YES or NO. And the news networks want to hear both YES and NO, hopefully from two separate people who will shout at each other and pound the table and drive up ratings on a slow Wednesday night.
So what about the Bayesian method? Bayesian predictions are overwhelmingly inevitably right. They treat uncertainty as an inherent quality of the unknown, and express themselves probabilistically. I think of the difference between the Bayesian and frequentist methods as the difference between quantum and classical mechanics: quantum mechanics are more uncertain but more accurate, while classical mechanics are more certain but less accurate.
The frequentist method essentially treats information the way Laplace’s demon treats the motion of particles: frequentists imagine that if you just had access to all of the information, you could make perfect predictions 100% of the time. Bayesian analysts, by contrast, understand that even if you polled every single person in America a week before the election, you still would not know with perfect certainty what the election outcome would be.
So how does it work?
Bayesian analysis invites you to subject the meat and ephemera of your prediction to the scientific method. (In fact, the scientific method itself is a special case of Bayesian reasoning.) You must:
1. Identify a source of uncertainty
2. Make a hypothesis about the missing information
3. subject your hypothesis to testing
4. analyze the data
5. and revise your prediction.
In the video, I use the example of false positives in a mammography, which shows how Bayesian thinking goes, but doesn’t show its potential: because all of the variables are known. Much MORE significant is a problem like the following:
Imagine you come home to where you live with your significant other. For the sake of this example, we will say that you are a woman, and your partner is your husband. You go into the bedroom, and under the bed, you find a pair of women’s underwear that is not your own.
1. The source of uncertainty: Is your husband cheating on you?
2. Hypothesis: yes.
3. And now Bayesian reasoning gets really slick. I go over this in the video, but I’ll do it again in brief here: you now need to provide a number of variables.
X: The odds that your husband was cheating on you prior to finding the underwear. Essentially, the inherent odds of infidelity taking place. This is called a Bayesian prior, and it’s at the heart of all Bayesian predictions. In this example, we can assign X with a reasonable amount of confidence: studies have shown that 4% of spouses cheat on their partners in any given year. So we will put X at 4%, or 0.04.
Y: The odds that an observed event will take place (in this case, you finding a strange pair of women’s underwear) if your hypothesis is true. In other words, what are the odds that you would find this underwear if your husband were having an affair? You might be tempted to put this pretty close to 100%, now that you’re looking at the underwear, but try to be objective. On the one hand, if your husband is having an affair, there’s no mystery as to where the underwear came from. But on the other hand…you’d kind of hope he wouldn’t be this stupid? Let’s put it at 30%. Just to make it perfectly clear: we are guessing that the odds your husband would leave another woman’s underwear where you could find it if he were having an affair are 30%.
Z: The odds that the same event would take place if your hypothesis is false. What are the odds that the underwear is there for totally innocent reasons? There are a few possible explanations. Maybe he bought them for you and forgot about them? Maybe a female houseguest you trust left them in your house at some point? Maybe they’re his underpants, and you two need to sit down and have an open and loving talk about his love of little lacy things. Collectively, I’m going to put the odds that a pair of strange women’s underwear would end up in your bedroom for innocent reasons at any point in your lives at 2%. Unlikely. Quite unlikely. But hey. It happens, right?
A bit of math ensues - the proof itself looks like this:
4. …And the result comes out to 38%.
5. So according to our prediction, there is a 38% chance that our husband is having an affair. He’s still more likely to be innocent than not! But that has a great deal to do with how low our prior was to start with: how low the odds were that he was having an affair in the first place. The magic of Bayesian prediction is that our new conclusion is our next prior. So if the husband comes home next week with the old lipstick stains on his collar, X is no longer 4%, but 38%.
To show you how big an effect that has on the outcome of the prediction, I’ll run the same numbers again (with Y and Z at 30% and 2% respectively) but with the revised prior. The outcome is 90%.
When I submitted this to the SciShow editor, he sent it back to me. “I don’t understand where X and Y are coming from,” he said. “It seems like you’re pulling them out of thin air.” I was three paragraphs into an explanation of how that was exactly what I was doing, and that’s the genius of Bayesian prediction, before I realized that my explanation of this one point was going to exceed the length of a regular SciShow episode, and just scrapped it from the script.
But THAT’S THE GENIUS OF BAYESIAN PREDICTION! Blake was absolutely right: X and Y were only guesses. They were guesses that seem reasonable to me, but by their very nature, they are the values that they are because of my biases. Maybe you would set X or Y higher or lower, if you were the one making this prediction. That’s not important, however. What’s important—what’s absolutely friggin incredible—is that YOUR prediction and MY prediction will inevitably trend towards 100%, so long as we keep recycling our conclusions back into our priors. No matter how wrong you are to start with, applying Bayesian analysis to the data available to you will, beautifully, inexorably, make you less and less and less wrong.
By treating uncertainty and human bias as inherent parts of any prediction, Bayesian analysis elegantly enfolds them into the predictive process, and guides even the most stubborn and wrong-headed of us towards the truth.
Bayesian reasoning is not totally obscure: this is what statisticians learn about at the postgraduate level…and there’s one other type of person that relies on Bayes: professional gamblers. These same methods can be applied to horse racing, blackjack, sports betting, you name it. And all of the world’s most successful gamblers are—whether they know it by its proper name or not—practitioners of Bayesian analysis.
Side with the guys making money in Vegas. Bayesian reasoning is incredible.