The fundamental insight of Thomas Bayes
Suppose you are taking a walk, and you see an animal dart between bushes. You only catch a brief glimpse of something furry. Was it a cat or a raccoon? If it is light outside, you will most likely conclude that you’ve seen a cat. You may not be aware of it, but you have just performed a computation: You’ve combined unreliable data with previous experience to find the most likely among different possibilities.
The word computation invokes complex symbols, machines and algorithms. However, we continously combine the input from our senses with previous experience to act and decide. But most of the information that we use is uncertain. How can we compute with quantities whose values we can’t pin down?
The answer to this question was given in the 18th century by the English mathematician and Presbyterian minister Thomas Bayes. Bayes was interested in how our beliefs about the world should evolve. For instance, imagine you just returned from Africa and experience headaches and dizziness. Your doctor suspects a rare disease and orders a diagnostic test. The test results are positive. However, the test is not perfect, and sometimes gives positive results even in the absence of illness. How certain can you be that you have the rare disease in question? What if the results were negative?
Bayes was the first to show how to combine prior information with new evidence to update our beliefs. In the previous example the prior information consists of the fact that you travelled to Africa and now experience symptoms of a rare disease. The positive test provides evidence that you do have the disease – but it does not remove all doubt! Bayes’ Theorem allows us to translate this situation into a mathematical relation. We can then determine precisely the chance of having the disease after a positive, or negative test result.
Bayes’ idea applies whenever we combine prior knowledge with new information. It even applies when we combine uncertain pieces of information. When you listen to your friend talk you do more than just hear her speak. You watch her lips move and you read her body language. What she just said will help you understand the words that follow. If we remove all of these cues, and only leave the sound of speech it is much easier to misunderstand people. Indeed, this is precisely what happens when you listen to a song on the radio. Mick Jagger’s promise that “I’ll never be your beast of burden” can be heard as “I’ll never leave your pizza burning,” or a hundred other variants.
Bayes’ idea is so powerful precisely because in most situations we deal with uncertain information. The best we can do is to quantify our certainty. How certain are you that your car is sufficiently insured, or that you will make it across an interesection before the light turns? Whatever information we have is never beyond doubt. And this is why the insights of Thomas Bayes apply to every aspect of our lives.