Probabilistic Reasoning – Judea Pearl
I have been reading Judea Pearl’s influential book “Probabilistic Reasoning in Intelligent systems”. There are many wonderful ideas here, but I think the following section is really something:
On the surface, there is really no compelling reason that beliefs, being mental dispositions about unrepeatable and often unobservable events, should combine by the laws of proportions that govern repeatable trials such as the outcomes of gambling devices. The primary appeal of probability theory is its ability to express useful qualitative relationships among beliefs and to process these relationships in a way that yields intuitively plausible conclusions, at least in cases where intuitive judgements are compelling. … What we wish to stress here is that the fortunate match between human intuition and the laws of proportions is not a coincidence. It came about because beliefs are formed not in a vacuum but rather as a distillation of sensory experiences.
He than argues that any calculus of beliefs that has evolved – and I’d say both in humans and animals – is necessarily based on computations with probabilities.
There are a number of ways to view this statement: One is to say that we naturally, but not consciously, represent and understand events probabilistically. Moreover, we naturally compute with probabilities – computations that can be quite complex when written down formally. If we are to understand the way our brains operate it is therefore necessary to understand these computations, how nervous tissue performs them, and when it fails to perform them “correctly.”
Another way to view this is that a probabilistic representation of the world around us is natural to us. We intuitively understand how to ascribe probabilities and how to interpret them. It is an ability that we can develop and formalize, but it is not foreign to us. We should therefore develop theories of the world using the language of probability theory. This last part is my interpretation, but I would tend to agree – even when we use deterministic models we know that we cannot take their predictions as completely accurate. We implicitly ascribe some uncertainty to them, even if this uncertainty is not explicitly stated.
Mathematics in everyday life 
I like your assertion that we should “develop theories of the world using the language of probability theory.” However, I think I disagree with you and Pearl on your interpretations of human beliefs. I haven’t read much of Pearl’s book, so perhaps I don’t fully understand the points you two have made. Pearl applies the term “belief” to people as though most people have a sense of likelihood about something. In his book he uses the example of P(Fly(a)|Bird(a)) = HIGH. However, I don’t think that humans or animals naturally store the probabilistic model. We may work our way through a probabilistic model, either in our minds, through the experience of trial and error, or even through the process of biological evolution. Yet it seems to me that what we store in our brains are deterministic approximations of these probabilistic models.
Shortly after the section you quoted, Pearl goes on to say this: “For reasons of storage economy and generality we forget the actual experiences and retain their mental impressions in the forms of averages, weights, or (more vividly) abstract qualitative relationships….” I think he got this wrong. The brain isn’t quite so limited in memory capacity. What the brain lacks is a mechanistic ability to computationally process all the data that it stores. I agree that the brain generalizes by making a model of the “qualitative relationships”, but I believe such models are deterministic as opposed to something that offers a likelihood value of “p”. You could argue that people can easily estimate a degree of certainty in their beliefs. It is common to hear someone say, “I’m 99% sure about that.” However, I would argue that is not because we have a probabilistic model in our minds, but that instead we have estimated the accuracy of our deterministic model. I think this was necessary biologically, as seen in the common example of “fight or flight”. I don’t think animals’ brains first compute the probability of being eaten and then flee if it is above some threshold. Instead, they have a (possibly) complex deterministic model that was formed through a probabilistic series of experiences and genetics. I think humans are the same but are capable of evaluating our own deterministic models.
Am I missing something perhaps? By the way, you may not remember me, but I was the lost colleague in Marseille that had lunch with you and Carson that first day. Really enjoyed the privilege of talking to you and learning from you. You and Carson inspired me to start my own blog as well (fietkiewicz.wordpress.com), mainly to generate ideas.
Hi Chris – I definitely remember you. Thanks for joining us for lunch in Marseille, and I am looking forward to reading more on your blog.
To answer your comment – I have to disagree. However, you will see that this is a mixture of personal belief supported by some evidence. First, I think that there is now good evidence that people do use probabilistic models when making all sorts of decisions. The best examples are probably cue combination experiments (Ernst and Banks, Nature vol. 415 (6870) pp. 429-33 is a classic). However, there is evidence that people compute with probabilities when doing many other tasks, as shown by Pouget’s, and many other groups. Mike Shadlen’s lab has shown that there are neurons in area MT whose activities seem to represent internal representations of probabilities, something that has also been shown in mice (Kepecs, et al. Nature 2008 vol. 455 (7210) pp. 227-31). You could argue that this represents the animals certainty in a particular outcome, but I think that this would again mean that probabilities enter the picture. Personally, I believe it would be difficult if they did not – some deterministic modeling is fine. Animals learn Newtonian dynamics, for instance. However, they need to not only be aware of the uncertainty in the world around them, but also take them into account when making predictions. Thus they need to compute with probabilities, ie have probabilistic models of nature.
About our memory – I think this is similar to perception. We live under the illusion that we have this amazingly detailed, HD picture of the world. In reality, I doubt that we actually compute with all that much: Our brains likely create an illusion of the details from experience, and only use the incoming information to adjust the unexpected things, and perhaps add a few details here and there at the locations that we are attending to. I do think that we deal with averages as Pearl says – we don’t recall individual trees that we have seen in the past. We probably have a general tree in memory. That tree gets a few embelishments when it is recalled to distinguish it from other trees in memory. However, I truly doubt that we store a picture for each tree individually in our mind. In fact, it is this ability to abstract and generalize that allows us to think in the way we do, as Rodrigo Quian Quiroga writes in his book about Borges’ stories. More specifically, consider people who are burdened with a photographic memory – for instance, the subject Luria’s Mind of a Mnemonist. It can be much more of a burden than a blessing.