Theory of Mind, Common Knowledge and Aumann’s Theorem
In a memorable scenes from the movie “The Princess Bride” the Dread Pirate Robert challenges the Sicilian bandit Vizzini to a battle of wits. The pirate puts a cup of wine in front of himself and one in front of Vizzini. He then asks the little Sicilian to guess which cup is poisoned. Vizzini’s answers by taking into account not only what he knows, but also what he knows that the Dread Pirate Robert knows that he knows, and so on. Unfortunately Vizzini, lacks a crucial piece of information, and suffers one of the most understated ends in movie history.
This exchange between the bandit and the pirate relies on ideas about how we reason about the world, and about each other – features of our thinking that are so ingrained we often take them for granted. First is our theory of mind: Our belief that other humans have thoughts, desires, and beliefs like we do. While our thoughts are not the same, I assume that you have an inner experience that is similar to mine. I can understand your motives, intents and feelings. We take this theory of mind for granted. Imagine thinking that other people have no desires or thoughts, that you are surrounded by robots with no inner life. Such mental blindness is a feature of some psychiatric disorders, and can be devastating.
Another idea takes us a step further: We say that something is common knowledge if we both know about it, and you know that I know about it, you know that I know that you know, and so on. Say two children see that I have hidden a cookie in a jar. If they both just know the cookie is there, they might wait until the other leaves to have the cookie for themselves. But if both know that the other knows about the cookie, they might both rush to get it before the other does.
When negotiating over a car, or a house we try to understand what the other person knows, and maybe what they know that we know. A result of the mathematician Robert Aumann may come as a surprise: Aumann won the Nobel Prize in Economics for his work on conflict and cooperation. In his work he gave a mathematical definition of common knowledge. Surprisingly, he then showed that two rational beings who start with the same information cannot disagree about anything that is common knowledge. Say you offer to sell me stock at $10 a share. That price can mean you have information that I don’t. I can therefore update my belief about what the stock is worth, and suggest a price I think is fair. Aumann’s result says that after this negotiation ends, we should both believe the same price is fair.
Mathematics therefore says that we can’t agree to disagree if we are rational. So why do we disagree about so many things. Perhaps we are never completely rational. Or we can never get to the bottom of knowing about knowing about knowing. But we do have a theory of mind – the gift of gleaning what it is to be in another’s skin. This gift will not erase discord, but we can use it to lessen it.
In writing this I used some ideas from Scott Aaronson’s post on the subject. The link leads to a transcript of his lecture on the subject. I highly recommend reading other posts in this blog. Prof. Aaronson is a deep thinker, and gifted writer.
For a related example (also mentioned in the lecture above), consider the Muddy Children puzzle, or the more dramatic Blue Eyed Islander Puzzle. The second link leads to Terry Tao’s blog where he writes in more depth about the subject.
There is evidence that some animals have a theory of mind. However, since it is impossible to know what animals are thinking, the extent to which this is the case is unclear. Here is an interesting recent paper on theory of mind and crows.