Toward Cognitively Robust Synthetic Characters
Posted by Jeriaska on February 4th, 2009
At AGI-08: The First Conference on Artificial General Intelligence, Andrew Shilliday of the Rensselaer A.I. and Reasoning Lab reported on the attempts by researchers at the organization to enable artificial agents to reason about the beliefs of others, resulting in game characters that can predict the behavior of human players.
The following transcript of Andrew Shilliday’s AGI-08 conference presentation “Toward Cognitively Robust Synthetic Characters” has not been approved by the speaker. Video is also available.
Toward Cognitively Robust Synthetic Characters
I’m going to be presenting on our approach to cognitively robust synthetic characters. The names listed here are all our collaborators, all of whom are in the Rensselaer Artificial Intelligence and Reasoning Lab at RPI.
The problem is probably pretty easy to explain to you guys, but I should preface it by saying that the domain in which we are particularly interested in this research is virtual environments, either entertainment-focused or war gaming. Either of those domains could apply here.
We see here a number of older videogames, a number of which I’ve enjoyed quite a bit. They all use NPCs, or non-player characters. In other situations, where we do not have an NPC, we might have an avatar, which is supposed to be a representation of the human player in the virtual world. What all these systems have in common is that while they make an attempt to produce a compelling person, I do not think they are expecting that anyone is actually fooled. More to the point, nobody is fooled and the reason for this is that these synthetic characters that are produced are effectively autistic. That is to say, they do not have mental states about other people’s mental states, which is the definition of autism.
This is the state of even the most popular recent games. This is a screenshot from the player creation interface in The Sims, where you create an avatar. The intelligence behind the avatar is described in a series of five attributes, with each attribute having one of possibly ten values. Really, you only get about 100,000 different characters out of the system. They’re cute, if you have ever played The Sims. They do things that you might ascribe to being humanlike, but they are in no way convincing.
What we would like to do is produce a cognitively robust synthetic character. The question is what are the key components that would need to go into that. Now, we are not doing AGI exactly. We are focused on particular aspects in particular domains, but we do hold to the thesis that any attempt at AGI would require a theory of mind. Now, that theory of mind would either need to be designed or, in some cases, it might emerge. Without it, you would be hard pressed to having an avatar or synthetic character which is truly general. You can take the case of natural language processing: How can we expect to understand what a human is saying if we cannot reason about the beliefs of the individual?
Some characteristics that we need to capture–Our agent needs to have beliefs about his or her own beliefs. It needs to have beliefs about other people’s beliefs, and beliefs about those people’s beliefs about our beliefs, and so on, for however complicated you want it to be. Some of the behaviors that we have focused on where this requirement is particularly important would be for example where you want to have a system that can lie to another person–to actually convince a human that something that is completely false is true. What goes into lying? We need to know how the person will respond to the false statements I am going to make in order to convince them of whatever I want to convince them of.
That is just one example. There are a number of other situations where this applies. Just to give you a brief background on some of the prior work related to this research, we have a synthetic character “E,” who is designed to be the embodiment of evil. The idea here is that E is able to reason about his own beliefs and, to some degree, can reason about the beliefs of others, but does so incorrectly. In this particular situation, he is not able to correctly reason about how his own actions will affect other people. We have a demonstration of this–the video of E walking through the reasoning that he went through when he decided to perform certain actions that are fairly obviously morally questionable is available online.
Another problem that is very relevant to our work is what is called the wise man puzzle. In this puzzle there are three wise men, although you could extend this problem for arbitrarily many wise men. This manifests in a number of different ways, but in this case we have each wise man being given a hat. That hat is either colored black or colored white, and any particular wise man cannot see what color hat he is wearing. He can, however, observe the hats of everyone around him.
All of the wise men are given hats and they are told that at least one of you is wearing a white hat. The question is, What can you infer? Wise man A takes a look at the wise men B and C, and concludes that he cannot make any rational decision about whether or not he is wearing a white hat. This tells wise men B and C that one of them must be wearing a white hat. If neither of them was wearing a white hat, then A could have inferred that he was the one wearing the white hat. Both B and C know that one of them is wearing a white hat. As B also announces that he does not know whether or not he is wearing a white hat, this allows C to infer that he in fact must be wearing a white hat. If he had been wearing the black hat, he knew that B would have seen that black hat and been able to deduce that he was wearing the white hat.
We have taken this problem and encoded it using an epistemic logic and solved it in the general problem. For any n wise men, we can use automated reasoning systems to solve that problem. There are a few demonstrations that we have put together that relate more closely to the virtual worlds environment, where we think that these worlds are a really good platform for testing out the systems we produced to deal with reasoning about other agents’ beliefs.
Video: False Beliefs in Second Life
What will happen in this scenario is there will be one automated agent, this is all done in Second Life, and the agent is going to be presented with two suitcases. In one of the suitcases there is going to be a gun. The synthetic character is the non-human-looking one. He observes two suitcases–the red one has a gun in it. He acknowledges that he sees that there is a gun in the red briefcase. One of the experimenters, the one in the suit, is going to leave the room so that he can no longer observe the situation. While he is gone, the other experimenter is going to take the gun, move it into the other briefcase and make sure that the synthetic character, named Ed, is aware that this gun has been moved into the other briefcase.
The robot sees that the gun is in the green briefcase and acknowledges that. We bring back the original experimenter that left and we ask Ed if the experimenter wanted to find the gun, in which suitcase would he look? Now, children of a young enough age who cannot reason about the beliefs of other people would get this problem wrong. The child, who knows that the gun is in the green suitcase, would not recognize that the experimenter was not there to see the gun moved. In this case, the agent comes to the correct conclusion that the experimenter would look in the red suitcase. That’s pretty much it.
