Stages of Ethical Development in Artificial General Intelligence Systems
Posted by Jeriaska on December 8th, 2008
Photo by brewbrooks
At the AGI-08 post-conference workshop Ben Goertzel presented on a paper with Stephan Vladimir Bugaj on the theory of stages of ethical development as applied to artificial intelligence systems. Incorporating prior related theories by Kohlberg and Gilligan, as well as Piaget’s theory of cognitive development, the theory is then applied to the ethical development of integrative artificial general intelligence systems that contain components carrying out simulation and uncertain inference – the key hypothesis being that effective integration of these components is central to the ascent of the AGI system up the ethical-stage hierarchy.
The following transcript of the AGI-08 post-conference workshop presentation “Stages of Ethical Development in Artificial General Intelligence Systems” has not been approved by the speaker. Video is also available.
Stages of Ethical Development in Artificial General Intelligence Systems
This was work by Stephan Vladimir Bugaj and myself. Stephan has been my collaborator on AI work for a long time. It reflects a different approach to the AI ethics problem. We started taking a look at theories of ethical development in human developmental psychology and wanted to see how you could port those ideas over to the AGI context. Did human ethical development theory make any sense for machines? We concluded that there are senses in which it does. There is a lot of detail, but I will go over the main outline.
The main thing that we tried to do is to align stages of ethical development with stages of cognitive development, then see how that works out in an AGI context. I will start with Piaget’s classic stages of cognitive development, with infantile, concrete, formal and then a reflexive stage—this is a name I use. It’s commonly called the “post-formal” stage in the psychology literature.
In the infantile stage, you are almost an animal. You perceive, act and sense. By the concrete stage you can have representations that are symbolic and a bit abstract. In the formal stage you can basically do science and mathematics, manipulate formalisms far beyond the context in which they were originally learned. In the reflexive stage, you can reprise and rewrite your whole way of approaching the world.
If you look in the psychology literature there is the sense that some people never reach the formal stage without formal education, and most people never reach the reflexive stage where they revisit their approach to the world. Instead, they keep in the same psychological paradigm that they had their whole lives. How this relates to the development of ethics is a question that has been addressed before in the human psychology literature.
Stephan presented a paper in the AGI-06 workshop where we explored how Piaget’s stages relate to AI systems that operate using uncertain logic. An infantile system can carry very simple inferences, but does not know how to control inference in a subtle way. When you get to the concrete operational phase, you need to be able to carry out more complex chains of reasoning. In the formal stage you get inference control as a subject of learning. You are learning how to reason, and reasoning about how to reason.
The reflexive stage almost correlates to revising your own inference heuristics. One example of the reflexive stage in human beings is trying to calibrate your own probability estimations. You realize that you tend to make estimates the wrong way and actually try to change the way you estimate probabilities. That is reflective—trying to modify your internal structures in a way to change how your own inference engine operates. It’s a hard thing to do and most people never get there.
What can we say about the stages of ethical development? One of the theories is by Perry, who is a psychological theorist. He looked at infantile approaches to morality. He called the lowest level as corresponding to a dualistic attitude. If you are not smart enough to understand much, finding the right answer to things is hard enough, let alone considering that there may be multiple solutions to every problem. He looked at the next level up, which he correlated with Piaget’s concrete operational stage. Some solutions are known, while others are not known. You have to figure out how to find the correct solutions, and maybe some answers are a matter of personal taste. You start to see that there are different approaches to problems.
In the formal stage you get beyond this multiplicity, and you realize that you need to evaluate the multiple possible solutions and use a higher order of abstraction to choose a right solution for yourself, even though there may not be any one right solution. You need to choose something in order to perceive. In his study of humans, he found that people are able to reflect formally and abstractly, scientifically in the manner of Piaget’s formal stage, also tend to be able to make choices in a way that goes beyond the relativism versus absolutism dichotomy. This ties in a lot with existentialism and many other things in philosophy.
The final stage, which merges with the reflexive, becomes hard to formalize. You could tie it in with postmodernism if you want to go in that direction. You choose a solution which is the right solution for yourself, yet realize the whole process of choosing a solution is part of your coupling with the world, and that this whole choice process has got to adapt in a complex way as you go on.
A little more simply and easier to grab onto is Kohlberg’s theory. He looked at ethics as the evolution of justice, fairness. This is something that is pretty simple to grab onto and tie into logic. Young children really do not get the notion of fairness. A very young child just wants what is good for themselves—positive stimulus and to avoid negative stimulus. When kids get a little older, by and large they understand the social hierarchy, the pecking order. They know what is right and wrong to do, so they no longer just want to gratify themselves. They know that what is right is what the parent or teacher says is right. They categorize right versus wrong, but not in a very nuanced way. It still goes beyond a baby, to whom the only wrong thing is what gets you whacked on the head.
What he calls the “post-conventional” stage is where you get into stuff like civil disobedience. What the bosses say is right may not be right. You are abstracting the principles of justice, going beyond the social order that you are in. You can see pretty clearly there that having the law and order mentality requires a certain level of symbolism and abstraction beyond being a baby, whereas having the post-conventional mentality really requires a higher order of formal reflection.
An alternate approach is Carol Gilligan’s ethics of care, posed as a feminist alternative to the ethics of justice. The earliest stage is the same—you are concerned with your own survival. Then, you reach a stage where you can look at altruism, self-sacrifice for the common good, which requires going beyond stimulus, abstracting things a bit. The highest level is where you must integrate the desire to preserve yourself and the desire to help others, which requires a greater degree of abstraction and reflection. What Stephan with my help sought to do in the paper was to see how you can put these together. You can actually align these various stages from these various theorists in a fairly natural way.
Starting with the infantile level in an AGI context, an infantile AGI will be just like an infantile human being. There is no consistent pattern of respect for the rights of others. The thing just wants what it wants. That is definitely the stage that our Novamente-controlled virtual pet is in so far. They have a hard enough time gratifying themselves and they do not have any idea of their social obligations or fairness with regards to others.
The next step based on the concrete cognitive basis, you get the common sense of the golden rule: do unto others as you would have them do unto you. You do not abstract that yet into the Kantian categorical imperative. You have a practical sense of fairness and give and take in the world. Trying to abstract right and wrong from one situation to a new situation that you have not been in. In an AI context, it is not too hard to see how that comes out from an application of probabilistic reasoning. This ties in with the theory of mind stuff that Selmer’s group is working on.
Beyond that level, to go on to what is called mature ethics, you need to be able to really reflect on your own ethical judgments and decisions. You are not just abstracting, “What would my teacher want me to do in this new situation?” You are thinking, “What are the principles my teacher has taught me imply about this situation?” Maybe the teacher is letting go of the principles, and the principles become the thing, rather than emulating an internal avatar of what the teacher is supposed to be. If you take a logic-based approach, you can see how this kind of reasoning corresponds to the ability to control more and more complex types of inference. If you were teaching an agent in a virtual world, you can see how this kind of learning could come about part and parcel with learning about more practical, concrete things of interacting in the world.
If you want to look at reflexive or “enlightened” ethics, the end goal, which is where AGIs should get if we want to have an engineered utopia rather than an artilect war, you want to have an AGI system that really has some variant of the categorical imperative, acting as if what you are doing should be a universal principle. None of this categorization solves any real problems of how to make an AGI system be good.
What we are trying to do is look at what psychologists and theorists of developmental ethics have said and aligning different approaches with each other to see to what extent they make sense in an AGI context. They do seem to make sense at least in the context of embodied and heavily logic-based AI. Whether they make sense in terms of other approaches to AGI is another question.
