Future Current

Wendell Wallach is a lecturer and consultant at Yale University’s Interdisciplinary Center for Bioethics. He is recognized as one of the leaders in the new field of Machine Ethics, and designed the first course anywhere on this subject, which he has taught twice at Yale. Machine Morality: From Aristotle to Asimov and Beyond, which he is co-authoring and which will be published by MIT Press, explores the prospects for designing computer systems capable of making moral decisions.

Speakers Wendell Wallach and Charles L. Harper, Jr. at the Singularity Summit

The following transcript of Wendell Wallach’s 2007 Singularity Summit presentation “The Road to the Singularity” has not been approved by the author. An audio version of the talk is available at the Singularity Institute website.

The Road to the Singularity

I knew I would be following some tough acts, so I brought some actors to do this presentation with me. So, the Road to Singularity, starring… well, who does it star, after all? Maybe Data isn’t your idea of the Singularity. This came up in Eliezer’s talk. That’s one of the difficulties: what would really constitute a Singularity? Certainly we already have AI systems or computer systems in general that exceed humans in certain aspects of intelligence, but there are so many other aspects of, at least, human-level intelligence that we are only scratching the surface of with our computational systems.

As Rod mentioned, a black hole is sometimes considered to be a Singularity. I’m going to talk about the difficulties of implementing the Singularity, and I find that this conversation is sometimes like a black hole in that it’s very hard for light to emerge from it. On the one hand you have those who dismiss the whole project because of these complexities. On the other hand, these difficulties sound like they are a wet towel on the can-do engineering spirit that has animated the embracing of the project. So, today I’m going to position myself as your friendly skeptic. I’m skeptical in the sense that no one has convinced me yet that we understand enough about intelligence to really know whether we can pull this off or not. And in that, I imagine I am a surrogate for many others in the audience today.

But I’m definitely friendly to the project. We will no doubt see remarkable things in the next 10, 20, 30 years coming out of AI. And I particularly applaud this return to AGI systems: artificial general intelligence. It seems to me that this is part of a broader project in which we are being forced to look very deeply at how we humans function, and in what ways we are similar to artificial entities that we create, and in what ways we are actually truly different from the artificial entities we create. The difficulties can be broken down into three areas: complexity, thresholds, and bioethics. I’m going to quickly go over the first two because other speakers are really addressing them, and I hope to give a little more time to the third area, which is bioethics, and ethics in general, which is where I focus.

Models of complexity: the thresholds we are putting forward, Eliezer brings up years between 2020 and 2030 depending on how you look at the models. Ray Kurzweil has developed this. And it’s largely projecting Moore’s law out, and equating it with the computational capacity of the human brain, based on the number of synapses within the human brain. And, more or less, the basic computational theory of mind is equating synapses with bits of information.

We can go into some revisions of that, but that is the basic principle. And as I talk with neuroscientists, most of them will acknowledge that computational neuroscience, this kind of approach, has been wonderful in some of what it has helped us reveal about the way in which brains function, but most of them are very skeptical that we really know much about the modalities of the brain, or that all the intelligent capacities of the brain are really captured by this simple computational model. And the question is what role other elements of the brain - and I’m just throwing a few up here: glial cells, neurotransmitters, variability in neurons, microtubules - play in the faculties that we identify with intelligence.

Glial cells, for example, they far outnumber neurons. Until recently it was presumed that they mainly serve a protective function, but not necessarily a processing function. And suddenly we’re having theories arising that say maybe that’s not the case at all. I’m not here to judge those theories. I’m just here to point out that it’s not at all clear that our computational models of the brain are really the whole story at all. Microtubules: Roger Penrose and Stewart Hameroff have proposed that the brian is a quantum computer, not a simple digital computer, and that possibly microtubles, which are the lattice structure of neurons, play some role in the computational quality of the brain. Is this true? We’re so far from knowing. But it’s a fascinating theory.

If you are going to use brains as the model for artificial general intelligence, and you are going to use our human capacity for intelligence as a model, they differ very much from the computational models that we have today. It’s massive parallel processing, it’s extensive looping, and the feedback mechanisms are, to coin a phrase, mindboggling. The brain is capable of kinds of learning that we know very little about from computational systems. And if you damage your brain, there’s limited degradation. This is not the brittleness we see in computer programs, where one bit of information goes haywire and Windows locks up. That’s largely talked about in terms of the plasticity of the brain, not just the plasticity for youngsters, but that plasticity seems to be occurring throughout life.

So, here’s a quote from Albert Michelson. Albert Michelson, as some of you know, won the Nobel Prize for physics. This is a quote from him from 1894. “It seems probable that most of the grand underlying principles have been firmly established.” Little did Michaelson know that his own research would set relativity in motion. Anomalies coming out of his findings were really the stimulating factor for Einstein’s work and also played a role in the development of quantum physics. And I guess all I’m trying to bring up is, here we are again. We think that the combination of evolutionary theory and our computational models of the brain more or less provide us with an explanation for the complexity of why humans are the way they are. But it’s not at all clear that our computational models are adequate, and it’s not at all clear that our science is adequate. There may be missing pieces that we are not really addressing yet. At least, for myself, I feel that looking at certain phenomena, I’m not sure we’re addressing them at all.

Thresholds: there are all kinds of thresholds in computer systems. Even basic things like vision, language, and locomotion, there are all kinds of difficult problems that scientists are addressing. There are other areas, as I said, where we don’t have very robust learning. We still are confronted with the framing problem, which I happen to think in humans has an awful lot to do with our emotional foundations, the emotional heuristics out of which our reason emerges. Semantic understanding was brought up in the last talk. Evolutionary complexity: we’re creating these artificial life systems, but they are not going into robust development or breaking through into levels of complexity that we would expect of evolution simulations, in spite of the fact that we are able to run countless iterations in a matter of a second, countless generations of evolution in milliseconds.

Scaling scanning resolution: if you are going to use the human brain as a model for how you organize your bits, then you need to be able to see the human brain on that level. You need the temporal and spacial resolution. And there’s no doubt we’re seeing it with FMRI systems. We are seeing increased resolution in our ability to look at brain phenomena, but we are still on a remarkably crude level. This is not to say that most of us don’t believe that we will go to much deeper levels of understanding and make significant advances in all these areas, but there are technological thresholds that it is not clear whether some of these may actually be much more tenacious than our projections into the future would suggest.

Machine consciousness: there are some wonderful projects out there in terms of initiating some research into machine consciousness from scientists like Owen Hollin, Murray Shanihan, Edgar Hanlin, and Stanislov Dehain. But I was driving over the Golden Gate Bridge this morning, and I was remembering being in the Bay Area some forty years ago. When we talked about consciousness in those days, we used much more mystical language than is implicit in this phrase “machine consciousness.” Now, I don’t think that you need to believe in spirituality or to have a view of an immortal soul to recognize that consciousness is a very difficult concept and it means a lot of different things. It has come to be kind of the representational issue about the limits of not only computational models of how humans function but also even neuroscientific models.

This is a remarkable phenomenon. We are all sitting in this auditorium right now, aware of this wonderful space, and you are engaged with me in reflecting on some remarkably deep questions. How do we do it? How do we have this experience? How do you make that leap: from underlying cognitive mechanisms that process information to having this phenomenal experience? Is it simply an emergent property out of complexity? Is it epiphenomenal: a side benny from all this unconscious processing going on. Or does consciousness really do something? Is it important and participate in the kinds of general intelligence we’re talking about? And are we really approaching developing systems that are conscious? Is it just a question of arranging the bits in the right way? Or, for example, might it be a phenomenon that is particular to carbon-based matter. We don’t know. Is consciousness required for semantic understanding? How deep is the understanding if you don’t have consciousness? Is it a quantum phenomenon? Is it dependent on the science we have, or is some other science involved?

Integration: the next level of complexity. Here we are, thousands if not hundreds of thousands of unconscious computational mechanisms, perhaps individual modules performing specific functions, all working together to allow me to give a presentation of this caliber at this moment. Do we know how to put all that together? Perhaps our computer systems may help us in that integration, but I’m afraid we are confronted with something similar to the Humpty Dumpty project. You know, he sat on the wall, he had a great fall… king’s soldiers, king’s men, couldn’t put him back together again. So, we’re hearing so much about different aspects of what is happening on different levels neuroscientifically and computationally, but putting it all together, that would be a remarkable feat. And is putting it all together just a question of assembling the modules together? How does that integration happen? Perhaps evolutionary robotics or some of the other trajectories we have in our scientific approaches will help us. But, it is still not very clear.

Bioethics: this is where I would like to focus a little bit more. Bioethics has two concerns. One is the ethics of the machines, their sensitivity to ethical considerations, and that’s what I want to focus on more. But let’s first talk about the societal concerns. This pitting of promise and perils against each other, how is that going to be handled from the perspective of public policy? I think we can see that we should not underestimate the political power of fear. And AI research is being combined with concerns about a broad range of issues in bioethics: from genetic engineering, from nanotechnology, from neuropharmacology. How is this going to be handled from a public policy approach?

Now, most of us would argue it’s not likely to stop scientific research, but it certainly could slow it down if the fears take hold. I will make one prediction. I’m going to predict that we are just a few years away from a major catastrophe being caused by an autonomous computer system making a decision. Now hopefully we aren’t talking about anything on the order of 9-11, but I think something very tiny could precipitate the kind of public policy response we are seeing comparable to 9-11. There is this public policy issue of assessing the risks. There is this massive gap between speculation hype and what the existing technologies can do. And therefore one can argue that if there are public fears, a lot of them are unreasonable. But reason I don’t think totally dictates public policy.

The precautionary principle, just rejecting any research that could potentially have some problem out there into the future, that’s not particularly helpful either. How many of us would have given up present-day computer technology based on the robot takeover fears of the 1950’s? Not many of us. There will be a policy debate, and it’s already started. And there are those who will make political fodder out of resisting scientific progress. And I think one thing we seriously need is some mechanism for evaluating when thresholds that hold real potential dangers are likely to be crossed and helping both public policy leaders and the public at large discriminate between the real challenges and the highly speculative challenges. And I think there is a responsibility for all of us in that also. Because we may be able to get our funding by making big promises to the government and to other funding sources about what we are going to do in a few years, but if we over-promise we are also likely to feed the fears.

Machine morality: this is a new field of inquiry. It has many names. Eliezer mentioned Friendly AI, which is a term he coined. Peter Danielson came up with the phrase “artificial morality.” I happen to particularly like the alliterative “machine morality.” “Robo-ehtics” is a term being used in the European Union. So what is this new field of inquiry about? Well, it ain’t about house-training your robo-pet. Nor is it about ensuring that the governator and his future minions don’t take over the world. But it is about implementing moral decision-making faculties in artificial agents. And it’s being necessitated by the fact that we are producing more and more systems with higher degrees of autonomy that make decisions that can’t be totally predicted by their developers.

To the extent that their developers can predict all the territory of action for those systems, and the systems do stay within a realm that I would call “operational morality,” meaning the programmers can more or less anticipate the kinds of decisions they will make, even though they may not be able to anticipate when and where they will make those decisions. But there is also a territory that gets beyond that, that I would call “functional morality,” which is when computers become explicit moral reasoners. There has been a trajectory in the growth of AI, and that trajectory is toward greater autonomy - and it’s not just AI, it’s technology in general - and also greater sensitivity. This is about developing AI systems that are sensitive to the moral considerations, the information that we would want a sensitive system to factor into its decision-making.

So, think about a hammer. A hammer has neither autonomy nor sensitivity. But a thermostat might have a little bit of autonomy, a little bit of sensitivity. So, this is the trajectory we’re talking about: from insensitive or dumb machines, to operationally moral, to functionally moral, to artificial moral agents - meaning, at least sensitivity comparable to human sensitivity, though that may not be considered satisfactory for an AI system, and it’s not always satisfactory for humans.

There are four questions which animate this field. Do we need artificial moral agents? I would say, unequivocally, yes. If we are going to develop more autonomous systems we need them. Do we want computers making ethical decisions? That’s a question that the philosophy of technology has been considering for quite a while. It’s a good question. We need to look at it seriously. What kind of decision-making systems do we want and not want, and will decision-making systems undermine human autonomy and human initiative.

Whose morality? What morality? Age-old question. I’m not going to answer that here. But the question I would like to look at a little bit more is how can we make ethics computable? What role, for example, should ethical theory play in helping us design the control structure for the systems we build? Well, you can think of that in terms of two approaches. One would be the top-down approach, which is largely thinking about any ethical theory - take your favorite, Ten Commandments, the Koran, Asimov’s three laws (if we are talking about robots) - and taking that ethical theory to, in effect, dictate the control architecture for the system.

That would be a top-down implementation. A bottom-up implementation may or may not have a prior ethical theory. And if there is an ethical theory, it’s only a way of specifying the tasks or goals for the system. But it’s not a way of specifying how that approach will actually be implemented. It won’t give the details of the implementation or necessarily define the control structure. When we think of top-down approaches, there are, as I mentioned, Asimov’s three laws which fall within that. But, as you’ll remember, Asimov’s three laws were a literary device for him, and though they have captured the popular fantasy, he demonstrated in story after story after story how the system would break down with just this simple laws. He basically demonstrated that rule-based morality is not adequate.

The other two major contenders are utilitarianism - the greatest good for the greatest number, just compute it. Of course, how you compute it is another story. Or duties, the ontological logic, respect for rational agents, a consistent moral logic, such as the one proposed by Immanuel Kant. I don’t have time to go into the details of these, but if you start to look at it computationally, all these approaches have a number of problems. But the main one that the two biggies, utilitarianism and the ontology have, is they both suffer from the frame problem. They both have problems due to the computational load requirements of psychological knowledge, knowledge of effects of action in the world, and estimating the efficiency of the initial information. This is one of the areas where we humans are unbelievably remarkable. We have a way of handling the fact that we can’t predict the results of our actions. We have ways of handling the fact that our information is often far from adequate. We have a way of handling uncertainty.

When you think of bottom-up morality, you think of three different trends. One is evolution, and there has been a great deal of focus in evolutionary psychology and in game theory recently in developing or on understanding what may be the evolutionary roots of human morality. Meaning that what we think is right and good is not just about what we think ought to be, but a lot of what we think ought to happen is informed by our psychology… controversial, but nevertheless interesting. So, if you wanted to implement that evolutionary approach, you might be able to develop some bottom-up morality from alife systems, through genetic algorithms, through evolutionary robotics. Those are various evolutionary approaches. Then there’s development and learning, the way a child learns about morality. And we have many models from psychology, largely from Jean Piaget, Lawrence Kohlberg, Carol Gilligan.

Presuming that you had a robust learning platform, presuming that Cog was functioning as, let’s say, a six or ten year-old might function - Cog, of course, is now retired - but our learning systems really have that kind of robust learning capability, then perhaps we could take that kind of a system through learning about the moral dimensions to the kinds of decisions it would encounter. And there might be an advantage to that, presuming you were producing thousands of systems, if you could take one through that developmental course, the moral module you could transfer to that. I’m not convinced that that’s how it works, but that is a possibility.

And the third bottom-up approach is just simple fine-tuning of a system that every engineer does. You have certain goals you want the system to approach, so you alter the weights of various inputs in order to reach those goals. But in all of this we need to keep a distinction in mind. Something that distinguishes us from these systems we’re developing. We’re biochemical platforms. Our intelligence emerged out of emotions and instincts. The cerebral cortex grew out of the emotional brain. Our higher order faculties emerged. Computers start as logical platforms. And if they have emotions or instincts at all, it’s only because we elect to introduce them into the systems.

Now, this may provide computers from an ethical point-of-view with certain advantages: calculated morality. The speed of a computer, like the speed of Deep Blue 2, can look at more options than we humans can look at. Herb Simon, one of the fathers of AI, received the Nobel Prize in economics for his theory of bounded rationality, where, in effect, he made it clear, that we are very limited in the options we can consider. So a computer system may be able to consider a much broader, wider array of responses to a challenge, and in that, may select better responses than we would.

Absence of base emotions. There won’t be greed in this system. Of course, I’m saying that’s a somewhat dubious conclusion, because we may actually introduce greed into an evolving system as a way of helping it evolve. Is the absence of a nervous system subject to a moral advantage? No sexual jealousy. Well, I grew up in the age when stoicism still dominated most moral reflection. Emotions were a bad thing. They got in our way. If you were going to be a moral person, you had to learn to reason and transcend your emotional prejudices. But we now live in the age of emotional intelligence. It’s not just that we are recognizing that there is a value coming out of many of our emotions, like our moral sentiments - an idea that goes back to David Hume - but it’s that our emotional heuristics, our unconscious emotional responses, may have an awful lot to do with what we call reason.

This suggests that at least moral decision-making faculties may require something more than simply rational faculties that a logical system can build up. Not just emotions, but they are also going to need other super-rational faculties, such as being sociable. Rod showed you some examples with Kismet. They may need to be embodied in the world. They may need to have a theory of mind to function as moral agents in many contexts. For those of you who don’t know that term, it’s a rather clumsy phrase for the ability to impute goals, motives, and intents into other people. So, if we’re going to facilitate trust in machines, are they going to have to have the full array of human faculties? And if they do have the full array, and emulate humans as these two robots by Hiroshi Ishiguro do, are we going to feel comfortable with them?

What’s been done so far? Not much. Some very small experiments, and yet very important because they are helping initiate thinking about building moral decision-making faculties into systems that do very discrete tasks. And that’s where we should direct our attention. Not at some future entity that we may or may not have. But if you put it all together, what do you get? Well, this is a model of an AGI. This happens to be Stan Franklin’s LIDA, which he built together with Bernie Baars, and it tries to capture Bernie Baars’ global workspace theory, which is perhaps one of the most respected theories of consciousness in neuroscience these days.

I’m just flashing this up on the board for you, not because I think this is the only approach - I think the Novamente approach and many others may be utilized in this way - this just happens to be a model I’ve been working on. Stan Franklin and I have been working on whether this model of how humans make sense out of all this lower-level information we get, to make a decision about what their world is like and what we should do next. Whether that kind of model can be captured and adapted for moral decision-making.

So, in conclusion, there are clear limits in our ability to develop or manage AMA’s, and it’s going to be incumbent upon us to recognize those limits so that we can turn our attention away from a false reliance on autonomous systems, toward more human interventions into the decision-making processes of computers and robots. Thank you very much.

This entry was posted on Monday, November 19th, 2007 at 9:10 pm and is filed under Singularity Summit, Artificial Intelligence. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.