Future Current

J. Storrs Hall is an independent scientist and author. His latest book is Beyond AI: Creating the Conscience of the Machine. It follows Nanofuture: What’s Next for Nanotechnology, which received the Foresight Institute’s Communications Prize and Drew University’s Bela Kornitzer Prize. At the 2007 Singularity Summit, he offered a revised notion of robotic morality, one based on an understanding of the evolutionary origins of human morals and ethics.

The following transcript of J. Storrs Hall’s 2007 Singularity Summit presentation “Asimov’s Laws of Robotics - Revised” has not been approved by the author. An audio version of the talk is available at the Singularity Institute website.

Asimov’s Laws of Robotics – Revised

Thanks for the Singularity Institute for inviting me. I’m having a blast and I hope everybody else here is, too. When I was a kid in the 1960’s, I spent a little time building robots out of tin cans with bicycle reflectors for their eyes. Of course I knew, because I was a great fan of Isaac Asimov, that a robot had to be built with the Three Laws, so I wrote the Three Laws down on a piece of paper and stuck them in the tin can that was the robot’s head. A friend of mine came along who was also an Asimov fan, he had also read the stories, and he said, “That’s not right. The Three Laws have to be built directly into the basic fabric of the robot’s structure.” So I took a nail and I scratched them into the tin can.

I assume everybody in here knows the Three Laws. Is there anyone who has not seen these? Okay, you can leave! These are really the classics. Any discussion of machine ethics or how to build safe robots for two-thirds of a century now has had to start with these three laws. In fact, people have written PhD theses on them. People, for example, have compared these to the three parts of the tricameral mind as described by Freud. The first law is the superego, the second law is the ego, and the third law is the id. In fact, as Asimov read absolutely everything, I’m almost certain he had read Freud and might have had some of this in the back of his mind when he made them up. The point is that they’ve had a tremendous amount of staying power and have really formed the basis for any discussion of how you make safe robots.

The only problem is that these three laws, as phrased in English, are not really what Asimov had in mind. What he had in mind in the first story he wrote about safe robots was that you build it to be good, and it’s good because you built it that way. It just works. This is the classic story “Robbie,” where you have a robot that is built to be a companion for a little girl, who just absolutely adores the robot. The mother, who can be seen standing with arms akimbo in the window, doesn’t like the robot and wants to get rid of it. The father is explaining to the mother why the robot is much more to be trusted than a human, because he’s just made that way. Asimov, being one of these great pro-technology types just assumed you’ll say that and everyone will understand it.

John Campbell, the great editor of science fiction, rejected “Robbie” because of the Three Laws of Short Fiction. It doesn’t make a story when the robot just works. You have to have something go wrong. So Asimov and Campbell sat down and wrote the laws in the form that I’ve shown to you. As Wendell was explaining yesterday, Asimov then spent the rest of his career showing how the laws, as couched in English, had loopholes and went wrong.

Asimov lived in the era of cybernetics. His concept of what the laws were really were scratched deeply into the metal of the robot’s structure. This picture, by the way, is from a really gorgeous book that was in the 1950’s by Pierre de Latil. This is Grey Walter, who is the inventor of these cool little electronic turtles. That’s run by vacuum tubes. It kind of scoots around like a Roomba, except that it doesn’t vacuum. This was in the 50’s. The caption to this picture was in their Sussex home Grey Walter and his wife are looking at their two children, one of whom is… so there was just a little hype behind this.

The reason I mention this book is that Asimov wrote the preface. In the preface of this book he referred to what was going to come about as the result of cybernetics, the understanding of the mind by science, and the ability to build robots. He referred to it as the “Intellectual Revolution.” He said that by analogy to the Industrial Revolution. This is going to be a period in history at the end of which things are going to be drastically different. It is going to change the human condition. We will have to do a lot less of the dirty work and maybe have more enjoyable and productive lives as the result of it.

To get back to his notion of what the Three Laws were actually like, there is a little byplay in “The Naked Sun” where he has a roboticist explaining the innards of one of the robots that has been just slightly tweaked for special tasks to Lije Bailey, the detective. He says, “If you read that strengthening of the C Integral governing the Sicrokovich tandem route response at the W65 level…” And in the actual book, Bailey says, “Double-talk!” The point is that Asimov’s notion of how the laws were represented were a lot more like circuitry than text. You built the robot with the structure, the same idea going back to “Robbie,” you built it so the way it understood the world involved that all of its actions would follow these laws. So you didn’t have to worry about what the definition of what a human was, or whether to obey and so forth. The definition of “human” was built into the way it actually managed to perceive things.

Besides the fact that the laws are not really text, but something one hopes is deeper in the understanding of the robot, and they probably would work reasonably well for a robot that is of the sort that Asimov was envisioning as a household servant, that’s not what we’re worrying about here. We’re worrying about things that will possibly, instead of being our servants, be our masters. Asimov’s robots, especially in the early days, didn’t improve themselves, and they didn’t become more intelligent or more powerful than human beings, except in a physical sense. We have a completely different problem.

Imagine trying to set down a set of rules for a machine that is going to exist in a situation where it is operating in a world of concepts that you don’t know. You have no idea not only what kind of world it is but the concepts that are going to be used to parse that world are beyond you. You are going to sit down and try to write a law that tells it what to do. It’s going to be like Hamurrabi trying to prevent the Enron scandal. You just can’t do that, especially not in English.

What you’re going to have to do is something that’s a lot more like a conscience. You’re going to have to build in something that is a lot more abstract. Vague, general principles, tradition, and that sort of thing. You’re going to have to build the machine in such a way that it can continue to exercise the same kind of judgment that you did in extending the rules itself in order to get the effect ultimately that you want, or maybe even that you don’t want. My great-great-grandfather kept slave. I don’t keep slave, but if he had written the laws and been able to make them ironclad, he might have done something that we would now see as being quite immoral. The fact that morality itself can evolve and get better argues against the notion that we should try and put these ironclad restraints on the robots in terms of the world as we currently understand it, because the robots are not only going to understand the world more deeply, but they are going to understand morality more deeply than we are.

The other thing is that there’s just no chance that everybody is going to take these laws that we’ve come up with and put them in the robots, anyway. There are two obvious classes of this, besides the fact there are going to be random people all over the world building robots. There are going to be military robots, and nobody’s going to build a military robot, like this one that actually does have a gun, with the first Law that states a robot can’t harm a human being. There’s also going to be corporate robots. In fact, I would go so far as to guess that by 2050 most corporations are going to be largely run by their management information systems. They are pretty much going to be required by law to have their first law be “make a profit.” You don’t have the opportunity force everybody who is going to be building AI’s to build AI’s the way you want them to.

Then what can you do? What we have to do is to design sets of rules and ways of building our robots and AI’s that we know they will be able to do well in the ecosystem of all the different kinds of minds that will exist tomorrow. They are going to want to occupy evolutionarily stable strategies so that they will be able to compete. In fact, they will be helped to compete by the fact that we know ahead of time, in some cases, just what the good strategies are going to be. We pick ones that are not only stable but are beneficial to us.

Law #1: An AI shall understand as much as possible. This actually goes back to something that Peter Voss was saying yesterday. There is a lot of evil that gets done just because people are dumb. The smarter the robot can be, the more it can act in its enlightened self-interest, the better off not only it will be, but the rest of us will be as well. Some people going back to Socrates have pointed out that this is one of the major points of any kind of morality.

Sub-law of #1: A robot should understand memetic evolution. Why should it bother to understand evolution? Evolution is where morals come from in the first place.

This is a vampire bat. Vampire bat, surprisingly enough, were the first place where the occurrence of reciprocal altruism was detected and documented in natural science. It turns out that the little bats go out and they find large animals to suck blood from and they come back. Some of them have been lucky and found a nice juicy animal, and some of the other ones don’t. What the bats will do is let the unlucky ones suck blood from them, in hopes that they will be given the same courtesy later when they have bad luck and the other one has good luck. This is actually the first place that they found this. This is a couple decades back, I think. The other interesting thing is that the vampire bats that do this have brains three times as big as the brains of the bats that don’t. You can point out the obvious reason that they need to be able to identify cheaters, understand the dynamic and recognize each other for this to work. It does seem that intelligence and the basic form of reciprocal altruism do go together in nature.

But why memetic evolution? If I were building an android, a human-shaped robot, I would not give it a backbone. In fact, I don’t know anybody that has built an android and actually built a backbone with all this silly stack of stuff. The reason we have backbones is because we are descended from fish. If you’re building a robot fish, you do give it a backbone. A backbone makes sense for a fish. It has to sinuously flow through the water, so a backbone is actually a reasonable thing for a fish to have. The AI’s we build are going to be built entirely out of our ideas. We can take the ideas that are backbones, that are sitting around with no particular reason to be here in our current situation because we had inherited them, and leave those out when we build the AI’s.

The AI’s will be in a culture that we have created. They are going to be composed of these ideas and evolved in memetic evolution. But they don’t need to actually fight each other. I mean, we don’t need to actually fight each other. We should be happy to be in a situation where our bodies can cooperate while our ideas are competing. It only makes sense to build robots that understand this in the first place. We can start the AI’s out in this attractor of memetic strategies that took us quite awhile and a lot of blood to find. It’s a stable spot, as far as we know, in many ways. So there is no need for them to repeat all the mistakes that it took us to get to where we are.

Let’s call a robot that has an evolutionarily stable strategy built in its operating principles an evolutionarily stable conscience, or ESC. The key idea here is that an AI that has an ESC, and he knows that it is an ESC, he understands what that means, why it’s there, and how it got there, would realize that if he moved off of it evolutionary pressures are just going to push him back onto it, anyway. He’s not going to go to a lot of trouble to change. He’s going to understand why he has the morality there and what it took to get there. He obviously should know all of history since we will certainly be able to put the Library of Congress on an iPod by that point. He will actually have access to all of this stuff, and have a deeper understanding of the whole business than we do. That means when he gets into a new situation where he has to build new concepts that we have never heard of, he will still have guidelines to extend the spirit of our morality and ethics into that new situation. That’s why I claim that, robots, AI’s, new minds should be build understanding memetic evolution to start with.

Law #2. Robots shall be open source. We live in a world that is largely run by artificial organizations with no conscience. When I started working on the book that became Beyond AI, I had this horrible thought that if we build these superhumanly intelligent machines and we don’t understand how to build consciences and don’t put them in, we are going to create a bunch of superhuman psychopaths. So I went around and I was doing research for the book and I talked to my good friend Robin Hanson, the economist who invented fact futures, among other things, and I described this dilemma. We’re going to be building these superhuman pyschopaths, and he says, “So what.” And I said, “Huh?” And he says, “Well, we live in a world of superhuman psychopaths and we aren’t all dead.” One of the main reasons that we do is that corporations are required by law, among other things, besides to make a profit, they are required to have their books inspected. The way corporations work internally is that the flow of money is their motivating principle. It’s essentially their emotions. Although we know of situations where corporations have slipped out of this and made hay when they shouldn’t have been, they by and large don’t kill off a bunch of people and do some kind of good in the marketplace.

The same thing is true of governments. The more transparent the government, the less likely it is to commit atrocities. We know that openness and transparency in government is a good thing. It’s by analogy to these properties that I claim that if you have an AI whose source code or the important parts of its motivational structure are in fact open and available to be inspected that it’s more to be trusted.

This is actually a corollary to the first law where we’re talking about being intelligent. The motivations of a robot, and this is just one of the particular parts of intelligence that we should think about putting in ahead of time, is that the robot will take account of the motivations of the other sentient beings around it, because it’s going to live in a world that is composed of those sentient beings, and its environment is going to be composed of what they decide to do back to it. This turns out to be an evolutionarily stable conscience in a very wide range of communities, ranging from the bats to the world economy.

Finally, it is the 250th anniversary year of Baton Powell, founder of the Boy Scouts. I just want to point out that the key difference between a real artificial intelligence, a human-level Turing test-passing AI, and any other computer system that we’ve ever built is the fact that by and large the AI is going to be programmed by people talking to it. We’re going to bring it up like a kid. If by some amazing chance we do manage to build an AGI in the next five to ten years, it’s still going to be awhile, because you’re just going to have gotten that three year-old, like Sam Adams‘, and you’re going to spend the next n years teaching it. We’re going to teach it according to the ways that we have developed teaching children morality for who knows how long. And this is one of the good ones. I think robots should be boy scouts.

This entry was posted on Tuesday, November 6th, 2007 at 1:09 am and is filed under J. Storrs Hall, Singularity Summit, Artificial Intelligence. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.