Future Current

At the AGI-08 post-conference workshop on the ethical implications of artificial general intelligence, J. Storrs Hall, author of Beyond AI: Creating the Conscience of the Machine, presented on “Engineering Utopia.” The paper asserts that the likely advent of AGI and the long-established trend of improving computational hardware promise a dual revolution in coming decades: machines which are both more intelligent and more numerous than human beings. This possibility raises substantial concern over the moral nature of such intelligent machines, and of the changes they will cause in society. Will we have the chance to determine their moral character, or will evolutionary processes and/or runaway self-improvement take the choices out of our hands?

The following transcript of J. Storrs Hall’s AGI-08 post-conference workshop presentation on the paper “Engineering Utopia” has not been approved by the speaker. Video is also available.

Engineering Utopia

I’m Josh Hall, and I wrote the first book about machine ethics. If you think I don’t cover all the subjects in this talk, I spent an awful lot of time covering them in the book.

The point of interest is that with AGI, all of us more or less believe this is going to be developed sometime in the future. With coming nanotechnology, or simply following Moore’s Law a few decades out, not only are we going to have machines that can do whatever a human can do, we are going to have machines that can be bought for a dollar that can do whatever a human can do. That is going to have a fairly significant economic impact, I think. The question is, what do we need to do now, if anything, to make the world in which that economic impact has happened a better one for us to live in?

The first thing is that it seems reasonable that we would like our AIs to be good, as opposed to evil. The question is, can we actually engineer them so that they are good? The first obvious question is: “Are we going to have a chance to?” The reason why we worry about that is that there are some people who have claimed that AI, once it actually got to the point of being intelligent, might just take off. We will have to have designed it perfectly ahead of time. We need to analyze just how fast might it actually take off.

This is something that has happened in history: about 30 to 40,000 years ago there was a reigning paradigm—it happened to be a different species of human. Technology did not change over roughly 100,000 years. About 30,000 years ago, depending on what part of the world you’re talking about, a slightly smarter species came along… that’s us.

Our brains are actually 90% the size of theirs, but we appear to have been just a little more creative—enough so that in 30,000 years, the technology changed from stone knives to what we have now.

We are looking at this to happen again. The point is that the overlap period is the critical one. For the Neanderthal-to-human transition, this took about a thousand years at any given place, although the process was extended in time because it happened at different times all over the world. If this is going to happen with AIs in ten minutes, we had all better go to Eliezer and hope he has figured out how to build a Friendly AI and that he gets it absolutely right the first time with no bugs before he even runs it.

On the other hand, if this is going to take even as much as ten years, we have at least a little chance to play with it and maybe fix some bugs while it is happening. So it is of some interest to us to know whether it is going to be ten years or ten minutes.

How fast can an AI improve itself? I claim that the interesting part of self-improvement is exactly the same as the standard economic reinvestment problem. Essentially, what you have here is: Q is the total quantity at a given time t. This is the capital you start with, and this is simply an exponential. The crucial variable here is the investment rate. What is the investment rate? Well, the thing is, the rate at which hardware is improving, the rate at which software is improving, the rate at which AI and AGI are improving, are all a function of the amount of effort and monetary resources that are being put into them. We actually have a handle, a fuzzy handle, on how much investment effort is necessary to get a certain amount of improvement.

The first true AGI is probably going to be a baby. Or, it might be a dog in a virtual world. At the point when it actually is working so that it is learning and on the path to self-improvement, it is basically just going to be learning how to walk and talk. Some years later, we hope, we will get the AGI to the level of being an intelligent, well-educated adult, perhaps even an AGI researcher.

Let us call t=0 the time where the first AGI that gets to that point actually has the productivity of a single human researcher. At t=0, the self-improvement due to that AI is essentially flat. It is the amount of improvement that would go on through only one human researcher.

By assumption, at t=0 the capabilities of the AGI are the same as the human at t=0. What will happen instead is a reprise of the ’90s, when everyone started investing in the internet and you got the Dot-com boom. People will see that the AI is actually working, and all of a sudden they will go and read all the science fiction books and say, “Wow, we had better get in on this!”

What will happen actually is that there will be a huge amount of money from ordinary human investment sources shifting into AI, and that will account for the vast majority of the growth of AI in the early years, and not self-improvement on the part of the AIs.

In the paper I have a bit more analysis of this. What I claim is that once you start throwing money into it in another Dot-com boom phenomenon that you are going to get a growth rate in the total amount of intelligence that AI accounts for that is approximately twice the Moore’s Law growth rate. This estimate is roughly reasonable according to historical things like the growth of the internet.

If you allow that growth rate in AIs, what happens is that ten years after t=0 there is going to be a total AI of about 150,000 humans, which is a decent-sized company. They will be beginning to make a significant impact in the world economy, maybe like a Google or something. Presumably, we will have them all doing important jobs that require smart intelligences. We will have at least ten years before something really starts to hop.

At t=20 years, we are talking 26 billion human equivalence, and whatever the human race does is pretty much in the noise by that point. At t=10 we are still in the game; at t=20 we are out of the game… or, we are a minor player, let’s put it that way. This is what I think our margin for error is.

Obviously, we want to make our mind children be good people. I cannot go over to China and bop Hugo over the head and say, “Make your AI good!” On the other hand, what I can do is make an AI on my part that is going to form part of the environment that his AI has to operate in.

It turns out that the environment is a huge part of the determiner of the character of the people in it. I’m a tennis fan, for example, and there is a funny phenomenon in tennis that the players who grew up playing in clay are more honest and better sportsmen than the ones who grew up playing on concrete. The reason is that when you play on clay, someone can come over and look at where the ball actually hit, so if you claim it was out when it was in, you can get caught. The fact is that even though they are now all pros, they are being watched by judges, they are on a professional circuit and are all playing on the same court, the ones that grew up playing on clay are still more honest and better sportsmen than the ones who grew up playing on concrete. That sort of thing extends to every place where character is developed.

How long from now until we get to that first AGI? Some people think they have it already. Some of us think it may be as much as ten years. How long is it going to take to raise that baby AGI into a researcher equivalent? Ten years after that we have an AI Google, and ten years after that, AIs overshadow humans. We have a short window of opportunity, but we do have one. A word to the wise is sufficient.

This entry was posted on Thursday, January 1st, 2009 at 8:45 pm and is filed under AGIRI, Artificial Intelligence, J. Storrs Hall. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.