Accelerating Future Transhumanism, AI, nanotech, the Singularity, and extinction risk.

The Uncertain Future — Simple AI Self-Improvement Models

There is substantial disagreement between myself and 99.5% of other futurists that talk about human-level AI. The 99.5% talk about human-level AI as if it would be useful like computers are useful, allowing us to have automated secretaries and insurance agents and travel planners and logistics experts. They see AIs plugging into our economy just like humans do today. The AIs are usually used for rote tasks, somewhat analogous to the role of unintelligent software now.

I see scenarios like this as extremely confusing. Even with conservative assumptions, the economic impact of human-level AIs seems to be likely to be much larger than Rosie the Robot type scenarios. Last summer, myself, Steve Rayhawk, Anna Salamon, Rolf Nelson, and Tom McCabe, with help from a few others, sketched out some simple frameworks for modeling possible AI self-improvement speeds, among other things. We used simple variables like "X is how many dollars per hour the AI could earn", "Y is how many ops/sec is necessary to run said AI", and "Z is the ability of the AI to acquire physical power relative to its economic productivity", and saw unanticipated results which led some of us to believe that a typical takeoff curve might be steeper than we previously imagined. Our project was called The Uncertain Future.

Note that simple models of AI takeoff speeds can be useful even if human-level AI is 500 years away. (Although probably not if you think that human-level AI is impossible in principle.) Such models inspire us to think about a variety of questions unrelated to concrete future scenarios, including the usefulness of intelligence, how any agent might improve its own capabilities, and how humans perform on simple metrics of increasing their own efficiency and productivity. Investigating the question of potential takeoff curves is inevitably related to the question of whether we should have the audacity to speculate about futures with human-equivalent AI at all, and for me at least, the answer is yes, we should.

Modeling stuff

Looking at AIs and humans as caricatures in a simple model, the first thing that pops out is the differing modes of reproduction. Humans double their number about once every 40 years. AIs can double their number as long as they have the available computing power to do so.

So, pretend that the first human-level AI created needs 10^17 ops/sec to survive. If the AI or its owners want it to reproduce, they need to obtain another 10^17 ops/sec to create another AI. Because the computational requirements for intelligence are fixed while the cost of computing (currently) constantly changes, the cost of this computing hardware is likely to either be prohibitive (say, tens of millions of dollars) or reasonable (say, tens of thousands of dollars), with a somewhat smaller chance of being in-between (a million dollars). If the cost is prohibitive, then the AI will serve as little more than a curiosity (unless it displays qualitatively transhuman intelligence, in which case even tens of millions of dollars might be worth it for another). If the cost is reasonable, then the AI could be duplicated many times as long as its economic productivity pays well enough for buying more computers. Because of the problem-solving flexibility of human-level AI, much computing power would likely be taken away from conventional software and given to AI, though that is speculation.

Another fascinating element of simple AI takeoff models is the possibility of renting computing power, i.e., from cloud computing. If you rent computers, the barrier to entry on a new challenge is very low. Of course, this assumes that the AI doesn't take an entire Internet's worth of computing power to run, but if it's so computation-hungry, then how did it get invented by anyone to begin with? (Setting aside "Digital Gaia" type scenarios for now.) Anyway, say that I'm an AI looking at craigslist. I see 100 contract jobs that pay $50/hr, all in my field of expertise, and I want to do them all, but if I don't do them now, the employers will hire somebody else. What to do? Well, if I have the money, I can rent 100 computers to run temporary copies of myself until the jobs are all done. I take complete advantage of the available tasks, and I didn't have to spend huge amounts of money to buy and cool and maintain 100 me-equivalents of computing power. As long as the jobs I did are enough to pay for the rental costs and then some, I can keep making money this way.

(Yes, these scenarios are somewhat anthropomorphic, that's the point.)

Another possible scenario that comes to mind, which we didn't address in our model, was the idea of an AI dividing itself into parts to complete tasks that don't require its full complement of computing power. In a conservative model, an AI will always require all of its computing power to complete any task, but in the real world, self-division and self-recombination seem like perfectly possible moves. You could even have a collective of AIs that exist as either one big AI or a million little AIs as is required by the challenges that face them.

Once you have a model with 1) how much money the AI can initially make per hour with its skills, 2) computing power required, 2) cost of computers, and 3) how much money the AI can typically make per hour with its skills, there are other interesting variables you can introduce, like 4) will self-improvement make a difference, 5) where do returns from self-improvement level off, 6) will AI(s) acquire independent physical power, 7) will something halt the AI's progress, 8) how much physical/economic power is necessary for a monopoly on AI research, and 9) how much physical power will the largest AI/human group have at any given time. You can set 4, 5, and 6 to low values, put 7 and 8 at high values, and the takeoff curve is very gradual. That is, if you didn't already set very high values for 1 and 2 and low values for 3.

The point here is that AI takeoff speeds can be analyzed as a debate about different numbers that make up the latent variables in any takeoff scenario. It's not necessary to go off the deep end and accuse reasonable people of being Apocalyptic or Millennial, like James Hughes and some of the commenters here have done. What we have here are disagreements in the values of certain numbers. The arguments for the values of these numbers come from places like cognitive science and computer science, though no one can know them for sure in advance. The arguments shouldn't come from appeals to faith, or social pressure, or being ironically hip or nihilistic.

The critics of fast AI takeoff scenarios act like those who see a hard takeoff as likely are engaging in an escapist techno-fantasy of wish-fulfillment. To the contrary, if we're so nervous about AI, then wish-fulfillment would consist of a slower AI takeoff that is more manageable, not a fast one that takes us by surprise. Nothing could make me happier than if the challenge of AI could be handled by slow, careful approaches that have worked in the past -- introducing or recalling products based on their popularity, usefulness, and feedback from society. But I so happen to think that the first human-level AI created could bootstrap itself to godlike status in a relatively short period of time, making it necessary to focus on that discrete event and not a slow, unfolding process. This belief about takeoff speed comes from the values I set for the 9 variables I listed.

Why don't we take a look at my personal speculations about values for the variables, to get discussion going?

Variables that matter for AI takeoff speeds

Again:

1) computing power required
2) cost of computers
3) how much money the AI can typically make per hour with its skills
4) will self-improvement make a difference
5) where do returns from self-improvement level off
6) will AI(s) acquire independent physical power
7) will something halt the AI’s progress
8) how much physical/economic power is necessary for a monopoly on AI research
9) how much physical power will the largest AI/human group have at any given time?

We'd better get started. What is 1? Well, many people believe that intelligence can't be simulated in a computer. Then, this whole exercise is over. If this is your position, you can stop reading now. Much of the confusion in this whole area of study comes from people who disagree with the entire premise coming in and attacking blindly. If you disagree that building the dam is even possible, then debating you on the details of how much the dam will cost or how many cubic meters of water it will hold back become moot points. In the real world, such a person would realize that they are bored by the discussion and leave the room, but on the Internet, people will offer their opinion on everything even if (especially if) they disagree with the premise.

Continuing, with only the group of people who believe that intelligence corresponds to a certain finite quantity of computational capacity, we have estimates of this value that range between 10^14 ops/sec (Hans Moravec, also Ray Kurzweil using Lloyd Watts' estimate in TSIN) to about 10^17 ops/sec or even 10^19 ops/sec. There was even one odd paper that came out around 2005 that estimated human brain computing capacity at 10^100 or something like that. (Does anyone know where that went?)

Moving on, how much will computers cost at the time human-level AI is created? I assume that a human-level AI's worth of computing power will not likely cost more than $10 million, or it would have been too costly to create to begin with, though your mileage may vary on this one. If you believe that creating AI requires unlocking the "secret of intelligence" (and brain emulation doesn't work for some reason), and that secret won't be here for centuries, though computers will continue to get faster until they slam into physical limits, then perhaps this value will be only $1 or less. It all depends on when you think AI is likely to happen and whether you believe Moore's law will continue.

Next is how much money the AI can initially make per hour with its skills. This doesn't really matter -- say $10/hr. It can improve its skills and make more money as time goes on.

Will self-improvement make a difference? This is one of the more interesting questions and where intelligent people who believe in a soft takeoff and those who believe in a hard takeoff legitimately diverge. The soft takeoff thinkers appear to believe that self-improvement won't make a difference because humans already have "broad-brush" general intelligence that makes us about as smart and productive as it's possible for any intelligence to be. In that scenario, AIs can only increase their productivity by using up more computing power to make more copies of themselves.

In the model, I wrote the introduction to this variable as follows: "If an AI has human-level intelligence, it will be capable of analyzing its own programming and improving itself to some degree. How much more productive will a self-improving AI become per work-hour of self-improvement invested?" So, say the AI invests 100 work-hours, similar to a series of training seminars (though these might involve the AI actually reprogramming its mind directly), and makes itself 1% more effective. We can translate this as saying the AI makes $10.10 an hour instead of $10.00, or that it can make $10 in 36 seconds less than an hour. It makes little difference to the model. You can measure the benefit in a broader magnification of thinking and abilities if using money as the indicator makes you uncomfortable.

So, you can set any value you want for the difficulty of self-improvement. Depending on how difficult it is, there is an optimal growth-rate-maximizing amount of money/time to delegate to self-improvement instead of other activities, so choosing how much time the AI devotes to self-improvement is somewhat superfluous, though you can add that in too if you want.

Here's an example of improvement difficulty in the human realm. Say that going to college increases your earning power by 50% (doubtful, but this is just an example). If so, then assuming around 6,400 hours of class attendance and homework, the improved efficiency per hour of investment is roughly 0.0075%. In reality, the difficulty of self-improvement is not a smooth value, it could zig-zag all over the place as you go through successive S-curves of self-improvement. Still, in today's world, expected self-improvement rates of 0.001% to 0.01% per hour of work seemed typical, based on the examples I looked at. You can use this value if you want, or put in something higher if you think that AIs would be better at finding out ways to self-improve than humans, or if you think that having complete read-write access to every bit of your own mind would make any difference. In a hard takeoff, I speculate that improvement rates of 100-1,000% per hour could be achieved, but the conservative answer is much lower.

#5 asks where returns from self-improvement level off. You can't self-improve forever. Maybe a superintelligence can program 10^17 of computation as part of a 10^38 ops/sec ******* Brain such that it is a million times more effective at using the same computation as humans are. (Here, "effective" can be defined in any number of ways, but for the sake of this model it is ability to get money/computing, which leads to physical power based on another variable we will talk about in a second.) Or maybe humans are pretty much as good as you can get for 10^17 ops/sec and further improvement is impossible. (Yeah, right.) Or maybe we can pick something more in the middle like saying that becoming 100 times more effective is possible but 1000 is not. Whatever you pick becomes the growth ceiling for variable #3 (how much money the AI makes per AI-equivalent of computing power) as it evolves over time due to continuing investments in self-improvement.

#6 asks how many AI-equivalents of computing power will be “exchanged” for human-equivalents of physical power. Again, smart people have major disagreements on this. If you think that AI will always just consist of brains in boxes ready to do our bidding, and programmed not to want power, then the answer is that AIs will not want to exchange their computing power/money for physical power very much at all. Still, this value can't be zero. AIs with any input into the world whatsoever are certain to have some degree of physical power, at the least by making suggestions to humans about what to do. At the other end of the spectrum is a transcending AI plugged into a gigantic living sphere of active robotics. If the latter could defeat a million humans worth of physical power (your definition of what that is can vary arbitrarily, we only need to agree if we are comparing notes directly), but it only used 1,000 AI-equivalents worth of computing power, then you'd say that there is a money/computing to physical power ratio of about 1,000:1,000,000, or 1:1,000. The ratio could just as easily be 1,000:1 if you want, it all depends on your opinion.

The reason I decided to introduce physical power to the model aside from computing/money is the large variance and disagreement about how readily one transfers to the other. Some models make the mistake of including computing/money only, and part of the reason few people comment on these models is that computing/money doesn't always translate to power in the real world.

Question #7 asks if something unusual will halt AI progress in general. This question is just a probability. For instance, the probability that all AI research gets banned or that a meteor strikes the planet sometime when the AI is in the middle of figuring out ways to improve itself. I would put this value at less than 10%, but others will put larger values, especially if they think that some other disaster is likely to wipe out civilization before we make it to AI.

Question #8 asks how many human-equivalents of physical power would be necessary to monopolize all AI research and development. This means that the entity/union with this level of power would be able to prevent the creation of AIs they consider dangerous or restrict the growth of AIs with sufficiently divergent goals from their own. I'd hazard to guess that the answer is more than 100 million and less than a billion.

Question #9 asks how many human-equivalents of physical power the largest AI/human collective has at any given moment, as a percentage of total power available. Today, the United States has about 500 million human-equivalents of physical power, related to the fact that about 500 million people live here, which translates to about 8% of global total power, though you can see that this is just a wild estimate because the USA's power is really greater than just 8%.

If everyone in the United States had a human-equivalent robot that took their orders, then we'd have 1 billion human-equivalents of computing power, which would give us 15% of total power or whatever. The answer to this question is intimately related to what you said for every other question, especially how readily computing power/money translates into physical power, but I'd say that if the takeoff curve is sufficiently sharp, the resulting collective may control a substantial amount of total global power, say 30% or more. (Preferably, this would be a democratic collective consisting of everyone on the planet, but some forms of AI might disagree with my preferences here.)

That's it! There are other questions we put in our model, but those are the ones relevant to AI takeoff speeds. By doing calculations using all the relevant variables, we can sketch out various takeoff scenarios. Some other time I might discuss how to connect all these variables together mathematically, but for now I just leave you with the variables and thinking of how to put them to use is up to you.