From Nick Bostrom’s “Ethical Issues in Advanced Artificial Intelligence”:
Emergence of superintelligence may be sudden.
It appears much harder to get from where we are now to human-level artificial intelligence than to get from there to superintelligence. While it may thus take quite a while before we get superintelligence, the final stage may happen swiftly. That is, the transition from a state where we have a roughly human-level artificial intelligence to a state where we have full-blown superintelligence, with revolutionary applications, may be very rapid, perhaps a matter of days rather than years. This possibility of a sudden emergence of superintelligence is referred to as the singularity hypothesis.
I don’t think “singularity hypothesis” is the best phrase to describe this, because of the dozens of meanings associated with the word “Singularity” already, and this particular meaning being a narrow slice of those. The classic term, and the one which I prefer, is hard takeoff.
Many people believe a hard takeoff is likely because a ‘human-equivalent AI’ would in fact have human-superior capability. An AI with roughly human-equivalent, or, we might say “human-similar intelligence ” (to account for variations in domain competencies between humans and the first AI) would in fact have a much higher practical intelligence. Note that “human-similar” is a qualifier only pertaining to cognitive capability, like whether or not the AI can solve hard problems in a complex, real-world environment. The phrase does not pertain to motivations, worldview, or habits. An AI could be “human-similar” in intelligence while having a motivational system and modus operandi more foreign than any alien species described in any sci-fi book, ever.
So what cognitive and practical advantages would a human-similar AI have over a human actor? From “Relative Advantages of Computer Programs, Minds in General, and the Human Brain”:
More design freedom, including ease of modification and duplication; the capability to debug, re-boot, backup and attempt numerous designs.
The ability to perform complex tasks without making human-type mistakes, such as mistakes caused by lack of focus, energy, attention or memory.
The ability to perform extended tasks at greater serial speeds than conscious human thought or neurons, which perform approx. 200 calculations per second. Computing chips (~2 GHz) presently have a 10 million to one speed advantage over neurons.
The in principle capacity to function 24 hours a day, seven days a week, 365 days a year.
The human brain can not be duplicated or “re-booted,” and has already achieved “optimization” through design by evolution, making it difficult to further improve.
The human brain does not physically integrate well, externally or internally, with contemporary hardware and software.
The non-existence of “boredom” when performing repetitive tasks.
An increased ability to acquire, retrieve, store and use information on the Internet, which contains most human knowledge.
Lack of human failings that result from complex functional adaptations, such as observer-biased beliefs or rationalization.
Lack of neurobiological features that limit human control over functionality.
Lack of complexity that we have acquired from evolutionary design, e.g., unnecessary autonomic processes and sexual reproduction.
The ability to advance on the design of evolution, which is continually constrained by blindness, the requirement to maintain preexisting design, and a weakness with simultaneous dependencies.
The ability to add more computational power to a particular feature or problem. This may result in moderate or substantial improvements to preexisting intelligence. (AI does not have an upper limit on computational capacity; we do.)
The ability to analyze and modify every design level and feature.
The ability to combine autonomic and deliberative processes.
The ability to communicate and share information (abilities, concepts, memories, thoughts) at a greater rate and on a greater level than us.
The ability to control what is and what is not learned or remembered.
The ability to create new modalities that we lack, such as a modality for code, which may improve the AI’s programming ability-by making the AI inherently native to programming - far beyond our own (a modality for code may allow the AI to perceive its hardware machine code, i.e. the language used to write the AI, and other abilities).
The ability to learn new information very rapidly.
The ability to consciously create, analyze, modify, and improve abilities, concepts, or memories.
The ability to operate on computer hardware that has powerful advantages over human neurons, such as the ability to perform billions of sequential steps per second.
The capacity to self-observe and understand on a fine-grained level that is impossible for us. AIs may have an improved capacity for introspection and manipulation, such as the ability to introspect and manipulate code, which would be the functional level comparable to human neurons, which we can’t think about or manipulate.
The most important and powerful capacity of minds-in-general over the human brain is the ability to recursively self-encapsulate and self-improve its intelligence. As a mind becomes smarter, the mind can use its intelligence to improve its design, thereby improving its intelligence, which may allow further improvements to its design, thus allowing further improvements to its intelligence. It’s unknown when open-ended self-improvement may begin. A conservative assumption is human-similar general intelligence; but it may begin before then, and it is important to plan nonconservatively.
Some people read all of the above and are still ask, “won’t the AI need its human handlers to help it do everything?” At this point it’s hard to know what to say, because it should already be clear that we are dealing with something profoundly foreign and different from us. Something profoundly more capable. Deserving of our respect, and indeed even fear.
Here I am assuming that the first AI of human-similar intelligence will be a seed AI - an AI specifically designed and oriented towards improving its own mind and body to achieve its goals. Even if the first human-similar AI isn’t built specifically to be a seed AI, it is bound to become a de facto seed AI sooner or later, improving itself freely and enthusiastically. Improvement of capability is a subset of any goal as long as the goal does not expressly forbid it, and as long as there is additional utility to be derived from more goal-seeking activity. One might even call it a convergent subgoal, to use the Creating Friendly AI terminology.
To get from a data pattern on a computer system to some form capable of influencing the real world, the AI will need to use some sort of manufacturing technology. It seems more likely for it to invent its own than use a preexisting human-invented technology. Prior to the creation of its manufacturing base, the human-similar AI would be dependent on primarily social, financial, and informational channels of influence rather than physical channels. Even with these limitations, we can assume such an AI would be quite capable. With adequate hard drive space and processing power, the AI could extend its cognitive functionality by making numerous duplicates of useful subsystems, not “copying” itself into different entities per se but ensuring that it has the capability of more than one being while still retaining the unity necessary to pursue goals in a streamlined fashion.
At some point, maybe in days, maybe in minutes (we can’t think of every possible option), a human-similar seed AI is going to get ahold of some flexible manufacturing technology it can use to give itself a physical instantiation. Molecular manufacturing seems likely, although there may be other, less capable manufacturing technologies it uses to bootstrap its way to MM. Once a seed AI gains a manufacturing technology of this flexibility and power, it should be able to start converting local matter into actuating systems and processing elements. This process could happen very quickly. Recall that some bacteria can self-replicate in 15 minutes using readily available carbon compounds. Viruses can self-replicate at an even greater rate. A self-replicating nanoscale manufacturing technology based on covalent rather than ionic-and-below strength bonds could use it to convert raw materials into useful actuators at an even greater rate than the fastest bacteria. Using solar or nuclear energy, or something we can’t even imagine, this process would not be limited to using energetic compounds as feedstock, in the way that animals need to, but will be able to restructure dead rock just as easily as organics.
Very quickly, an AI could go from “just a computer” to a rapidly outflowing wave front of autonomous matter-transforming machinery. This hard takeoff scenario is part of the reason for the concern about Friendly AI - if such a powerful entity doesn’t explicitly desire our continued well-being, we are toast. But if it does… it could be the greatest thing ever to happen to the human race. A huge percentage of the world’s ills could be wiped out, if not overnight, extremely quickly. Material goods could be manufactured from raw materials on command and for zero cost. AI subsystems acting as peacekeepers and mediators could ensure that acts of evil such as the massacre in Darfur are stopped before they start. Risks such as asteroid strikes or nanoplagues would become old news, because this newborn superintelligence would be so powerful there’s no reason why it couldn’t distribute passive nanomachines to every square millimeter of the Earth’s surface and use them to nip any malevolent self-replicators at the bud.
Yes, yes, it’s all very shocking, and some people will never buy it simply because the consequences are too huge for comfort, but the hard takeoff conclusion is impossible to escape if you accept 1) a human-similar AI would have various cognitive advantages that put it significantly above us in capability, 2) these could be used to develop a superior manufacturing technology based on autonomous self-replication. I suspect there are numerous transhumanists not clear on the details of 1 and 2, and I would encourage any reading to go have a look at AI and Global Risk and Nanosystems respectively. It seems easier to get hung up on 1 than 2, because it’s easier to convince oneself that one understands intelligence thoroughly than it is to be self-convinced of an understanding of molecular nanotechnology, though both happen.
After saying that I think AI will be capable of utterly transforming the world in mere days or weeks, the whole point of originally writing this post was to say that a hard takeoff is not necessarily incompatible with some continuity, even high degrees of apparent continuity. A superintelligent AI could conceal its primary computational substrate underground, for instance, or conduct distributed computations across tiny robotic elements scattered across the planet’s surface but entirely invisible to the naked eye. It need not turn the skyline into dark ruins of megacities accompanied by unwelcoming lightning storms, a la the Matrix. In fact, there’s no reason why such an AI couldn’t restore our world to its formal environmental glory, sans deadly diseases, for instance. If given the seed motivation to care about human interests and aesthetics, it could make the world a very pleasant place to live. Because humans evolved many hundreds of thousands of years ago, our preferences tend to be aligned with creative spins on ideal ancestral environments, including but not limited to: fruit trees, flowers, green, rolling hills, a healthy animal population, flowing water, et cetera. There is absolutely no reason why the Earth’s surface can’t have a Luddite-friendly appearance while individuals desiring more radical self-enhancement go deep underground, into space, or into virtual worlds. I’m not going to speculate on the complex property rights and legal implications of such a system, but let’s just say that just because the challenges seems foreboding today does not mean that our superintelligent future selves or descendants will not be able to figure it all out.
So when people ask me, “do you think everything will change radically after the Singularity?”, I say, “yes and no”. An AI not aligned with human interests could change the world so suddenly and radically, ignoring us all the while, that the most likely outcome would be “the irreversible rearrangement of our molecular structure into alternatives conferring positive utility” (from the AI’s perspective), i.e., sudden death, not “cyborg wars” or whatever else some people were expecting. A hard takeoff also invalidates many of the conventional transhumanist visions, where incremental invention of products is the primary transformative means.
9 Responses »
