Intelligence Augmentation vs. Artificial Intelligence
To some, it seems "obvious" that significant human intelligence augmentation will come before human-level AI. To others, it's the reverse that's obvious. I don't think either is obvious, but I believe there's a strong likelihood AI will come first.
In the IA camp, one of the arguments goes, [Brain+Computer] will always be more intelligent than [Computer] alone. But this is untrue, as the I/O channels between brain and computer make all the difference, and with today's technology, these channels are quite limited. Even if we had million-electrode brain-computer interfaces, it would be a cybernetics problem to ask which outputs to plug into which inputs, and what changes might need to be made to the central executive to handle the new cognitive architecture without information overload or psychosis. Reprogramming the executive center of the human brain would require advanced neurosurgery and extensive knowledge of the brain, knowledge that could take decades of research and advanced experimental techniques to uncover.
Other cons for IA, in my view:
- Experimentation on the human brain is likely to be made illegal globally
- The design-and-test cycle is on the order of weeks or months
- Lack of human volunteers willing to die for the cause of IA research
- Someone left out the line notes for the brain's code
- Experimenting on the deep brain is difficult because neocortex is in the way
- All that medical hardware is really expensive
- The human brain was not designed to be upgraded
- Gene therapies not likely to give enough improvement for takeoff speed
A remark on that last one... the issue of takeoff speed. It's not enough to create an Einstein with IA. You have to create an Einstein that can go immediately to work on new intelligence augmentation techniques, and actually come up with something of use in a reasonable amount of time, before AI is developed. It seems more likely to me that an intelligence-enhanced human would just go into the business of creating AI. Smarter-than-human intelligence cannot just be a really smart human being - it has to be something qualitatively off the scale. Manipulating the genes associated with genius, as James Miller suggested, would likely produce "only" human geniuses at first. You'd need to go an extra level of theory and genetic engineering to get something genuinely smarter-than-human in a human-like package. I'm not saying it couldn't be done, but that the whole process could drag on for a number of years.
Benefits of IA:
- Evolution has already done a lot of work for us
- Some might think a human seed is more predictable
- Sparks human-centric patriotism in ways AI doesn't
On to the cons of AI:
- Present-day computers might not be fast enough to implement AI
- You have to build everything from scratch yourself
- Everyone is working on narrow AI, but AGI is unpopular
- Requires strong theory of general intelligence, difficulty unknown
- Stigma of excessive past claims
And the benefits of AI:
- Design-and-test cycle can be very rapid
- All aspects of the AI are read/write friendly
- Line notes are included with the code
- Cognitive features can be optimized for self-improvement
- Computational power can be expanded as funds allow
- Virtual worlds are available as a flexible training zone
- Hardware can be used to "overclock" beneficial functions
- Probabilistically realistic, flexible learning can be implemented
- Nascent AIs can share information with each other rapidly
- Much larger regions of the mind configuration space can be tested
- AIs can be copied indefinitely, allowing to commercial spin-offs
- Substantial advances in AI, but not IA, have already been achieved
- The hardware itself is inherently cheaper
- Little to no legal concerns
Comment away. Whether or not IA or AI reaches smarter-than-human intelligence first is pretty important, as the step into this new domain could spark a runaway self-improvement process, something I.J. Good called an "intelligence explosion". This is normally what we think of when we hear the word superintelligence.
“The Rapids of Progress”, by Mitchell Howe
From our earliest days as an intelligent species, it has always been more difficult to create than to destroy. From fire to fission, forces of great constructive potential have invariably been used as weapons against innocent people with tragic results. The cumulative losses to individuals, nations - and indeed, the whole human family - can never be fully understood.
Despite a pervasive - and in many ways false - sense of security that came with the end of the Cold War, we are far from being past the threat of technologically facilitated global ruin. The rise of trans-national terrorism may not on the surface seem nearly as dangerous as a full-scale atomic conflict. But the bold acts of hatred performed by those who place no value on their own lives remind us daily of the fact that, among billions, there will always be a few who would destroy civilization itself if they had the capacity to do so.
The day is approaching when this awful power will be all too abundant. Technological progress, that relentless engine that has refined our tools of creation and destruction, is not slowing down. It is accelerating. Technologies that seemed like fantasy a few years ago are now discussed as old news and common knowledge. Scientists recently built a lethal polio virus from scratch by assembling a custom strand of DNA. Nanotechnology - the engineering of materials and machines at the molecular level - is already churning out fibers and coatings incorporated into commercial products, and concept components for devices smaller than human cells are created daily.
Times of tremendous potential are upon us. Where we had only decades ago acquired the ability to observe the most fundamental processes of nature, we are now becoming masters of them. The most intractable diseases and disabilities cannot long stand against the perfect scrutiny and manipulation of genetic engineers. The endless drought of economic scarcity that lingers in so much of the world has no chance of resisting the impending flood of material prosperity unleashed by self-reproducing nanofactories that produce goods of unprecedented quality at negligible cost.
But this flood of prosperity cannot help but flow with a dangerous swiftness equal to the technological progress which propels it. Even ignoring the usual sources of murderous discontent, this radical shift in the quantity and quality of life will probably be sufficient to cause dangerous political upheaval. And, as has always happened with knowledge, the arts of genetic engineering and nanotechnology will inevitably see perversions into killing applications. But this time the danger will be far greater than the threat of nuclear catastrophe - an event entirely survivable by many who might nevertheless wish they hadn't. A custom-designed plague might be virulent enough to kill everyone, and a swarm of self-replicating nanomachines could swallow the biosphere whole.
Calls to relinquish technologies that could lead to such ends are unrealistic, as these are inextricably linked to positive applications - which greatly outnumber the negative ones. And any attempt to suppress technological progress through means of legislation and enforcement will only mean that when these technologies do inevitably mature, they will be in the hands of those who operate outside the law. Government bodies and committees certainly deserve respect for their ability to mediate disputes and create safety guidelines, but these have never proven capable of ensuring that a given technology is never once used for destructive purposes. And with advanced genetic engineering and nanotechnology, one single misuse may be all it takes to write the epitaph for the human race. We simply cannot rely on traditional organizations and regulations to guide us safely through these turbulent rapids of progress. The current is too swift and the hazards too numerous. And we know from history that somewhere, somehow, there is always a mistake. A human mistake.
Human mistakes are inevitable for the obvious reason that we are in possession of mere human intelligence. We also carry in our genes a myriad of irrational tendencies that do not serve us well, having been so far removed from the ancestral environments where they were useful. We often cherish our primitive instincts and delight in our child-like awe at mysteries "beyond human comprehension," but these are fertile ground for the kinds of critical failures that could send civilization crashing into the lodestones of oblivion. Our need, then, is for faculties beyond human reasoning, and for minds free of evolutionary liabilities. Whether collectively or in the minds of a select few, we need greater-than-human intelligence to skillfully shoot the rapids of progress and chart the seas of universal prosperity.
Fortunately, the means to achieve greater-than-human intelligence (a milestone called the Singularity by many futurists) are found within the very currents of technology pushing us to this critical juncture. Genetic engineering is one possible answer, but given the relatively long time it takes for a human baby to mature into an adult, this approach would probably not be timely enough even if there were no ethical questions to consider. Augmenting human intelligence by connecting brains directly to powerful computers is another option, but this may not do anything to reduce the likelihood of rash, biological mistakes being made, and may actually amplify their damage. At present, the only conceivable way to promptly give rise to greater-than-human intelligence free of the most significant human failings is through the creation of Artificial General Intelligence (AGI). (The "General" is sometimes added by researchers to distinguish it from the narrowly specialized programs that are often claimed, for marketing reasons, to possess Artificial Intelligence (AI).) Computer technology has matured to the point where most AI researchers feel that an AGI could exist on today's equipment, given the right design.
But of course the "right" design has not yet been developed, and will not be without determined effort. Any sufficiently intelligent AGI will be able to assist in the design of its own successors, making subsequent leaps in intelligence easier. But the initial design must do more than "merely" think in ways that match or exceed human capability. It must empathize with and care about the problems of its human comrades - a trait called "Friendliness" by some researchers. An AGI lacking this compassion would be as dangerous as any other technological nightmare. And since computer technology is so rapidly increasing in power and decreasing in cost, a time will come when rogue nations or sociopaths could create an unsafe AI - with consequences as potentially catastrophic as the misuse of nanotechnology or genetic engineering. Unless, that is, we already have greater intelligence on our side helping to discover ways to prevent such disasters.
The fate of humanity thus hinges on this question: When will we create greater-than-human intelligence that cares about our problems? There is every reason to act now. Without greater intelligence we are doomed to make human mistakes regarding forces so powerful that there may be no second chances. But, with the assistance of Friendly AI, we will have an extraordinary new capacity to not only safeguard our continued existence, but to meet every other challenge we currently face - or may face.
There is no greater or more responsible use for discretionary resources today than the advancement of this effort. Whether it be a few pennies, a few million dollars, or years of volunteer service, investments in Friendly AI will go further to improve the human condition than donations to any other charity or research project. After all, there are few causes that would not benefit from an infusion of Friendly superintelligence. But, more importantly, if we do not safely navigate the rapids of progress we will not be around to worry about disease, poverty or global warming. This is one swift ride that we are all along for, whether we like it or not, and it is up to each of us to help make sure the human family can survive the journey and come out on top.
(Read the latest short, Singularity-relevant story by Mitch at SIAI's new blog.)
Aubrey de Grey Talks to Google
Response to Cory Doctorow on the Singularity
Cory Doctorow is an editor of what, for a long time, was the most popular blog on the Internet, Boing Boing. (It's now #2 after Engadget.) He is also a science fiction author who is known for copyright activism on behalf of the Electronic Frontier Foundation. In the Spring 2003 issue of Whole Earth Magazine, an article of his was published, "The Rapture of the Geeks", that ripped into advocates of the Singularity and intelligence enhancement, such as myself. I will respond to the central accusations.
First, a couple definitions. The Singularity is the technological creation of smarter-than-human intelligence. We can further specify good Singularities, where this smarter intelligence is on humanity's side, and bad Singularities, where it isn't. So-called Singularitarians are individuals who advocate intelligence enhancement for global benefit. Rather than tackling the really hard problems - poverty, war, hatred, poor infrastructure, mental and physical illness - at our present level of intelligence, Singularitarians advise pursuing intelligence enhancement and then applying qualitatively smarter intelligence to these age-old problems. We also foresee a recursive self-improvement process resulting from smarter-than-human intelligence, where the first superintelligence is much better than humans at coming up with new intelligence enhancement techniques, and applies them iteratively, further magnifying the initial gains. To a Singularitarian, intelligence enhancement that improves benevolence as well as brainpower is the best possible investment in humanity's future.
Cory Doctorow: "The Vingean Singularity is at the center of a classic mystical belief system: to believe in The Singularity is to believe in the transcendence of human flesh and the ascension to a higher state—a belief that, in turn, depends on several highly dubious articles of faith."
Here, Doctorow associates the Vingean Singularity with transcending the flesh. While it is true that many advocates of the Singularity believe in the possibility of mind uploading, cyborgization, etc., none of these things are necessary to make intelligence enhancement a highly desirable prospect. Even if life and intelligence were somehow permanently affixed to proteinaceous water envelopes forever, it would still would be prudent to pursue intelligence enhancement for its own sake.
Humans share 98% of our genetic material with chimps, but the step from chimps to humans produced creatures that could walk on the Moon, exploit the power of the atom, and build skyscrapers. If a similar jump up in intelligence could produce the same discontinuous results, then wouldn't it be fascinating to take that step? And if that step is theoretically possible and will happen one day anyway, wouldn't it to responsible of us to help guide it so that the first superintelligences at least are given human-friendly initial conditions rather than human-unfriendly initial conditions? If the Singularity were sparked by human intelligence enhancement, would you rather the first augmentee be more like Fred Rogers or Vladimir Putin?
There are multiple reasons why the Singularity is not a mystical belief system, but the most obvious is that it is experimentally testable. If we cannot build smarter-than-human AIs despite our best efforts and human-equivalent computing capacity, if no brain-computer interface or genetic enhancement project gives rise to improved intelligence, then it will be proven that smarter-than-human intelligence is forbidden by the laws of the universe.
But if a reliable intelligence enhancement procedure is developed and can be applied to anyone for a low cost, would that not be an "ascension to a higher state" of just the type that Doctorow is belittling? It would be an ascension to a "higher state" in the real world, based on making deliberate neural modifications to let people think faster, more creatively, more empathically, with expanded working memory and capacity for complexity. This can be done by taking control of our own brains at the physical level, rather than the more superficial route (but so far the best we've had) of traditional learning.
Doctorow: "First off, Singularians ask you to believe that a model of a brain in a computer, properly executed, will become conscious—will, in fact, have a consciousness continuous with that of the person whose brain was scanned. While it’s true that consciousness depends on the brain— judicious experimentation with a bone saw and scalpel can readily demonstrate this—it’s an enormous leap to conclude that consciousness’s seat is in the brain."
Where else could it be? Doctorow is contradicting decades of work in brain science by suggesting that consciousness may reside somewhere external to the brain. Does consciousness reside in the stomach? The heart perhaps? Or is it floating by us at all times, on the supernatural plane? I'm not sure what he is suggesting, but it sounds pseudoscientific.
Regardless, Singularitarians are not asking anyone to believe that a model of the brain in a computer is continuous with its real-life counterpart, or that uploading is possible. It just so happens that many do believe it, but this seems like more of a common belief than a central component of Singularity advocacy, namely intelligence enhancement.
But if the subject is brought up, why not respond: if consciousness disappears when a brain is implemented on a computer, then it should be possible to observe consciousness disappearing in partly-computerized brains. For instance, people with hippocampal implants would be less conscious than ordinary human beings. Somehow I doubt this. Even if computers as we know them turn out not to be able to simulate conscious beings, who says we are limited to traditional computers based on the serial von Neumann architecture implemented in silicon? We could try parallel computers, biological computers, ultrafast neuron-equivalents, carbon computers... whatever works. The point is not to have a philosophical shouting match, but to dismiss carbon chauvinism - the idea that all life or intelligence must depend on traditional biological building blocks. For a humorous angle on this, see the short story "They're Made Out of Meat" by Terry Bisson.
Doctorow: "Then there’s the further presumption that consciousness exists at an atomic or even molecular level: that an atom-by-atom copy of the brain, properly modeled in a Turing Machine, will have all the data necessary to awaken into consciousness. As more and more subatomic particles are catalogued, particles whose properties range from counterintuitive to goddamned spooky, it seems equally probable that nano-disassemblers’ pincers will be far too clumsy to ever extract the important information contained in a brain. It’s cargo-cultism: the airstrips bring the airplanes, so if we lay down airstrips, the planes will come back. Put all a brain’s atoms into a brain-shaped pile, a mind will come back."
How come embryogenesis keeps putting the brain's atoms in a brain-shaped pile a third-million times every day, and the mind keeps coming back? If it is necessary to duplicate pregnancy rituals in order to create a conscious being in a non-carbon substrate, then it will be an inconvenience we'll have to bear. Again though, the issue of whether or not a human brain can be uploaded is irrelevant to the primary issue of whether or not intelligence enhancement (the Singularity) is worth pursuing.
Doctorow: "The Singularity depends on hypothetical technological events — nanotech, brain scanning, consciousness in the brain, sufficient granularity in the scan—but none is more wishful than the belief that the correct model will be lucked into."
It's interesting that Doctorow set out to write an article on the Singularity and ended up writing an article on mind uploading. This, along with his use of the term "Singularian" when he means "Singularitarian", shows that he didn't really research this article very well, but probably wrote it as a reaction to something he read that conflated the Singularity and mind uploading. A Google search for "Singularian" only yields a few results - all instances where people made up the word erroneously. "Singularian" is sort of like "irregardless" - a made up, etymologically incorrect word that gets spread about on a limited scale through repetition.
Doctorow: "After The Singularity, we’ll be immortal. All goods will be nonscarce. Entropy will be tamed. We will have complete mastery over our selves and our environment — we’ll be ascended masters. The best part is, we’ll get there using Moore’s Law. Write code, get smart, advance the cause and soon, you, too, will be immortal."
This seems like an attack on Ray Kurzweil's particular views, but it is a straw man because by definition, we cannot know precisely how things will go after the Singularity. The standard view is simple: if we aggressively enhance our own intelligence, the benefits could be large. Because most intelligence enhancement advocates are also transhumanists, the ideas of molecular manufacturing and radical life extension co-occur with discussions of the Singularity, but they are not the same thing. It's hard to tell if Doctorow picked this up while onstage at the Singularity Summit at Stanford, but I sure hope so.
Doctorow: "Your mystical belief: that everything will just transform on its own, for the infinitely better, because, well, because that’d rock."
From the beginning of an organized movement in support of the Singularity (around 2000), there has been the idea of personal responsibility and direct activism. So, as far as I know, there are zero thinkers on the Singularity that think it would be totally inevitable. Support of routes to smarter intelligence is a pro-active thing, that primarily manifests itself in Artificial Intelligence projects.
Synthetic Biology – Best Not to Ignore the Risks
Today's edition of Newsweek has an article on synthetic life, a topic of significant interest and concern. To use Alan Goldstein's classification scheme for various types of synthetic life, the kind being discussed here is Type 3, "synthetic biological", life forms with DNA/RNA programming, utilizing traditional biological building blocks such as proteins, with a genome synthesized from scratch in a laboratory. This is distinct from Type 2, "genetically-engineered biological" life forms, which are based on tweaks to preexisting genomes, and Type 4 life forms, "synthetic nonbiological", where DNA/RNA and traditional biological building blocks are not used and all functionality is engineered from scratch, like any machine.
The article reports that Craig Venter, famous for leading one of the first teams to sequence the human genome, has founded a new startup, Synthetic Genomics, which plans to make artificial organisms for converting sunlight into biofuel. Also interesting is that, apparently, some religious skeptics don't even believe that synthetic life can be produced. It's difficult to determine why. There are already millions of examples of functioning organisms coded by DNA, it seems odd that introducing a new one would somehow be physically forbidden. But creating life in a lab directly challenges religious fantasies that this is something only God can do. Everyone's favorite bioethicist, Leon Kass, is quoted in the article, saying, "I find it very hard to believe that, starting from scratch, we can somehow come up with a better [biological] system — one that's going to have much success." This is the same guy who believes that studying cadavers or eating ice cream in public are immoral.
Despite the odd pronouncements of anti-science dogmatists like Kass, we've been creating life and modifying genomes for thousands of years already, through selective breeding. Dogs, for instance. Many of the fruits we eat on a daily basis are modified versions of natural ancestors that were smaller, less nutritious, and more susceptible to the elements. Of course, there is a difference between selective breeding and creating new forms of life de novo. The latter is surely more powerful, but also more dangerous.
Rudy Rucker, a computer science professor made famous by his science fiction books, submitted a commentary on the topic of synthetic biology, also available on the Newsweek site. In the commentary, he dismisses away the dangers, saying, "What’s to stop a particularly virulent SynBio organism from eating everything on earth? My guess is that this could never happen. Every existing plant, animal, fungus and protozoan already aspires to world domination. There’s nothing more ruthless than viruses and bacteria—and they’ve been practicing for a very long time." He then goes on to talk extensively about some potentially radical benefits of the technology.
People like Rucker make transhumanism look bad, by spending all their time talking about the benefits, while handwaving away the risks. Synthetic biology will indeed be a serious global risk. The huge difference between intelligent engineering and blind natural selection should be obvious to someone as educated as Rucker, but apparently not. If I am knowledgeable about biology and have the tools to create new organisms from scratch, then it would be entirely plausible I could certainly construct something that poses a threat to all extant life.
The intelligent construction of synthetic organisms opens up a vastly wider design space than the one previously exploited by evolution and natural selection. In evolution, every genetic step must be independently adaptive, forcing a path through local maxima. Evolution cannot plan ahead, or intelligently construct adaptations oriented towards solving environmental challenges in the most general possible way. Evolution does not understand the concept of over-designing or fault tolerances - for an organism to be successful, it just has to reproduce a little bit faster than its competition, not ten times faster. When humans design a bridge, we design it to withstand a weight tens of times greater than its average load. Evolution can do no such thing.
One day, some synthetic biologist will become capable of designing a supervirus that can wipe out humanity. Then, ten will, then a hundred, and eventually, thousands. That's the nature of scientific knowledge - the bleeding edge of today is the used textbooks of tomorrow. Information wants to be free. Because synthetic biology will definitely become a real threat in the future, we have to start taking steps now to ensure that the field has proper regulation and oversight. SynBioSafe, a two-year, $312,000 project set up by the European Commission, is an excellent step in this direction.
Even if we think the chance of any given synthetic biology project in any given year leading to a global disaster is relatively small, over sufficiently long timeframes and for sufficiently many projects, the probability reaches unity. Synthetic biology is much more worrisome than global warming, nuclear war, or peak oil, because these things cannot kill everyone while synthetic biology can.
Recursive Self-Improvement and AI – Not a New Idea
"Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make."
-- I.J. Good, "Speculations Concerning the First Ultraintelligent Machine", 1965
"Once we have devised programs with a genuine capacity for self-improvement a rapid evolutionary process will begin. As the machine improves both itself and its model of itself, we shall begin to see all the phenomena associated with the terms "consciousness," "intuition" and "intelligence" itself. It is hard to say how close we are to this threshold, but once it is crossed the world will not be the same."
-- Marvin Minsky, "Artificial Intelligence", Scientific American, Vol. 215, No. 3 (September 1966), p. 257
"In a sense, artificial intelligence will be the ultimate tool because it will help us build all possible tools. Advanced AI systems could maneuver people out of existence, or they could help us build a new and better world. Aggressors could use them for conquest, or foresighted defenders could use them to stabilize peace. They could even help us control AI itself. The hand that rocks the AI cradle may well rule the world."
-- Eric Drexler, Engines of Creation, 1986