Perspectives on Artificial General Intelligence and the Singularity
Posted by Jeriaska on June 7th, 2008
In the introduction to the AGI-08 post-conference workshop on the futurological implications of artificial general intelligence, Novamente Chief Science Officer Ben Goertzel argued that it is worthwhile in planning for the future to put a certain amount of attention on the speculative aspects of present-day emerging technologies. The workshop’s introductory talk focuses on the significance of artificial general intelligence to the hypothesis of a technological singularity.
The following transcript of the AGI-08 workshop presentation “Perspectives on Artificial General Intelligence and the Singularity” has not been approved by the speaker. Video is also available.
Perspectives on Artificial General Intelligence and the Singularity
Vernor Vinge is a great science fiction writer and a brilliant thinker. He posed the notion of a Singularity and proposed that sometime–not in thousands of years or millions of years, but maybe tens of years–we could have AIs much smarter than humans with incredibly profound effects. This idea has been popularized most recently by Ray Kurzweil in his book The Singularity Is Near. Ray, being a prognosticator rather than a science fiction writer, has put a date on it: 2045. I’m 41 years-old–if I keep eating my vegetables and getting some exercise, I may make it.
If we take the Singularity as one family of scenarios that might happen, there are a whole bunch of sub-scenarios that are worth thinking about. We have the Terminator, which not only involves Arnold Schwarzenegger getting naked, but it involves AIs becoming superhuman and basically enslaving humans. This does not seem the most plausible future to me. I’m not sure why we would be that useful to them if they were superhuman, but it is certainly a scenario that has appeared often enough in the popular media that we cannot dismiss it entirely.
Another possible scenario, which I think is more plausible and more desirable, although not a utopia, is AGI developing to a superhuman level just enough to maintain the status quo: there are surveillance cameras everywhere watching us, the AGI may ensure that everyone is perfectly happy, until you start thinking about designing a new AGI system. Then, all of a sudden, you find you are thinking about what you are going to have for dinner instead. This is a plausible medium-term possibility, which is not actually the worst-case outcome. You can imagine a productive life proving theorems, playing music, hanging out on the beach under the watchful eye of the AI Big Brother.
Then we have Utopia, which is not very well defined. It is not even clear that it is consistent with the human motivational structure. The idea is that somehow AGI and the Singularity are going to bring us all paradise, which I think is not as common a point of view among futurists as parodists would make it out to be. This is, again, something that has been discussed.
If you are familiar with the book by Charles Stross Accelerando, he has an interesting point of view, where we have AGIs that become so much smarter than humans that they have no use for us. That is also a viable possibility: we make a superhuman AI and it says, “So long, and thanks for all the fish.” There are a variety of other scenarios that we could envision.
There is a point of view on all this, which is probably not all that common in this room, that these scenarios are just too scary, so stop working on AGI intentionally. Bill McKibben wrote a book called Enough. Let’s live as humans and develop simpler technologies, and maybe narrow AIs as tools to eliminate the need for annoying work, get rid of disease and violence, and live happily. Bill Joy has been a prominent advocate of this perspective. He wrote an article “Why the future doesn’t need us” arguing that if we did continue with AGI, we are almost certainly screwed, like the dinosaurs were when mammals emerged. Do we want that badly to create something better than ourselves that us and our wives and children are all wiped out?
Kurzweil has taken a more optimistic point of view, which he calls “fine-grained relinquishment.” He thinks that by judiciously choosing the right path to take and which paths to avoid, we can craft a future where powerful AGIs evolve and grow together with us. It will not be us versus them. We will be so fused with them–maybe by plugging them into our brains, interacting with them online, by becoming cyborgs–that AIs and humans will evolve into the future together through judicious partnership.
Eliezer Yudkowsky has been quite effective at pointing out the distinction between making an AGI and making an AGI that is going to be able to cooperate with humans and be beneficial to human beings as we evolve. He points out that failing to build an AI may be frustrating personally and bad for your research career, but succeeding at building AI while failing to give it a controllable trajectory and a proper ethical system is a lot worse. It not only ruins your prospects for tenure but may annihilate us all.
There are some very hard problems here. How to make an AGI system that can get smarter and smarter unlimitedly, yet as it becomes vastly more intelligent than us, still behaves in a way that we would desire. Once you figure out how to do that, actually getting that done instead of some alternative way to AGI that is less beneficial, is a very hard problem. There is the possibility of a pragmatic approach, where you teach AIs in robots and in virtual worlds, try and teach them ethics the same way you teach a human child ethics. The probability of success at that we really cannot estimate rigorously right now.
At the 2006 AGI workshop, Hugo de Garis made a comment that was fairly apropos. “The initial condition of the superhuman AI will determine its ongoing evolution… initially.” The other comment Hugo made in that workshop, which I have not gotten him to replicate–the question was asked, “When a superhuman AI has reached godlike powers, how is it going to respect human desires?” And Hugo said, “There’s a fly on my arm!”
I’m not saying that is necessarily the conclusion, but it is something to think about.
On the other hand, Natasha‘s partner Max More has pointed out something called the Proactionary Principle. Building AGI has dangers and risks, but the human race has a lot of other problems, and not building AGIs also has significant dangers and risks. As Stan pointed out in his concluding remarks, there is a large intersection between what we need to do to build AGI and what we need to do just to understand how the mind works. Understanding the mind has a big risk–we may build an AGI that does bad things. Not understanding the mind has very bad risks, which we see in human society all around us. It would just get worse and worse as technology develops.
There are no easy answers. There are a lot of scenarios that need to be explored. We do not understand any of these issues very well, but at least it is getting easier to talk about them together.
