The Notion of a Technological Singularity
Posted by Jeriaska on September 5th, 2007
SIAI Interview Series – Ben Goertzel
The following transcript of the SIAI Interview with Ben Goertzel has not been approved by the author. Video and audio can be found online at the Singularity Institute website.
The Notion of a Technological Singularity
I am Dr. Ben Goertzel. I’m an artificial intelligence researcher, and have been in this industry for the past ten years, and in this field as an academic for nine years previously. I have worked on AI from a variety of perspectives: mathematics, cognitive science, computer science, software engineering, theoretical and applied AI. And my involvement with the Singularity Institute came about because I became convinced that there are really fundamental issues regarding artificial intelligence that are just not being addressed anywhere in the world right now. Things that are critical to the future of AI research, critical to the future of humanity, and the future of the mind in this corner of the universe. Almost no one is thinking about these things at all. And the people who are thinking about these things just don’t have time to really bear down on the questions, and give them the attention they deserve.
The mission of the Singularity Institute is to ensure that advanced artificial general intelligence, when created, is positive. To ensure that the Singularity is a positive one, and that AI’s and humans can advance together beyond the current human condition into a mutually beneficial future. I believed for a long time, since well before Vernor Vinge and Ray Kurzweil popularized the ideas, in the notion of a technological singularity. I’m 40 years-old, and I’ve seen so much change in my lifetime. It’s been qualitatively apparent to me that the pace of change is accelerating. You can play with graphs and charts in various ways, which is interesting and is worthwhile, but qualitatively and intuitively, it seems clear to me that we are on the verge of some pretty dramatic technologies that are just going to change everything. In terms of science, in terms of technology, in terms of everyday life and what it means to be a mind and interact with other minds.
What technologies am I taking about? We’ve got nanotechnology. We’ve got biotechnology, including things like genetic engineering and neural modification. We have quantum computing, which I am pretty excited about. And we have artificial intelligence. All these things are really critical. Achievement in any one of these domains is going to accelerate achievement in the other domains. But I feel that artificial intelligence is in some ways the most powerful of any of these single enabling technologies, which is the reason why I have devoted my life to AI research in the first place. And AI, unlike all these other technologies, as exciting as they are, really has the potential to bring us beyond the scope of the human mind. Even if you have nanotechnology, the scope of what you can create is still limited by what you can think of. And life extension certainly solves a lot. It will eliminate a tremendous amount of human suffering and will eliminate a lot of just plain inefficiency in wastage of knowledge. With human life as it is now, as soon as you know what you’re doing your brain starts to decay. And that shouldn’t be the way it is. What if Einstein was still thinking about physics today?
But you’re still within the scope of the human with these other technologies. When you talk about artificial intelligence, you’re talking about the possibility of going as far beyond humanity as humanity has gone beyond mice, cockroaches, or bacteria. And that’s a pretty amazing and exciting thing with tremendous possibility for good, and tremendous possibility for danger. And not many people seem to realize that or take this prospect seriously. We have science fiction. We have The Terminator, various works of fiction that explore the good side, and more often the dark side, of artificial intelligence. And we do have university departments of AI, industry labs of AI, but no one seems to be getting at the crux of the AI problem. How do you make a general intelligence? How do you engineer a computer program that has the same type of autonomy as your general learning ability as a human. That’s a really important question. The other important question that I see is, once you’ve created a general intelligence, how do you guarantee that it does things that we, as its human creators, think are good things that it should be doing, and on the other hand, avoids doing bad things that we think it shouldn’t be doing? For example, annihilating all of us, or enslaving all of us, or doing any of the other bad things that have been portrayed so vividly in science fiction?
Each of these two questions is critical. How do you build it? How do you guide it appropriately? These are both very important questions. Almost no attention is being paid to either one of these questions. Now, in my day job in Novamente LLC, which is my AI company, we are working on the first of these questions. We are trying to build a thinking machine, according to a design that I conceived in my years as an academic. On the other hand, we have constraints. We’re a small commercial company and it’s all we can do to build a machine. Trying to understand how to guide the machine to make sure its behavior is ethically positive, it’s not that we don’t care about it at Novamente. But it’s not a major focus of our effort, because our focus is on building it. And what I have seen among the other small groups that I know of, working on trying to build an AI, is something similar. Building an AI is hard enough, and the ethical issues, the issues of the stability of the system as it improves itself over time, these issues inevitably get shorter shrift than you would like. And I think this is a role that a non-profit organization should play. Non-profits can afford to take a slightly longer timescale than a for-profit company. And they can afford to go a little further out there than a typical university department, which is typically held in thrall to relatively conservative government grant funding agencies. As a non-profit you can really focus on what is the most core issue. And the issue that I would see the Singularity Institute focusing on in our research program going forward is basically, once you have created an AI, how do you ensure that that AI maintains its initial goal system as it evolves over time?
And this is a hard problem. Of course it interacts very deeply and intricately with the first question that I mentioned. How do you build the AI? There may be some ways of building a human-level AI that just can’t be controlled by their very nature. Maybe they are just intrinsically spontaneously self-organizing, and they’ll just go in whatever direction they want to. And if that direction happens to be repurposing all our molecules to give themselves more RAM, then, too bad for us. There might be other ways of building a human-level AI that are stable and controllable, and will maintain their initial goal system as they revise and improve themselves. These seem to be very, very important questions. We want to know how to direct AI research in a way that will lead to beneficial outcomes. But who’s thinking about that? What I see myself doing as an AI researcher is thinking about that in my spare time. It’s something you think about on the weekend when you are walking in the mountains. There should be a team of AI professionals who do nothing everyday but try to understand how to make AI that is stable, safe, and beneficial for humanity.
As an initial step toward achieving the goals of the Singularity Institute research program, I would like to make two hires at the postdoctoral fellow level. One of which would be in the domain of experimental AI, and the other in the domain of theoretical AI. I would like to have someone work under my direction to implement the simple self-modifying AI system, with a mutable goal system, that learns through experience. The initial system I would build would be based on an AI methodology called MOSES, which stands for Meta Optimizing Semantic Evolutionary Search. This is something that Dr. Moshe Looks and I have worked out over the last few years. It is a component of our Novamente AI system, but it is in the public domain and is the subject of Moshe Looks’ PhD thesis at Washington University, which was granted in 2006 in the computer science department. We figured out how we can use the MOSES learning algorithm to make a relatively simple prototype case for a self-modifying AI system. Basically what we need is to pay someone to spend a year building that so that we can experiment with it under the direction of Dr. Looks and myself.
The second hire that we would like to make is in the theoretical domain, addressing the issues of how to take statistical decision theory, the mathematical theory of general intelligence, as developed by Marcus Hutter, Jürgen Schmidhuber, and others. And kind of downscale it so that it tells you something about AI’s run on a tractable computational infrastructure. This is a hard problem. We certainly can’t guarantee the solution will be found within six months or a year of working on it. On the other hand, we could have a breakthrough within three or four months of having the right person focus 100% of their time on the problem. But what I’m absolutely certain of is that this research needs to be done. We need to gain an understanding of these issues, and I think having these two post-docs work together can be very beneficial. Because we can wind up using the theory to understand what the experimental system is doing, and playing with the experimental system will generate hypotheses that will then guide the theory. And this kind of give and take is really going to be critical, and I think will be the seed that will tell us where to take things next.
Artificial general intelligence, the science of advanced AI pushing toward human-level intelligence and beyond, as you might expect, is a rather large undertaking. Around the world now we have hundreds of departments of physics, astrophysics, astronomy, entomology, ichthyology, on and on and on. If you look in AI, almost all the work going on in AI is in what Kurzweil has called “narrow AI.” People are making algorithms that solve very specific, narrow problem, which do require intelligence to solve, but they don’t require a general learning ability. They don’t require an understanding of self. They don’t require the ability to orient oneself in the world and grapple with new problems. Artificial general intelligence research right now is being carried out in a very scattered way. There are handfuls of people working on it at various universities, start-up companies, research labs and launch companies. But we’re talking dozens of people really actively devoted to the problem.
We have outlined a pretty ambitious research program for the Singularity Institute, which covers eight different areas, with a number of different sub-areas underlying each one of them. You can look on our website for the details of that. But we are covering areas such as the various general intelligence designs out there and looking at to what extent they can be synthesized. What is a common vocabulary you can use to discuss and compare them? Now, what’s the right IQ test for a general intelligence? What kind of simulation world, what kind of environment do you want to make for general intelligences to live in, and interact with each other and have their intelligence assessed? What sorts of mathematical theories do you need for general intelligence specifically, as opposed to narrow AI? A lot of that comes down to applications of probability theory, which is critical to narrow AI’s practiced in modern academia. But also critical to general intelligence and in different ways.
My firm belief that within the next three decades at most, and maybe much sooner than that, someone is going to create an artificial intelligence with a capability greater than that of the human mind. Not just greater than that of the human mind, but as much smarter than us as we are smarter than the cockroach. Now, this is an exciting thing and it’s a frightening thing. If you believe this is going to happen, if you believe this has even a 1% chance of happening, it makes a lot of sense to put a lot of effort into ensuring that it happens in the right way. I’m not an optimist enough to believe we can guarantee the outcome of the Singularity. There is an irreducible uncertainty in creating something more powerful than yourself. But nor am I a fatalist, or a pessimist enough, to believe that we have no control over guiding the outcome.
The reason to donate to the Singularity Institute is that we can make a difference in the outcome. We can make a difference in whether artificial general intelligence, when it happens, is positive, leading us all toward a beneficial and better future beyond the limitations of human life as we know it. Or whether it’s a negative outcome, which makes life worse for everyone, or in the worst case, just gets rid of us altogether. The chance to have an influence on the outcome of the Singularity, this is all we can ask for. There are a number of ways any of us can have an influence on the outcome of the Singularity. We can have an influence through our own work. We can have an influence through talking to people, helping to spread understanding. But one of the most highly leveraged ways that any of us can influence the outcome of the Singularity in a positive direction is by helping the Singularity Institute carry out its mission, which is narrowly focused on ensuring that the development of powerful artificial intelligence is done with a positive Singularity in mind.
