Nine Years to a Positive Singularity – If We Really, Really Try
Posted by Jeriaska on November 24th, 2007
Ben Goertzel is Director of Research for the Singularity Institute for Artificial Intelligence, responsible for overseeing the direction of the organization’s research division. He has over 70 publications, concentrating on cognitive science and AI, including Chaotic Logic, Creating Internet Intelligence, Artificial General Intelligence (edited with Cassio Pennachin), and The Hidden Pattern, and is the chief science officer and acting CEO of Novamente, a software company aimed at creating applications in the area of natural language question-answering. At the 2007 Singularity Summit, he discussed the current prototype work involved in the release of intelligent agents controlled by the Novamente AI Engine in Second Life and other virtual worlds.
The following transcript of Ben Goertzel’s 2007 Singularity Summit presentation “Nine Years to a Positive Singularity – If We Really, Really Try” has not been approved by the author. An audio recording is also available at the Singularity Institute website.
Nine Years to a Positive Singularity – If We Really, Really Try
Following up on a number of the previous talks, I’m going to give a talk about artificial general intelligence. This was the original title of the talk, which I chose because about a year ago called “Ten Years to the Singularity.” I didn’t want anyone to accuse me of being a liar. The thing is, it was “Ten Years to the Singularity If We Really Try.” Now it’s nine years, if we really, really try. The title’s going to get mutated year after year.
I still believe this is a reasonable order of magnitude estimate in the sense that if in an incredible amount of effort by the right people was put toward developing general intelligence we could get there. Why I believe that, I’m going to at least allude to during the rest of this talk. Of course, I’m not going to go into full technical details on the various research aspects that I mention.
A more in depth title of what I’m going to talk about is “Artificial Intelligence in Virtual Worlds.” I’m going to talk a bit about AI in general, but a lot of that has been covered by the previous speakers. So most of the talk will be devoted to why I think embedding artificial general intelligences in embodied agents in online virtual worlds is sort of the golden path to achieving AGI and bringing about a positive Singularity.
What do I mean by “artificial general intelligence”? A number of speakers have gone over this already, and I’m not going to go into too much depth on it. This is a term that’s come to the fore really only in the last few years, but it’s of course a very old concept. The idea of the AI field from the very beginning, back in the middle of the last century, was to create a real thinking machine, something at the human level of intelligence, and then beyond the human level of intelligence.
What happened historically, as a number of other speakers have outlined, is most of the AI field sort of went in a different direction, working on highly task-specific, narrow applications, which, as Neil has outlined, have in many cases been extremely valuable for the real world, so I wouldn’t want to put them down. But I think it’s useful to distinguish general intelligence from these sort of task-specific narrow AI systems.
What I mean by an “artificial general intelligence” is a system that can achieve a variety of complex goals in a variety of complex situations. Defining that formally and mathematically is a big job which I tried to do some time ago, and Marcus Hutter, Jürgen Schmidhuber, and other researchers have done with more precision in recent publications. Qualitatively, this comes together with a lot of other things, such as the ability of a system to understand what it is, to understand that it is as distinguished from the rest of the world, o understand there are other beings out there interacting with it, and to be able to take a situation in an environment and translate that into a problem that it has to solve, which is quite different than how most of the AI applications systems that we see around us work.
We look at what Ray Kurzweil has called “narrow AI” in some of his writings, and you have systems like many of the ones that Neil Jacobstein has described recently. And like a lot of systems that I’ve built in my career over the last ten years doing AI consulting work for various businesses and government agencies, these are systems that solve particular problems. In some cases they solve them very well, much better than any human being can solve them. They solve problems that we would qualitatively classify as requiring a lot of intelligence. These things are a lot of fun to work on. They’re cool, they’re valuable. They’re pretty different from creating a thinking machine that thinks in the same sense that humans do.
Some examples of narrow AI, most of which have been discusses already by our other speakers: Deep Blue. It’s a great chess program. It’s a heck of a lot better than me. It’s probably better than anyone here. What it can’t do, it can’t play go. It can’t play checkers. It can’t generalize the strategic knowledge you gain from learning to play chess. Without some tweaks to it, it would probably do terribly at fish or random chess where you permute the pieces on the back row in a different order. So there’s a lack of a capability to generalize: conceptualize what’s learned in one context and transfer it into another context.
The DARPA Grand Challenge is a lot of fun. I’m eager to have the other drivers on the road replaced by robots. Still, the nature of the work is that you’re really hyper-engineering things to the particular application. It’s not like Sebastian Thrun‘s outstanding car-driving software can drive a motorcycle. Yet, if you take someone who’s learned to drive a car and put them on a motorcycle, some of them will fall off, but a lot of the car-driving knowledge does transfer. The knowledge inside the DARPA Grand Challenge control systems isn’t really represented and configured in a transferable, generalizable way, because these are not AGI systems.
Google, I use it many times a day. I think it’s a great system. It can’t answer complex questions. It’s interesting, if you type in some simple questions it actually will answer them for you. You ask how long does a giraffe live, you’ll probably get the answer, maybe even in the snippets within the first couple questions. But if you ask it “How long does a giraffe not live?” you’re not going to get the answer. It doesn’t really understand what’s going on.
Barney Pell has outlined technology that will move in the direction of a real understanding of questions, which is the Powerset technology. I think that is an interesting case of something building a bridge between narrow AI and AGI, where if you make natural language processing software that gets more and more semantics incrementally, can answer more and more questions, you’re getting something that has more and more real understanding, but I still think there’s a long way to go between something like Powerset and something with the human ability to really resolve ambiguous semantics and interpret language in context. I’ll talk a bit about that a little later.
In my own work over the last ten years that I’ve been doing AI stuff in industry, I’ve done an awful lot of narrow AI stuff too. We made a system called RelEx, which is a lot like Powerset’s parser that it got from PARC. It takes sentences, outputs semantic structures, and interprets them. We did this in a government contract. The system was useful. We’ve done a bunch of biology work, which analyzes gene expression data, single nucleotide polymorphism data, tries to recognize patterns, so you can ask it a question like, “What genetically distinguishes people who get chronic fatigue syndrome from others with a similar profile who don’t get it?”
This stuff is very important. I’m proud to have had the chance to be involved in it. On the other hand, you’ve got to do a lot of work to prepare the data to feed into that system. You read papers, you figure out what the important questions are, you pre-process the data, then you run some code, and then you get an answer out, which you then interpret by reading a bunch of research papers and then you write a paper. The AI is doing something really important, but it’s doing just a tiny bit of the overall research process, and it doesn’t understand what it’s doing. It’s analyzing numbers in the same way it would if the numbers came from an oil drilling application or anything else.
This is a graph that came out of some of our biology work. This was actually about mice under calorie restriction. We fed a bunch of gene expression data from mice under calorie restriction and mice not under calorie restriction, and you ask it what distinguishes the two genetically. This is a graph of the gene interactions that distinguish the two. You can see interesting stuff. You see a gene Mrpl12 up there, mitochondrial ribosomal protein 12. It’s a hub of a little network there. That gene is probably pretty important for why calorie restriction extends life. This is not an AGI system; it’s a narrow AI system. It spits out this diagram. Now it’s my problem or some biologist’s problem to figure out what it means, why it’s important, what experiments do you do, and so forth.
I’ve been undergoing the effort over the last few years to spread the meme of AGI throughout the AI research community. One of the things we did was organize a workshop in Bethesda in 2006, where we had maybe 45 people there, including a number of AI researchers and a number of individuals from the futurist community, which resulted in a book that came out this year Advances in Artificial General Intelligence from IOS Press. Following up on that, we are organizing a conference which will be in March in Memphis, Tennessee at the FedEx Institute and University of Memphis. So, I’m giving a quick pitch for this conference, which is AGI-08, the first conference on artificial general intelligence. It is in large part a hardcore technical AGI research conference, but a lot of the themes there are of pretty broad interest to everyone in the audience who is concerned with AI and what it may mean as the future unfolds. I’m excited about this from the perspective of getting more and more of the research community focusing their minds on the big question of how do you make a thinking machine, as well as the very valuable and interesting questions of building narrow AI systems.
Onto the meat of my presentation. How do you create an AGI? Of course the full answer to that is very technical and I couldn’t tell you in ten minutes even if you were all five times smarter than anyone on the planet. I still can’t. I can tell you first of all how I think you can’t create AGI. I don’t believe you can do it by generalizing narrow AI applications. I think this is a nontrivial observation. It’s a lesson that’s come out of the last n decades of AI research, because it wasn’t obvious from the beginning that working on theorem-proving, chess-playing, natural language search and car-driving wouldn’t be the golden path to AGI. At first it seemed that it would be, but what we’ve seen now is you can build specialized programs to do all these tricks, but it doesn’t tell you much of anything about how to make a thinking machine.
I think the correct answer in terms of how to make a thinking machine at the level of high-level methodology goes back to what Sam Adams was talking about earlier. You’ve got to make an artificial baby… make pets, make bugs, make dogs, make birds, chimpanzees, infants that are a complete artificial idiots and can’t do anything except, hopefully, amuse the person who’s teaching them and playing with them. A baby is one of the stupidest things I’ve ever seen. I’ve had three children and they’re all highly gifted and quite intelligent by this point. When they start out, they just kind of lay there and defecate, urinate, scream and waive their arms around. Part of the maturation process is just the unfolding of things that are preprogrammed in the genome, but a large part of it is learning.
We need a system that can start from a relatively ignorant state and actually learn how to do things. It doesn’t start out knowing how to analyze genetics data or knowing how to drive a car. Let’s say you take this idea and come up with an obvious question of where does the AI baby live. Is it a robot? Is it a search engine? I think building robots is a fundamentally good approach, but in practice it’s a pain to build robots. It’s expensive. Sensor and actuator problems wind up taking up most of your time.
The approach that we have settled on is experimenting with embodied agents in virtual worlds. We are looking at a game engine, Crystal Space, that we’ve customized. This is a dog in Second Life, which is not something that we have built, but it’s something that exists there right now. It’s not a very smart system; it’s a simple expert system. But it’s an artificial agent that can perceive, it interacts. We have artificial birds, even artificial babies. This was actually the tamest baby we could find in Second Life, which tells you something about the Second Life community. More interesting virtual babies, you’re not tied to the humanoid form, along with all kinds of creatures. I feel working in virtual worlds gives you the ability to make systems that are perceiving, acting, communicating language embedded in a social network. Yet there are big advantages over working over physical robots, one of which, from out point-of-view is we have a distributed software team with people all over the world and they don’t all have to have robots in their labs and their home offices. They can collaborate on the same system.
Another really interesting advantage is the ability to roll out your virtual robots at low costs, potentially to hundreds, thousands or millions of users. If you create virtual babies and put them in virtual worlds online in Second Life, in Club Penguin and various videogames, and you roll out virtual babies in this way, if they’re amusing enough to play with, potentially you can get a huge mass of people teaching these babies things. That’s an advantage that AI just has not had in the past, having all these teachers. You look at why might AI be achievable now when it wasn’t achievable ten years ago, of course there are a number of reasons. There is the brain emulation argument, which isn’t the path I’m going down. There is just the fact that we have understood so much more of human cognition and cognitive psychology by now. There are the mathematical and AI tools we have developed, probability theory, evolutionary learning, formal logic. But there is also the fact that now via the internet we have the ability to harness the wisdom of crowds. A very large number of people teaching your AI system all at the same time, where all of the knowledge it gains from all these people teaching it can be stored in a centralized place where it can all be reasoned on. This is to me the most beautiful thing about the whole virtual worlds approach.
Of course, all that is just methodology. With that setup, unless you have the right design for a thinking machine, you’re still not going to get anywhere. Just like a human body is nicely embodied, it has good censors and actuators, but if you put ELIZA or Deep Blue inside a human head, it’s still not going to be an AGI system. So my approach to the artificial general intelligence problem on a technical level is something I call the Novamente Cognition Engine. This is a long story that I’m not going to be able to go into in the few minutes that are left to me her. I’ll say just a little about it that may be evocative to people who have some familiarity with the AI field already.
The theory of mind underlying the Novamente AI system in itself is something that I worked on for about ten years when I was an academic, before entering the software industry. A summary of that is in a book called The Hidden Pattern that I published last year, which elaborates a framework for understanding the mind, which views the mind as a pattern-recognizing system for recognizing patterns in the world and in itself embodies the patterns it’s recognized in itself and then recognizing patterns within those patterns it’s recognized. What I do in this book is basically go through all the different aspects of intelligence that are known by cognitive psychology – perception, action, reasoning, learning, memory, emotion, will, and so forth – and try to explain them in a pattern recognition and formation framework, a kind of grand unified vocabulary for understanding the mind. This doesn’t tell you how to make a thinking machine, but it gives you a conceptual framework within which you can flesh out various AI designs.
The Novamente AI design itself, just to go through some buzzwords describing it, it’s not going to be much more than that. We use a knowledge representation which is a mathematical structure called a weighted labeled hypergraph, which less formally means it’s a bunch of nodes and links that are connected together. Some of the nodes refer to concepts that would have an English-language label. Most of the nodes don’t, they’re just abstract stuff. Some of the nodes refer to percepts, some can refer to actions. Acting on this collection of nodes and links, there are a bunch of cognitive processe, which we call mind agents. The mind agents embody a bunch of different things. The key learning algorithms are something they call MOSES, which is an improvement of genetic programming utilizing probability theory to learn more effectively, and something we call probabilistic logic networks, which I believe is the first system to fully integrate symbolic logic with probability theory.
These are the key learning algorithms. They work on the node and link knowledge representation and they are embedded in a cognitive architecture, which is largely inspired by the human cognitive architecture. This is a block diagram that depicts a cognitive architecture at a very high level. I really don’t think these diagrams tell you very much. What’s important is what happens inside the boxes, and most of all, how the things inside each box really interact with the things inside the other boxes.
In one of the talk earlier, we saw a similar diagram from Stan Franklin‘s IDA architecture. In fact, I’ve gone through in detail how you map my boxes into Stan Franklin’s boxes, where we’re really saying the same thing. I break down mine into perception, action, language, short-term memory, long-term memory and the various parts that cognitive psychology has understood as being essential to how thought happens. Each of these boxes is filled in, in a certain way, by a combination of our core learning algorithms, which is the MOSES program learning approach and probabilistic logic networks. It can be a little misleading when logic is mentioned, because that makes eome people think of old fashioned AI systems. Although we use logic, the system is perceiving things through censors and is controlling things through actuators. So, it’s not acting like a theorem prover. It’s trying to understand the world as the world is presenting it to itself through its senses, and understanding how to act.
For those who are interested in more fully understanding the Novamente approach, you can look at Novamente.Net and there’s some conference papers on there. To very quickly roll through a few other things, we’re looking at potentially within the Singularity Institute rolling out a framework called Open Cog. This would be an open source approach to general intelligence. I would take some of the components we have developed in the Novamente system, which is a proprietary system, clean them up a bit, release the knowledge representation and some of the key learning algorithms open source, not the framework for controlling agents in simulation worlds. The idea would be to give a kind of playground that AI researchers within the community could use to develop their AI algorithms in a more AGI-oriented way, so the different AI algorithms can interact with each other and be more fully fleshed out. Another interesting avenue is to release AI components in a way that can run on a variety of machines on all of your desktops, which is a potential Singularity Institute initiative we call AGI @ Home.
I’ll wrap up by giving a pictorial overview of the ladder of increasing intelligence as we’re looking at it. We can start off with virtual pets, virtual ambient animals running around in simulation worlds, which you can train and which interact with you. Then move on to something like virtual parrots that can talk and learn more language by having hundreds of thousands of people interact with them and teach them. Move on to virtual babies: human babies, alien babies, virtual human beings. Beyond there, we can move on to virtual superhumans, virtual superhumans plugged in with AI’s, hybrids of course. Finally, what’s the end goal? Of course the end goal is something vastly transhuman. That’s a graphic I got from the online textual virtual world Orion’s Arm. This is supposed to be a superhuman god cluster.
This finally leads on to the ethical issues involved, which I’ve left to the very end so I have no time to talk about them. Don’t worry about it, leave it all to us. More seriously, I need to give at least thirty seconds to ethical issues. I think it’s very easy to think in a cavalier way about how cool it would be to create a superhuman god intelligence. Of course, it would be. I would consider that very fitting as my life’s work. On the other hand, there are major dangers involved here, which you can’t shrug at either. I find it’s easier to shrug off the human race, in some ways, than to shrug off yourself and all the people that you know and love, which is a well known cognitive error. We find it hard to conceive of a catastrophe on a really massive level, yet if you think about each individual person who could be threatened if AI went the wrong way, it becomes more amenable to our empathic system to understand.
I think about my own family. Do I want my kids and my wife to be wiped out by an AI program that I create? I don’t even want one of them to be run over by a car, let alone have all of them wiped out by an AI program. This leads into some very difficult problems. How do you make an AGI system so that it’s not only intelligent but, as it makes itself smarter and smarter and ascends to the superhuman god level, it doesn’t lose its original goal system, which presumably includes being fairly nice to us measly little hypercerebral monkeys. If you’ve figured out how to do this, how do you ensure that’s the AGI that gets built instead of some other AGI that someone else created which isn’t as friendly? These are very hard problems and are among the reasons that the Singularity Institute for AI, which has put on this conference, exists.
In my work at Novamente, building a proprietary AGI system, we’re pushing hard to create our own thinking machine. Being a start-up company strained for resources, we don’t often have time to worry about these pesky little questions of how to ensure that your AGI is really nice to people once it ascends to the superhuman level. But there are really no more important questions for people to be thinking about right now, and I’m very glad that the Singularity Institute exists to focus on these things.
Those of you who live in this area may be interested to go to the Virtual Worlds Conference which is in San Jose on October 10th, where we hope to be giving a preview of a Novamente-controlled virtual agent in Second Life. The AGI-08 Conference in Memphis shouldn’t be forgotten either.
