Fully Integrated Intelligent Systems
Posted by Jeriaska on September 18th, 2007
SIAI Interview Series – Barney Pell
The following transcript of the SIAI Interview with Barney Pell has not been approved by the author. Video and audio can be found online at the Singularity Institute website.
Fully Integrated Intelligent Systems
I’m Barney Pell. I’m currently the founder and CEO of Powerset, a company using advanced artificial intelligence to bring a new kind of search to the world that will let users use natural language to do search, instead of having to speak in terms that are easier for computers to understand. We have not created artificial general intelligence, yet. We are working on very early stages, but we are taking advantage of some real breakthroughs in AI that have been thirty years in the making.
I think, in terms of predicting the future, it’s very difficult. The things we can look at are trends. There has been really steady progress in terms of artificial intelligence, a lot of which is Moore’s law. Back in the early days when people were working on AI, you had to spend all your time just working to squeeze anything into small amounts of memory and computing power. So, AI has become much more enabled by advances. A good example is computer chess. So, when people first got together, it was one of the earliest problems in AI, and people said we will have a computer chess program that will beat a human world champion within maybe five years. And then those five years came with very little progress, and this went on for probably 25 to 30 years. After people were almost giving up on it, then finally things caught up. Science advanced and Moore’s law advanced, and we had programs that beat human world champions. So, some of these are very hard problems, they take a long time, and what we do expect is that technology will advance.
Specifically, for the Singularity, it’s very hard to say. I would say, I think within a hundred years, it’s a pretty safe bet that we will have computers that can work at human-level capability, absorbing information in the world and processing it faster than we are, and it’s a matter of us just trying to keep up. I’m an advisor to the Singularity Institute for Artificial Intelligence. SIAI is really a non-profit organization that is about looking to the future to the longer-term developments in artificial intelligence and what it will actually mean when it gets there for affecting the world. So there are some really interesting issues. We know from technology advances, people have their own interests applied for various academic, intellectual, economic reasons. But technology gets there, and often what happens is we then come to realize the consequences. SIAI is about looking ahead and trying to think deliberately abou, even though things are way in the future, the future will come faster than anybody knows, looking ahead and saying, “What are the issues we need to worry about now?” Maybe there are some things we need to do to prepare for those issues, or maybe there are ways we need to design these artificial intelligence systems to take into account, so they will be the right kinds of AI in the future, instead of potentially the wrong kinds of AI.
So, a lot of the things we are talking about here almost enter the realm of science fiction. The funny thing is that things that were science fiction ten to twenty years ago now become real, so we’re looking at those kinds of scenarios. The thing we are talking about here are maybe scenarios from like The Terminator. What happens when you get an artificial intelligence system that is as smart or smarter than people, how is it going to relate to people? What if it has its own goals? Maybe it goes out of control. One example is a system that is actually destructive and just wants to conquer the world. These are the sort of dark side of artificial intelligence in science fiction. Maybe it just views humans as a threat and, like a territorial human would, wants to go and basically take over everything. So, what happens when you have these incredibly powerful intelligent systems, that maybe are smarter than we are, out there with territorial instincts built in going and taking things over and potentially killing everyone? That would be a bad thing.
Now, there’s no reason that an intelligent system need be territorial. A lot of people think this is just built into our primate development. A lot of animals aren’t particularly territorial. It’s not necessarily required for intelligence. By realizing what is the architecture of an intelligent mind, and there’s a whole space of minds that could be designed, which ones are we trying to design? Should we build in certain kinds of negative emotions, or only the good ones? Can you have the good without the bad? These are the kinds of issues to be explored no, and it effects what kind of mental architectures we can create.
These issues are definitely far out in the future. But, as we know from technology advances, the future comes before anyone really thought it would come. In particular, we have this concept of accelerating change, which is promoted by Ray Kurzweil, where things happen faster and faster. So, computers get faster than they used to get faster. How far away is this future? We don’t really know. So, we could say we’ll deal with it when we’re close to creating these artificial intelligence systems out there, these Strong AI’s that come about. But maybe if we worry about it then, that will be fine. The problem is, then, certain choices may have already been made. So, the interesting thing is to think, in advance, are there issues we should be considering now while these things are still being designed before it possibly becomes too late? Because if we get to almost this point, imagine we’re very close, five years away from these systems becoming strongly intelligent, there might be decisions set in place before anyone has had a chance to respond to the consequences.
The Singularity is the idea that there is a singular point in the development of human history where computers catch up to be as intelligent as people are. And since the rate of progress of electronics is faster than the rate of progress of biological systems, there is a point where computers match people, at which things are irreversibly altered. Computers then keep on improving faster and people potentially stay with our own cognitive architecture and we don’t get to improve nearly as fast. So, everything starts changing fast. New levels of information become available and what kinds of minds are able to connect the dots? You wing up with this collapsing point where computers can keep up with information and people are not able to keep up as much anymore, and the world just goes really fast past that. The only people who can then keep up with the rate of change and information are actually the computers. That’s the concept of the Singularity.
So, in terms of my background, I’ve actually been interested in artificial intelligence for a very long time: really, ever since childhood. While I was an undergrad at Stanford, I majored in symbolic systems, which was all about the convergence of philosophy, psychology, linguistics and computer science, as relates to understanding what will become intelligent systems. I worked, even as an undergraduate, on natural language understanding at one of the world’s great artificial intelligence labs. Then I went and did a PhD in computer science and AI at Cambridge University, where again I was addressing the problems of artificial intelligence and how could machines learn from their experience and take in all kinds of new problems that have not been seen before, to figure out strategies and ways to cope. Then I went to NASA, and at NASA, I worked on artificial intelligence as applied to intelligent software agents that could potentially create and control autonomous spacecraft. So, we actually pulled together advanced intellectual capabilities out of the research labs, in terms of planning and scheduling, diagnosis, multi-thread execution, monitoring and control. And we put all of these things together into the first autonomous software spacecraft architecture, and we flew this onboard a spacecraft. We demonstrated that people could send up high level goals to the system, and it could then take those goals and plan its course of action to achieve them. And then it could actually track what was going on, and if there were problems, then it would diagnose those problems, try to repair them and then replan and recover with this new state of the world to go on achieving the mission. So, this was a way to free NASA to send spacecraft out to explore the universe without having to necessarily be essentially tethered on short-term communication lines back to earth, and create a whole new level of resilient systems. This work was partly the inspiration for the Mars exploration rover mission, where we had rovers operating, still not as autonomously as we had developed, but much more autonomous than they could before. So they could respond to their own problems, navigate around obstacles, and this kind of thing.
I have been involved in several start-up companies since leaving NASA. Currently, I am the founder and CEO of a company called Powerset. We are deploying artificial intelligence breakthroughs in natural language understanding. Our system reads all the documents on the web and converts them into a semantic representation, gaining an understanding of the meaning of these documents. And then, at search time, a user can come and express their interest and query in natural language, and the system then extracts the meaning out of that, and then does a match between the meaning of the users query and the meaning of all the content and information the system has, to deliver a whole new kind of search. So, we are actually bringing natural language and advanced artificial intelligence capabilities right into search, that will affect everyone’s lives. It will be really the first time that we are used to interacting with an intelligent system. This is still in the early days. We are exploiting thirty years worth of progress in the field of artificial intelligence. But we can already see in what we’re doing that once a system can read documents, it can start acquiring more knowledge directly from those documents. And it gives the possibility of various kinds of chain reactions where the system knows a little, is enabled to learn a little more, and keeps on going until potentially the system has a very rich base of knowledge with which to interact with the world.
In order to achieve true artificial intelligence, there are advanced required on many different fronts of technology. One area that people have worked on that I’m particularly interested in is natural language understanding. This is essentially endowing machines with the ability to reason about language: to read things, to communicat, to understand, and to relate, and to have discourse and dialog with other people and other intelligent machines. There are actually many other capabilities required. For example, vision. Visual processing abilities that humans have to be able to detect objects and project the consequences of things moving through space. These kind of things, we still don’t have anything like that in computers. And, you may have systems that have knowledge of the world, and of what things maybe mean, but until they can be grounded in perceptual experience you won’t have a truly intelligent system. There are many, many fronts. Some people believe that ultimately you have to have a body, that can engage with the world, perceiving things, forming its own associations and then engaging in a society, before you can truly have an intelligent system. So, I think it’s not going to come from just one particular problem being solved. It’s going to come from a whole architecture that integrates the different aspects of intelligence and then is able to go and learn from its experience engaging in the world.
This is not just about intelligent tools, like a refrigerator that would order milk. This is about fully integrated intelligent systems. This is about systems that might have their own bodies, maybe walking, talking robots that are physically as fully capable as we are inhabiting our world and perceiving things as intelligently or more intelligently than we are. That’s the world we’re talking about here, where they are not just little tools that are servants to us in some ways, but they are agents that have their own volitions, their own needs, their own drives, and are potentially existing with us as equals or, who knows, maybe our superiors. My role with the Singularity Institute is as an advisor. I think that what they are doing is great. I think they are serving as a collecting point for the set of people who really want to think about, not just the short-term issues in artificial intelligence and not just one application or another, but really the fundamental long-term issues about what it’s going to mean when this field actually succeeds and these developments come about. I’m really happy to support people who are taking the visionary perspective. It’s a small organization and they have had some really brilliant and dedicated people, and my role really is to be part of the discourse, help people to understand the value, be an evangelist, and bring other interesting people into the fold.
Given that I am an expert in artificial intelligence myself and my company Powerset is really pushing the edge of artificial intelligence, we are in this world where people are thinking about what the future will be with these intelligent machines. That generates a lot of interest in people wanting to collect the thought leaders and find out how I can be involved. So, it’s valuable to have an organization like the Singularity Institute that I can channel people into to help create this ecosystem of people interested in really fundamental issues. I think a lot of the world is driven by ideas, and if you are not thinking about an idea and go about doing your daily work, potentially you can do that work in isolation. Once in awhile, people come along who bring an idea to the forefront. And once you are aware of that ide, everything you do becomes shaped by it, consciously or unconsciously. I think that’s what the Singularity Institute is doing here. They are really bringing out this notion of the Singularity and the fact that the future of artificial intelligence is happening much faster than people would have thought. We really need to be thinking about it. The consequences really matter. Just the idea of being out there, and being exposed to larger and larger communities, means that in our own daily work, we’ll take that idea into account. That one idea can shift the balance between whether things happen in a disparate, ineffective, and potentially adverse way or whether they are handled in a thoughtful, coordinated and proactive way to achieve the positive future we would all like to have happen.
From a funding perspective, there are a whole variety of causes you can contribute to. Many of them have directly immediate consequences. We take this action, and then we see this outcome happen today. And many of them will affect a number of individuals in the next year. And that’s how you measure your results, by the number of individual’s whose lives were affected. I think the Singularity Institute is really addressing an entirely different perspective on investing. This is on the long-term future of humanity and the long-term future of technology. But the fact is, the long-term is created every single day and is being created right now. So, by participating, investing in, and supporting the Singularity Institute, you can really rest assured that you are having an impact on the eventual future of humanity.
