The Singularity: A Period Not An Event
Posted by Jeriaska on November 26th, 2007Rodney Brooks is Director of the MIT Computer Science and Artificial Intelligence Laboratory, Panasonic Professor of Robotics at MIT, and CTO of iRobot Corp (Nasdaq: IRBT). His 2007 Singularity Summit keynote speech, entitled “The Singularity: A Period Not An Event,” argued that the singularity will encompass a time where a collection of technologies were invented, developed, and deployed in fits and starts, driven not by the imperative of the singularity itself, but by the normal economic and sociological pressures of human affairs. While a Hollywood treatment of the singularity would have a world just like today’s, plus the singularity, as a singular event, in reality, the world will be changing continuously due to rapid growth in technologies that are both related and unrelated to the singularity itself.
The following transcript of the 2007 Singularity Summit presentation by Rodney Brooks “The Singularity: A Period Not An Event” has not been approved by the author. An audio recording of the talk is available at the Singularity Institute website.
The Singularity: A Period Not An Event
Welcome, everyone. I’m very pleased to be here talking at this summit. It’s the Singularity Summit. I haven’t been involved with the Singularity Institute before, so I thought I would check and see what is meant by “the Singularity.” The Singularity, as defined by the Singularity Institute, is the technological creation of a smarter-than-human intelligence, and the question is whether this will lead to opportunities and risks.
Now, predicting the future is sometimes hard, so I want to take you back to Paris in 1783.The first hot air balloon floats out of Paris. Marie turns to Francois, and what does she predict about the future based on that hot air balloon? Does she say, “This is great, we’ll be able to travel internationally anywhere within 24 hours”? Does she get upset about noise abatement at airports, which is a big issue from flying? No, that’s not the sort of things that she can think about. They are the consequences that happened from tha, but they weren’t the things she could think about. She was more likely to have said, “No one will be able to breathe in those things.” Or, “It’s going to crash and the city is going to burn down!” Or, maybe, and this looks a little quaint, we’ll be able to get up really close to God and touch his face.
I think at the time when things are happening, you understand the world as it is, but it is very hard to understand the world as it will be. And the sorts of questions we ask ourselves here may well be the wrong questions in the long term. So, I’m a little skeptical about some of the worries of an artificial general intelligence and I’m a little skeptical about some of the promises. I sort of see a little too much techno salvation and techno holocaust. I don’t think things are ever quite as good as we expect, in my experience anyway, and never quite as bad as we fear. I’m being up front about that. There are the questions that Hollywood asks. Will they be great, will we accept them, or should we fear them? This is a theme in movies.
When we think about the future, lots of people have thought about it. Niels Bohr said it’s really difficult to predict. Albert Einstein, this shows he was really a scientist, he was not a technologist, because technologists will not bear with that. And Yogi, he had something intelligent to say. As I was looking through quotes about the future I realized, we didn’t know how good we had it when Dan Quayle was our vice president. “The future is here, maybe, just not everyone’s got it.” And now every research lab in the world now uses this one as their slogan: the best way to predict the future is to invent it, and we’re the ones who are going to invent it. I actually think Arthur C. Clarke had it right. Arthur C. Clarke said that when it comes to technology, most people overestimate it in the short term, but underestimate it in the long-term. And that’s, I think, what we would have seen in Paris in 1783 - overestimating how good it would be in the short term but completely missing the understanding that we have 200 years later.
So, in the future, the way we think about the future often is through Hollywood. But Hollywood has a very specific way of talking about the future. This is from one of, I think, the best movies about the future, Bicentennial Man. And there’s Robin Williams being a robot, but look what else is in this picture. Here, he’s sitting there, he’s got a fully functional android and he’s reading a paper newspaper. She’s pouring orange juice, though she’s got a fully functional android. So, what happens in Hollywood, we tale the world exactly as it is, and then we add one thing. So, in AI, which is one of the worst movies about the future, it was the world as it is and then they had robots, and then they added emotion to the robot and then everything changed. In Minority Report, they still used regular guns, even though they were able to predict the future in great detail.
My point there is that when an artificial general intelligence appears, the world is going to be a very different place than it is today. So it’s not today’s world and add in this really super-intelligent being. It’s the world that’s going to change over time. By the way, I think by then “we” will be long gone, but in a positive way. I’ll come back to that.
The world is going to be different before we have these general intelligences. Notice I said “when,” not if. I firmly believe “when.” In underestimating the long-term, Star Trek is one of my favorite TV shows, but if you look in the ’60s version all the technology is wildly dated. This thing on the left there, I think that’s a folio or something, and that was Janeway’s computer four centuries from now, which already looks way dated. So, people just completely underestimate the long-term. Arthur C. Clarke, by the way, I think did pretty well with HAL. I think that is one of the least dating science fictions we’ve seen. It was done in the 60’s, but by 2001 it was still pretty good. That’s one of the reasons I’m a great fan of Arthur C. Clarke.
I think the future needs AI and robotics. There is a varying age group in here, but I see a whole bunch of people in my age domain, the Baby Boomers, who are about to get a lot older. This is the population of Europe in 1950. We’ve got five year age intervals. This is a histogram of men and women. You see the age distribution, you see the notch taken out by the second second World War. That’s 1950, 350 million people in Europe. This is in 2000, and that’s in 2050. Notice something? Everyone’s getting older. And there’s a lot more older people that are going to be of retirement age or infirm, relative to the number of younger people. So, not only how are we going to pay for social security, who’s going to be providing the services to those older people?
This is Japan. You see the same sort of dramatic change in the population profile. That is happening. The demographics just tell us that. I don’t have the same for the U.S. I have a different format here. But here you see in red the percentage of the population over 65. We are just here, where there is this rapid transition in our society of over 65 about to happen. That’s all the Baby Boomers getting older. Even the over 85’s are starting to go up, so there’s hope for us. My point here is, there are going to be so many market pulls on providing services, things that are currently done by the working aged between 20 and 65 are going to be a much smaller portion of the population, so their productivity will have to be increased through information technology and through robotics.
There is going to be tremendous pulls on those two things over the few years. So we will be getting a lot of push, a lot of venture capital, a lot of government research funding, pushing into AI and intelligent systems. There’s going to be rapid progress. There has to be, because of these demographic trends. There are big issues in labor, and we see that in our politics all the time. In Europe, North America, etc. we outsource low cost manufacturing labor, everyone hates that we’re getting rid of our jobs. We insource agricultural labor in Europe and North America - big fusses about immigration. So there’s lots of political pressure here, which, again, is going to get us to push different productivity models, and AI and robotics are really going to be part of that.
Given those pulls and pushes, what do we know about the future? One of the things we know about the future is about certain exponentials. Exponentials are important. Ray Kurzweil is not here today. I’m not going to imitate Ray, but I’m going to talk a little bit about exponentials for everyone. One of the great stories about exponentials was really Gordon Moore’s article in 1965, where he talked about the exponential increase in the number of components on a chip. This is taken from the start of his paper and was in a little magazine. He didn’t think much about this article. He didn’t realize what a big issue it was going to become. In the first two paragraphs, and I think this is one of the best examples of predicting the future I’ve seen, he says that integrated electronics is going to change us. We are going to get home computers - remember, he said this in 1965, when computers cost millions of dollars and filled many rooms. He said we are going to get automatic controls for automobiles. For those of you old enough to have experience with a 1965 automobile, there was almost no electrical system in there, let alone an electronic system, at the time. He also said we are going to get personal, portable, communications equipment. Cell phones. That was pretty good in 1965.
There was a cartoon in the paper, where you see the guy selling handy home computers right next to the people selling cosmetics, and I asked Gordon whether he had been involved in that cartoon. He said, no, this was a total surprise when the article came out. I think the cartoonist was sort of making fun of him. Computers and cosmetics being sold together, what a wacky idea. But it turned out to be completely true. So, sometimes cartoonists get it right accidentally.
The interesting thing about his prediction was he only had four data points. The integrated circuit in 1965 had fifty components. He only had four data points and just extrapolated that, and that extrapolation, it can be argued, drove the industry until today. Because people were expecting the exponential, they followed the exponential. Engineers would go into their corporate offices in the valley, and they would have up on their chart what the exponentials were going to look like. They knew what they had to be working on today for something three years out.
I will give you one other exponential just to convince you that it’s happening, for those of you that don’t know it. The iPod. I started tracking the iPod a couple of years ago. In mid-2003, ten gigabytes for $400. Mid-2004, twenty gigabytes for $400. There was no forty gigabyte model in 2005, but in mid-2005 these were the prices on the Apple website. You can sort of interpolate. In 2006, it was a little late, but eventually we got the eighty gigabyte model in September for $350, and this last week Steve Jobs announced 160 gigabytes for $350. Essentially, the formula for how you get for $400 is 2^year minus - 2003 x 10 gigabytes. It’s just going along like clockwork. Maybe Steve has it plotted on his wall and he knows what he’s got to deliver.
So, that means by 2025 we’ll have 40 million gigabytes in our pocket. 2025, that’s a long time away, but I’m sure some of you have kids in middle school, and you are still going to be supplying them with their iPods by 2015. In 2015, it’s going to be 40,000 gigabytes. And 2015 is not that long away. What can you do with that? The Million Book Project started digitizing a million books, and it has since been taken over by Google. A million books is 500 gigabytes. That’s an iPod two years from now. You can only carry around 320,000 books with you this year, I’m sorry. We think of the Library of Congress as being a big data unit. “More books than the Library of Congress” or “As much information as the Library of Congress.” That’s an iPid in 2013, the Library of Congress. The current iPod, 200 hours of video, a hundred plus movies. The last time I looked, this is a few weeks old, 370,000 movies on imdb. Let’s throw in Bollywood. 800,000 movies, say. By 2020, you will be able to have all of them in your iPod, every movie ever made. Now, if you read the reviews you know that most of them aren’t worth having, so by 2016, which is only nine years away, you’ll be able to have every movie ever worth looking at in your pocket.
So, things have to change. It’s overwhelming. New business models have to come through, have to triumph. I’m going to come back to exponentials. Exponentials are important. One of the new ones is the number of cores on a chip. It was only three years ago that our laptops had a single core. Then we got to two cores, and now we are starting to see four cores. Three weeks ago, Tilera, a spin out company of our lab CSAIL out of MIT, come out a single chip with 64 cores, each running LINUX on it. Now, we are just exponentially putting the amount of cores on chips. So, this is happening, but it is not necessarily enough. Let me now change direction a little bit and talk about some of the robots and AI that I’ve been involved with, just so you get a feel for where things are and where they might be going, and then bring back and couple that with exponentials.
I have been involved in two fronts. One is building robots through a company. We now have over two and a half million robots out in people’s homes. That’s up from zero in 2002. That’s a pretty big change. I know that some of you here have been involved in AI for a long time. Every one of these robots is programmed in LISP. At iRobot we’ve also been building military robots. From 1998 to 2002, from the initial development to getting them out there, we started going from academic experiments to really robust systems. Now, you’ll see some tests from around 2000. The Marines like to throw things through windows. So these were robots at the time to go ahead of the soldiers and look at things. You know, not just computation is important but physics is important. The thing that happened with these robots that were still experimental, along came September the 11th, and by midnight that night, we had robots down in New York looking in the pile, but we couldn’t really do much there. Then we put robots into all the semi-collapsed buildings with infrared sensors, looking for people, to see if there were any people still alive in these abandoned buildings. That really changed the mindset of the military.
We went from zero robots in the U.S. military in 2001 to by summer of 2002 we had robots in Afghanistan, and now there’s about 5,000 deployed robots in Iraq. This is one that we built, the Packbot explosive ordinance disposal, and we’ve got about a thousand of them there, going out everyday, multiple times a day looking at roadside bombs. Sometimes they get blown up. Sometimes they succeed. You can see there, that’s Scooby, that’s a head of one of the robots, and maybe you can see the marks that the operator of Scooby put on there. I think you can count seventeen IED’s, one UXO unexploded ordinance, and one vehicle-based IED, before Scooby, on the bottom-left, got blown up. His operator brought him into the depot and said, “Can you fix Scooby?” The people in the depot said no, but here’s a new one. He said, “No, I want Scooby.” There’s a real bonding that goes on. it surprised us.
Future combat systems by 2015, one third of all future military missions are meant to be unmanned. This whole restructuring of the army on the right there is with robots. We are now doing experiments with robots with real soldiers happening. So, there is a real transformation happening in the military, and that, of course, is putting a lot of money into robotics and therefore AI. That’s another driver here. That’s one part of my life. Another part of my life is at MIT as a researcher.
This is a former student of mine, Cynthia Breazeal. I’m going to show you a couple of the robots from the last few years. First I want to show you Cynthia’s robot, Kismet. Kismet interacts with people. The interesting question for us is, Are these intelligences going to interact with us, and in what way are they going to interact? In Kismet, we put a visual attention system, which looks at various features of the world, such as skin tone and in HSV space all people map into the same area of HSV space. Unfortunately, so does plywood. We just moved into a Frank Gehry building, but that’s another story.
So, it looks at skin tone, saturated colors, motion, weight those and decide where to drive the eyes, and then we have the habituation module there so that the robot doesn’t stare at something too long. So there’s cameras behind these eyeballs, and just putting eyeballs on there, humans are really good at estimating gaze direction, so when you see this robot, you are able to estimate where it’s looking as it’s looking around. It happens to be looking for a toy and it’s got a little emotional response here, so you as an observer can tell how it’s reacting. But its visual attention system will also pay attention to important things. So, here comes a big thing from the right of the image. It’s Cynthia Breazeal, and it pays attention to her, as you would expect a person to pay attention. By putting a visual attention system, which is very similar to a human visual attention system, people understand the robot’s intent a lot more. I think that understanding intent of our intelligent systems is going to be important. If we don’t understand intent, we won’t know what they’re about to do, and we won’t trust them.
So, on this robot we have an emotional system, a fairly simple emotional system. Here is the robot giving emotional displays to different parts of its emotional space, saying a string of phonemes, it doesn’t understand them, but you will read emotion into its different displays. The lips are purely there for display purposes. Here we have the robot interpreting the emotions of humans and responding appropriately. These were mostly naive subjects being asked to give basic messages in various languages. The robot is interpreting that emotional content in their voice and responding appropriately. Here we asked the people to get the robot’s attention and indicate whether they thought they had its attention, which, again, is important if you are going to be interacting. Here we’ve got prohibition. This one’s pretty tough.
A common reaction is that the robot does something and then people imitate the robot. It hangs its head, she hangs her head. These are a fairly simple set of perceptions but people really start to feel these things are alive. Now, we added in turn-taking and we had three hundred different subjects come in and videotaped them. We told them to talk to the robot. Those were the instructions. We didn’t tell them the robot didn’t understand English, we didn’t tell them that it didn’t speak English, it just says English phonemes and babbles, but it does know about turn-taking, which we use when we’re talking to someone pretty well. It does follow eyes, and follows motion cues, so it has a visual attention system. About 10% of the people just didn’t get it. About half the remainder got that there was turn-taking going on but didn’t know what to say to the robot. About 45% of the people overall just talked to the robot.
This is Richie. He talked for 25 minutes. Right near the start he says, “I want to show you something. I want to show you my watch.” And the robot looks at his watch. It didn’t understand the word “watch,” but he used the right motion cue that I would use with a person in order to get their attention. The robot notices that and does the appropriate thing.
Here is another thing that we use all the time. We estimate gaze direction. This is something that humans do, but chimps are not real good at. This is a distinguishing feature. It sees her gaze and looks where she’s looking. That’s very important if you’re trying to construct a robot or interact with a robot up close, having it understand what you’re paying attention to.
Now, I’m going to show you another robot, Domo. Aaron Edsinger is in the audience somewhere here. This was his PhD project, and Jeff Weber, the mechanical engineer is here, too. Jeff and Aaron have a new company here in San Francisco called Meka Robotics, where they are exploiting some of this technology. This is the visual attention system for this robot.
The visual attention system for this robot is much more sophisticated than the earlier one, tracking people and dealing with self-motion. You notice as it’s doing its head motion, it’s still keeping track of the person, distinguishing self-motion from Aaron’s motion, etc. And then, being much safer to interact with than any industrial robot, because I think we’ll want to get up close to our robots and our AI’s and interact with them.
Here the robot is reaching for this green ball but Aaron’s not afraid of getting right in there with its arms. If you’re in an automobile factory with a robot with an arm, don’t get this close, because you will regret it. And the robot is also aware of external forces. Watch its eyes. Whenever there’s some sort of force on it, it’s interpolating where that force is, so it’s aware of itself. I think we need to make our machines self-aware as they interact in the world if we are going to be happy with them.
If you go to Google and search for “Domo” and “Edsinger,” you’ll see all these five-minute movies of Aaron going through different tasks with the robot. He’s giving it objects that it’s never seen before. It’s moving them around to get an idea of the visual extent. The robot sees the shelf but it’s not exactly sure where it is. It reaches out with its left hand, just to feel it and be aware of it. It’s using forsensing in its arms. Now it wants to put the object on the shelf, but it can’t reach it with its right hand, so it looks down at the object, moves the object around a little bit to figure out the extent, does a visually guided change of hands, where it also uses forsensing in its hands to go from one hand to the other. Ultimately, if you let that play, it’s going to put that object up on the shelf. If you go on the web you’ll see lots of other things.
It’s aware of its world, it’s able to interact in some ways. You might say that’s a demonstration but it’s a long way from reality. I came to the U.S. in 1977 and had the chance to work with Hans Moravec at the Stanford AI lab, down in Palo Alto. This is the Cart, which was his robot in October of 1979, when everyone had gone home and before the computer musicians came in at 6:00 AM, we would set up this room with this robot and try to get it to drive across the room. Now, this is being filmed. The flashes are when the film stops, and you see that when it stops, the camera moves from side to side on top of the robot there. That’s because cameras were 50,000 bucks each. We could only afford one. Then it computes for fifteen minutes and moves a meter. So this is a six hour run you’re seeing here of the robot driving across the room.
That was 1979 at the Stanford AI Lab. In 2005, the Stanford AI Lab had a different car. Instead of 20 meters in six hours, it went 200 kilometers in six hours. Same lab, 26 years, four orders of magnitude improvement. This is the power of exponentials, and we will see that more and more in our robots.
While exponentials are probably important, and I say “probably important” because I’m not sure evolution exploited exponentials, I don’t think they are sufficient. There’s lots of things that we still don’t know how to do. So, this is an example. I’m going to ask Sam, what do you see up on the screen there? It’s kind of hard to see with this lighting but these two squares, one’s a light one and one’s a dark one, let’s just strip away some stuff here. They’re identical color, but to us we perceive them as light and dark. We do a lot of interpolation, we understand the world quite well. Human vision is very sophisticated.
This is one of the things that I use for my class to show that simple biological tricks are actually fairly complex sometimes. These are polyclad flatworms. These are worms that are on coral reefs. They’re pretty simple. They eat and they use their fluffy sides to put food into their mouth orifice, and they’ve got a brain which has about 2,000 neurons. The brain sits up there at the top and its got these four longitudinal nerve chords that it sits on. In the 50’s people started doing experiments with these where they would cut the brain out of one worm and put it in another worm, do a brain transplant, to see if the body could adapt. I suspect the graduate student made an error. They never say that in the papers, but then there’s a whole series of papers about what happens when you take the brain out and instead of putting it in the right way, you put it in backwards, or you flip it over, or you put it in backwards and flipped over. In each case, after a few days, the animal is as good as new.
If you put the brain in backwards, the first day it walks backwards a lot, but by the next day it’s walking forwards. If you rotate it 90 degrees, it never works, because these longitudinal chords never adapt. Imagine just taking an x86 out of its socket, putting it backwards in a Power PC socket, and expecting it to work. Our technology doesn’t know how to do this, and maybe we need to understand how to do this sort of adaptation. So, the future, it’s not just exponentials, we still need to figure out some stuff.
Now I want to present some alternative futures to think about for the Singularity here. Perhaps we’ll build a general intelligence and it won’t realize we’re here and we won’t realize it’s there. Could that be possible? Well, thirty years ago we didn’t know about archaea, which are one of the three main forms of life on earth: the archaea, the bacteria, and the eukaryotes. We didn’t know about them. Lo and behold, we’ve discovered in the last thirty years, 50% of the world’s biomatter we didn’t know existed, under the ground. Total ignorance about these things. Maybe we won’t know about them. That’s an alternative that I think people aren’t thinking about. In fact, maybe it already exists on the Google servers and we don’t know. But I sort of don’t really believe this, because this relies on it coming into being accidentally. And I don’t think Johnny 5, cute movie but the lightning strikes the robot and suddenly it’s conscious? I don’t think that’s how this is going to happen. “I accidentally built a 747 in the backyard. I didn’t mean to.” I don’t think these things are going to arise accidentally.
Before we have the fully general one, we’re going to have one that’s almost that good, in the same way that chimpanzees are almost human, gibbons are almost chimpanzees, etc. So, it’s not going to happen accidentally. It’s going to happen because we want it to, although, maybe, there will be some accidents. Here’s an accident that could happen. We start to see large-scale, unexplained oscillations in the internet, and we see coupling of these oscillations at a distance, and the neuroscientists say, “It must be conscious, therefore,” for those of you who know the literature. It will be disruptive and bad news, but it won’t be at the level of consciousness. We will get over our fears of cyberterrorism and put in cyber-inductors and dampen it out. So, those sorts of accidents might happen. There might be some annoying alternatives, too, that could happen in the future. We build the AGI by 2029, Ray Kurzweil’s date, and it knows we’re here but it ignores us. For those of you who live on the east coast, it’s like the way you treat chipmunks. They live out in your yard but you ignore them. You don’t worry about their families, you don’t worry about their relationships. You don’t work harder to feed them more. They’re just a different part of the fabric. Perhaps that’s how we’ll be treated.
Another alternative future, and this I think is more likely, we become more and more dependent on home robots for care. They’re networked, for all sort of reasons, and a bad virus arises through some mechanism or another. That virus goes out over all the home robots looking after me and all my buddies, and one night in 2028, five million aging Rolling Stones fans die because their robots breakdown. It’s a big fuss, robotkind gets set back fifty years, and it postpones the Singularity for a long time. By the way, Keith Richards survived that, too. This is yet another alternative future. Patrick Winston used to say he had a pet raccoon that was really dexterous but he never thought that that pet raccoon would build a robot version of itself. So I sometimes wonder as I’m working away at MIT whether the aliens from Alpha Centauri are up there looking down and saying, “Look at that cute little Rod Brooks at MIT with his little dexterous hands and his typing. He thinks he’s going to build an artificial intelligence. Isn’t that cute?” Maybe we’re just not smart enough.
Still yet another alternative future, and I think this is really likely, that direct neural implants become elective. We already have over 50,000 people with cochlear implants directly connected to their ears. We’ve got visual implants in clinical trials, other things happening. Drug enhancement becomes accepted. Instead of the Tour de France worrying about it, they get over it and live with it, as does baseball. Genetic enhancement becomes normal. Neural enhancement catches on, especially for the aging Baby Boomers, and out of that “we” and our world won’t be “us” anymore. We’re going to change our world as these intelligences come online. We’re going to be different beings with different capabilities. Who is “us” and who is “them,” I think is going to be a different sort of question. There may not be an “us” and a “them,” and this could all happen before 2029, to use Ray’s date. I talked about all this in a book a little while ago.
People say, “Why is your company named iRobot just like the movie?” The answer is we stole the name from the same place. This is our IPO. You know, I like the movie iRobot. There was the oddly academic scientist, and then there was the beautiful engineer scientist, and then there was the hard-pushing CEO. So, there’s me and Helen and Colin. But we noticed that before the opening credits, the elderly scientist is dead and by the closing credits the CEO is dead, but Helen likes to point out that she ended up with Will Smith. Humans, robots, we’re all the same. We’re all going to mix up. Virtual reality, entertainment, it’s all going to happen together.



May 29th, 2008 at 4:47 pm
Transcript of the question section or at least the bits I liked:
The question is, can I talk about the inspiration for the user interface on the combat robot?
Yes, on the combat robot, we started out with engineers designing it, very expensive, joysticks with force reflecting, we put it out in the field, the kids out in the field, the 19 year old started doing *bang* *bang* *bang* pulse width modulation with their hands, umm, we changed it then to a game controller and now the 19 year olds in Iraq pick it up, zero training, know what to do.
Great.
[question about flat worms, etc]
[different question about humans merging with ai, losing emotions, etc]
[question about research funding]
The question is, I used to talk about insect level intelligence, what’s my attitude to that.. well, I’ve got 3 million robots out in people’s homes with insect level intelligence. It’s a real commercial success. But it doesn’t mean we should stick with just that. Some of the principles from that we’ve been using in these humanoid robots and I was trying to explore a different set of space, but really, I tend to think that, humans are just bit insects. [laughter] Ha ha, we’re not as smart as we like to think we are. I still believe that, at its core.
The question, is about [soldiers] becoming emotionally attached to the robots and has that caused us to rethink at all. No, we haven’t done that in the military space, but in the home space we’ve seen people getting attached.. there’s a whole set of third party industry making clothes for roombas, there are skins for roombas that you can get, there’s some web sites, so I think those, ya know, we’ll have Facebook for robots [laughter] I mean, there really is part of this attachment that’s an interesting phenomena going on there. Sherry Turpils looked at it with Furbies a lot. There’s a lot of projection onto these devices which they don’t really deserve from a rational point of view. But we’re not rational beings.
The question is, there have been reports of packbots being equipped with machine guns and what do you do worrying about friendly fire. Actually, that’s not true, none of the packbots have had a machinegun, the Talon from Foster Miller has had a weapon on it,
all with safety circuit and a human in the loop. I think it is an interesting question, when (if ever) do we want to allow robots to have independent targeting authority. I think now is the time to act. There’s a bunch of ethics conferences coming up in the next year. I think its time to put this into the Geneva Conventions - some governments do go along with the Geneva Conventions - and [laughter] I think its time to think about that. Absolutely.
[Audience member asks a follow-up:] You said “some governments” follow the Geneva Conventions, but apparently not that you’ve done some work for. Is it a good idea for you to be developing AI and robotics for the US government? and, umm, in my mind, that could lead to some of the worst nightmare scenarios and I’m wondering how, ya know, what your thoughts are on mitigating against…
Yeah, I think that, in a sense is nothing to do with AI, that’s been a question which has faced scientists in the past since the time of Da vinci, who was completely funded by military, doing military work for his patrons. So that’s an issue that scientists have had to deal with for hundreds of years. Independently, of AI. And I think it is a big responsibility of scientists to worry about controls of how things are used and I think, actually, the Geneva Conventions have been a good way of doing it. We’ve seen very little biological weaponry, appear, because it is banned by the Geneva Conventions, so it has been successful but there are perturbations, governments do change, governments can change, people can change the governments and I think there’s going to be an on-going question, for a long time. But I don’t think its AI specific.
Ok, I think I’m finished.
October 5th, 2008 at 12:31 pm
[…] Brooks at the Singularity Summit about strong AI (here is the video and here is the transcript): “Before we have the fully general one [general AI], we’re going to have one that’s […]