Photo by Vincent Diamante

What new kinds of games will we play in the future, and what key knowledge and skills will game developers need to invent them? Futurist and game designer Jane McGonigal argues that over the next decade, games will become a powerful interface for managing our real work, organizing society, and optimizing our real lives. Increasingly, she predicts, game developers will be charged with the task of making people happier, smarter, friendlier, greener, and healthier — and hundreds of millions of new gamers will be playing together at home, at school, at work, and everywhere in between. The result? Game design and development expertise will become a sought-after talent in virtually every industry and field, from Fortune 500 companies to top government agencies.

The following transcript of Jane McGonigal’s 2009 Serious Games Summit keynote at the Game Developers Conference has not been approved by the speaker.  Audio is also available.  Related text and video is on Boing Boing.

Learning to Make Your Own Reality

Are you optimistic about the future?  If you are, please raise your hand.

That is not surprising at all, because I think game developers have some of the most reasons to be optimistic out of anybody on the planet.

Here is someone who is not too optimistic about the future.  This is some graffiti in London.  It’s kind of an angry guy yelling at people to “INVENT THE FUTURE.”  The person who posted this on flickr had a really interesting caption.  “‘Invent the Future,’ it says…

“Well, it’s probably better than inventing the past, but I wonder what proportion of people in the world really have the slightest chance of altering their own futures.”

Now, I am here with an answer to that question.  I would say about one in two thousand.  That is not a super scientific number, but a back-of-the-envelope estimation based on the prospect of having three million game designers on a planet of 6.7 billion people.

I think it is game developers that have the opportunity to invent and alter their own futures. I am the director of research development at the Institute for the Future.  If you don’t know the IFTF, it is a non-profit think tank in Palo Alto, California that has been going for 40 years. We have outlived many of our forecasts about the future and can be held accountable for whether or not they are helpful.

This is one of our recent logos that we made.  It is an adaptation of a World Fair sticker from the 1930′s.  It used to be that you would go the World Fair and see the future. At the IFTF we are not really interested in trying to predict the future. We want to decide what the future should be like, and then make it.  We are all about figuring out what the tools are to imagine a best-case scenario for the future and go about building it.

In my research over the last couple of years, I have determined that game platforms are probably the most robust tool that we will have for inventing a best-case scenario for the future.  That is what I am here to talk to you about today: learning how to make your own reality and teaching game developers, game designers and gamers how to make a better reality.

I believe that the great work of game designers over the next decade will be to reinvent real life as we know it.  We are not talking about getting rid of games that are mostly for entertainment.  We are going to use the entertainment games industry as a pipeline for innovation that we can use to redesign the real world.

There are five key forces that I think are driving us toward a game designer’s future.  These ideas are sustainable happiness, persuasive technology, the engagement economy, programmable reality and superstructing.  These five ideas will give game inventors the best odds of reinventing life as we know it.

Let’s start with sustainable happiness.  This is a measurably higher quality of life that first flows from scientific findings about optimal human experience and also requires fewer non-renewable resources to generate well-being.  Sustainable happiness is growing in part out of the new field of positive psychology.  This is a field that a lot of game designers are paying attention to now.  We all know that games in part are designed to make us happy and to give us optimal experience when reality fails to do so.

We are learning a lot about what makes people happy.  It is a science of physiology, of cognitive science, of social science.  There has been a lot of research over the past decade, but the findings that peaked across cultures and across demographics can be boiled down to four things.  We want satisfying work to do: something concrete and specific, so we can figure out when we’re done.  We want to experience being good at something.  We need time spent with people we like.  Finally, we need the chance to be a part of something bigger.  You want to feel like there is something that extends beyond your own life that you can contribute to.

That is what the positive psychologists are figuring out, and they are turning this into metrics that we can use to measure how much well-being there is in particular cities, schools or households. There are even websites now that will help you graph and chart how changes you make in your life affect your well-being.  There are some large organizations trying to measure well-being.

One of my favorite is nef, whose slogan is “Economics, as if people and the planet mattered.”  They are kind of at the forefront of sustainable happiness.  They have created an index to look at how well countries are using their resources to deliver longer lives and life satisfaction.  They had to change the name of this from the “happy planet index” to the “unhappy planet index” because they realized that we are doing a completely horrible job of trying to sustain happiness with the non-renewable resources that we have.

Of course, game designers would know better.  This is a classic quote on happiness: “Happiness consists in activity.  It is a running stream, not a stagnant pool.” (John Mason Good)  However, for decades now we have been living as if happiness were some sort of pool of resources that we can accumulate for ourselves and horde and consume.  That is obviously not true, and what we are seeing happening in the economy or with climate change is an indication that we have been pursuing failed methodologies for achieving sustainable happiness.

As game developers and game designers, you might look at this list of four things that make people happy and you might think these are the four things that games do.  That’s what I think. I think positive psychology is coming to the conclusion that multiplayer games are the best platform we have now for creating sustainable happiness.

I want to show you an example of trying to use positive psychology to improve one person’s happiness. That is me doing research for my dissertation at UC Berkeley, not seeing anybody, working all the time and that was a model that I pursued diligently even after I finished my degree, living a completely work-obsessed, technology-mediated life. I started looking at positive psychology and sustainable happiness research to figure out how I could make it better.

One finding that shows up in every country and every neuroscientist agrees on is that dancing with other people is one of the most perfect happiness engines.  If we are dancing with someone else, this unleashes an incredibly strong, evolutionarily adaptive pleasure, like the best, strongest opiates in the world.

If we could dance together with people more, that would be great.  The problem is that a lot of people are too shy to dance in public.  For most of my life, I have been too shy to dance in public.  However, when I did more research into positive psychology, I found out that humiliation actually makes us happier.  Across cultures, when we get embarrassed there is a physiological response.  This is a way of showing people that we are vulnerable, they realize that in this moment they have power over us, and that means they like us more. There is this synchronicity of people’s emotional needs being met.  We are happier when we are humiliated around people we trust.

I have invented a game in which you have to dance on the internet, figuring that I will both increase the number of people I dance with and also humiliate myself.  Your first initiative is to create a real-life avatar for yourself and then you have to dance in disguise, so that you don’t have to be too afraid to dance in public.  All the players pretty much cover up their faces.  Then you have dance challenges.  Your first challenge is to dance in disguise without moving your feet.  You upload videos showing that you did this, then other players on this social network give you choreo-power ups.

Like any good game, it gets harder the more you play.  The third dance quest is to dance at a cross-walk.  This is not a serious game, this is just a game for fun, but it is seriously trying to improve people’s happiness by getting people to dance more and humiliate themselves. Other game designers are trying to do this.  There was a great keynote this morning by the creator of a multi-layer card game connected to a social network online, where you are asked to do good deeds in public, prove that you did them, and get rewarded by the social network.

There are also more mainstream games looking at positive psychology, like flOw.  Also, Flower is a gorgeous game.  Jenova Chen, talking about it, says Flower has wonder, twists, despair and catharsis. These are all ways of being happy.  If you look at the whole spectrum of optimal human experiences, here is this great game developer that I am totally inspired by and clearly taking seriously the idea that we can create games to explore the space of sustainable well-being.

Here is my forecast around sustainable happiness. Over the next decade, game designers will become happiness hackers, called upon to help individuals, communities and entire populations better structure their everyday lives for authentic happiness and sustainable well-being.  This is moving beyond entertainment to think about quality of life. If you want to learn more about this, here are three books that I recommend reading, and these are some organizations to track.

One design challenge is, if you could make one person measurably happier, who would it be and what game would you make for them?  In my spare time now I am trying to make a game for my grandfather, who is 96.

The formal definition of persuasive technology is a human-computer interaction that is intentionally designed to change the way someone thinks, acts or behaves. If you have heard of this field, you have probably heard of B.J. Fogg.  He is the father of the field of persuasive technology.  He talks about three ways to change somebody’s life through technology.  You can either increase their capability by building them a tool, you can change the social dynamics in their life by being a social actor, or you can provide them an experience that offers them a new point-of-view by using technology as the medium.

One of the most prominent examples of persuasive technology are hybrid cars with a dashboard that tells you exactly how much you are consuming at different speeds.  This is a blog post by someone who drives a Prius, who says, “My car is a videogame!”  Now she can guage her fuel consumption with realtime feedback just like a game.  Google has a great new technology prototype that will let you see exactly which appliances and utilities in your home residence are using how much power.  You realize how much you’re paying to have your toaster plugged in all the time, so people who have used it have dropped 80% of household usage, just by knowing when they are consuming.

Coming out of the game space, Chore Wars is really persuasive for kids who don’t want to make their bed or do the dishes.  It’s a game where you can claim experience points for doing chores, but it’s a very persuasive medium, I have heard from many parents. Looking at more traditional games, Left 4 Dead persuades gamers to focus more on collaboration than competition.  A friend of mine, who is a hardcore first-person shooter gamer said basically he can’t be an asshole anymore when he plays the game online.

If you want to change people’s behavior in the real world, you can think about asking them to do something that is hard, like stop using oil.  The last time I talked at a GDC Summit I was just launching “World Without Oil,” which was a six-week simulation of an oil shortage.  Everybody says they are going to use less someday, but we asked people to pretend that we had already run out of oil. How would you cope?  What imaginative solutions would you come up with?  You really had to do it, by showing us your videos and blog posts or any other social media you could create that would be persuasive.  At the end of the six weeks we had all these real-life solutions that people had tried to live without oil.

Here is the forecast for persuasive technology.  In the next decade we will see game designers  becoming change agents. They will be called upon to create systems that change minds, teach new habits and transform everyday practices. It is going to be a huge market.  These are some things that you might want to read as you start preparing for this future.

A design challenge for you, if you would like to design persuasive technologies, if you could change either what one person does every day or if you could change what one group thinks about one thing, what would you change and how would your game do it?

The engagement economy is about the growing need to compete for and harness crowd resources and participation bandwidth.  This stems out of a lot of the recent interest in the wisdom of crowds, collective intelligence and mass collaboration.  How do you get lots of people together, combine their diverse knowledge and the brute force of lots of people thinking about something?  In order to have this kind of scale of collaboration, we need to gather up a lot of participation resources.  That includes cognitive resources (what people know, what they are capable of doing) and also cognitive diversity (how many points-of-view on a problem do you need to get it right?)

If you think about the participation resources required to create Wikipedia, Clay Shirky has a great statistic about how many cognitive hours it took to create that. He calculated with someone at IBM that it took 100 million mental hours from a highly diverse knowledge community to create Wikipedia. You could parse that out lots of ways, like a million people contributing 100 hours each, or other ways.

That is a lot of resources to do a mass collab project. What a lot of people are finding out now is that it is really hard to get that many people to contribute to a project that is not their own. This is the prediction market from the front page of a really popular prediction market site, which is like a stock market but for future outcomes.  This is the flatline of activity on the featured prediction market for the whole month.  Nothing.  It’s hard to get people to participate.

This is a screenshot from one of the most popular crowdsourcing sites, where you are supposed to come and collaborate with other people on business enterprise projects.  If you look at the number of members for each project, this is the first page, it’s one member for everything.  Mass collaboration is hard.  It’s hard to get people not to just play by themselves.

Game developers are figuring this out in powerful ways.  It is not so hard to get people to contribute cognitive hours to a game project or game world. We are talking about 100 million mental hours—if you multiply 10 million games x 16 hours a week, you realize that it might take just five days of World of Warcraft gaming to create Wikipedia, the whole thing.  Big companies are starting to figure that out, and they want to know how they can make their mass collaboration problem as engaging as World of Warcraft.

Nick Yee, one of my favorite game researchers, says “The purpose of all videogames is to train a player to work harder while still enjoying it.  Success of online games demonstrates how seductive and concealed the work treadmill can be.”

There is no reason why we cannot take real-world work and real-world problems and seductively conceal it in an awesome game.  World of Warcraft players are some of the most active mass collaborators in the world.  Theirs is the second biggest wiki in the world after Wikipedia. 5 million people go there a month.  They have over a million registered users and over 71,000 content pages about World of Warcraft.    Gamers have no problem doing work and contributing to collective intelligence—they just have to care about it.  There is a project from the University of Washington, where they have created a 3D game environment where you can fold proteins and they will analyze the folds that you come up with to see if it might cure cancer.

At the Institute for the Future we have just launched a game platform for micro-forecasting, where you have 140 characters to make a forecast about the future. We have been running a game called Free Space, which is about our ten year forecast for personal satellites.  When we bring the cost of sending a satellite into space down dramatically, we have been asking people, “What will you do when it is as easy and affordable to launch a personal satellite as it is to launch a website?”Right now it costs $30,000, but it looks like it will be well under $1000 within the next 20 years. We asked people to tell us their most optimistic idea and also their most pessimistic idea.  Other people can argue with them with “antagonism cards.”  They can build on their stories with “momentum cards.”

I think the most impressive project to date that is trying to use a game style to do real scientific research is Galaxy Zoo.  This is an amazing project that not a lot of people know about.  They take in a quarter of a million high resolution scans of other galaxies by means of a robotic telescope, they release them to the public and ask them questions.  These are questions that you cannot just run through algorithm detection—you need a human eye.  For instance, “Is this galaxy smooth or rounded?” People look at it, and they identify if it is or not.

For the first 24 hours after launch, they had 70,000 classifications an hour, and this is not just about participation.  They are actually finding stuff out.  They have four published, peer reviewed scientific papers and four more in progress, just based on what players of this game have found.  They actually discovered a new solar object that nobody has seen before in all of science, and they named it after the player who found it.  It’s called the “Voorwerp mystery.”

Part of the challenge will be matching what normal people can do with engaging platforms, and then turning that into real results. Your forecast for the engagement economy over the next ten years, we are going to think about game developers as “fun engineers” and “mass collaboration architects.”  We will be asked to invent fun new ways for more people to participate in mass collaboration efforts to contribute their collective intelligence.  We will be charged with developing platforms that allow an average person to contribute to a larger good.

If you want to do a game design challenge around the engagement economy, if your game can get one hundred people to do one thing online, what would it be and what would it add up to?

Number four is programmable reality.  Here is the formal definition: Our growing ability to embed sensors, microcontroller boards and networks in physical objects, built environments and ourselves.  We are talking about location-aware technologies, so my iPhone knows where I am.   We are talking about sensors like the Nike+ system, where my show knows how far I ran, and bio metrics like the NeuroSky headset that will measure attention and arousal.  We are even talking about augmented reality—things that overlay data on the real world.

Those are going to be platforms that will increasingly be used for gaming.  They are a perfect fit for gaming.  Nike+ turns your runs into a videogame. You are competing against friends, and it’s telling you when your best runs are, when you’re being really lame.  It can give you special challenges, and it’s already naturally very much like a game.  They have recently taken the next step, though only in Hong Kong, by creating a virtual MMO using Google Earth overlays so that when you do your Nike+ running, it matches up with a virtual quest in an MMO.

Speaking of bio metrics, this is one of the first big game to use the bio sensor headset.  You control the game and are wrestling with these spirits by concentrating on them.  Instead of getting all worked up, you have to concentrate on silence and peace.  It’s a very different gaming experience.  I was able to play a prototype, and it was four of us sitting around the computer, all trying to concentrate.  Then we would get really excited because it was working, and then it would be terrible and we would fail.  It was a totally different game dynamic.

One of the classic examples in this field is called shark runner. Real tags were put on real sharks, and then you would play a game with real sharks online. You are chasing sharks online and they are in a real ocean, swimming around. You and sharks playing together… to me, that’s awesome.  People aren’t just interested in virtual worlds—they are interested in making the real world more engaging and interactive.

Your forecast for programmable reality is that game developers are going to need to learn to blend realities.  For every screen-based experience, there will be an equal and complementary real-world object, physical environment or social interaction.  I think this is going to be the most transformative out of all five of these.  It will perhaps happen faster than the others. Learning how to hack reality, how to hack stuff, there is a great book called Getting Started with Arduino, which is a super-cheap microcontroller board you can stick in anything.

If you want to program reality, the challenge for yourself, if you could make a game by embedding one microcontroller board or one sensor in one physical object, what would it be, and how do you want to play with that?

Finally, superstructing is the most serious out of all of these.  It is a method of extreme-scale problem solving that emphasizes flexibility, transparency and rapid prototyping.  It requires a high threshold for frustration and the ability to handle uncertainty. I think of this as a super combination of a game designer’s skills and the gamer mindset.  Gamers are probably capable of dealing with a higher degree of frustration because they know they have to stick with something to get past the challenge.

I was at a conference recently where Dan Newman, who is a leading strategist for governments and public agencies, and in the middle of his talk he said, “And wouldn’t it be easier to innovate if life were more like a game?” He was not really talking about games, but I was like “Yes, of course it would be.” He is not the first one to think of that. Albert Einstein famously said, “Games are the most elevated form of investigation.”  He was an avid chess player and he thought about chess as a problem space that you would examine again and again over the course of your life from different points-of-views.  Now we have a lot more than two people sitting over a board.  We have lots of people online exploring big ideas, weird ideas.

Spore is an interesting example of moving toward the superstructing concept in commercial games. There was an amazing interview/ interrogation between Jill Tarter and Will Wright. I just want to read you part of this exchange.

This is Jill: “I’m interested in the possibility that learning to be good at a game makes you good at life, makes you good at changing the world, and gives you skills that are going to allow you to reinvent your environment, because in the game you play against an environment that has been given to you.  You do everything possible to try and make an impact on that world and learn how to optimize your standing in that world.”

Will fired back, “There’s real value in being pushed toward global awareness and looking long-term.  That’s one of the things that I find very useful about games… I think these are the timelines we need to be looking at, the 100 or 200 year horizons.  Most of the really bad stuff that’s happening right now is the result of very short-term thinking.”  He agrees that learning to be good at having an impact in a game could teach you how to change the real world.

This idea that games could teach us to think in longer time cycles and to feel like we have agency to change the world around us is what I call superstructing.  We ran a game at the Institute for the Future this fall called Superstruct.  We decided to give the biggest problem that we could think of to gamers. We created a supercomputer called the Global Extinction Awareness System. The game was set in 2019, and we had the supercomputer predict the collapse of the human species in the year 2042. This was not a real forecast from the Institute for the Future, but a provocation to our gamers. We gave them five ways to try and change the world between now and then to prevent the extinction of the human species: reinventing energy, reinventing food, health, society and security.

We then created a social network where people came up with ideas about how to do that.  It was basically like the Global Game Jam, but for prototyping really strange ideas for saving the world. Our motto was “Together we can re-invent the way the world works.” Over six weeks we had a community of over 7,000 people, everyone from classrooms of high school kids to CEOs of Fortune 500 companies. They created over a thousand stories about this future, trying to save the world. They had five hundred active strategies going, and they created five hundred prototypes for how to reinvent society as we know it.

I was working with futurist Jamais Cascio on this, and our secret mission was to create what we call super-empowered hopeful individuals, or SEHIs.  I encourage you to look that term up online. Basically, SEHIs are people who feel personally empowered to make a difference and try and change the world.  They don’t need the government, they don’t need funding, they need social networks, technology and good ideas. At IFTF we are running a new superstructing project now that is documenting the future of caring, called Ruby’s Bequest. It is going on in the next four weeks.

The forecast for superstructing: game designers will find themselves in charge of extreme scale global collaboration jams.  We might find ourselves being asked to design games that help diverse, massively multiplayer communities handle real-world, open-ended problems.

Your design challenge if you want to help superstruct the world would be to think about this challenge.  If you could make a game that connects two unlikely communities who would never collaborate on their own, and you could ask them to do one extraordinary thing together, who would it be and what would they collaborate on?

Now, to wrap up, what’s important?  The idea here is that game developers over the next decade and beyond will be able to remake reality to make real life happier, to make us smarter, to make everyday work more engaging and to make the planet more resilient.  These are five significant trends in society, technology and science that are making game development one of the key solutions to global-scale problems.

I will just close with this picture, which is more graffiti from London.  It says, “Are you optimistic about the future?”  Then there is this little thing up there that says, “253 babies are born every minute worldwide.”

Right now 60% of kids in developing countries play games on cell phones or one laptop per child.  60% of kids in developing countries are gamers. In countries like the U.S., 97% of kids consider themselves gamers.  When I think about all the people who are born every minute, I think about the next generation growing up gamers, who are ready to be engaged, who care about collaborating and have great skill sets that we can apply to making each other happier and solving some really big problems.

At the AGI-09 post-conference workshop‘s Roadmap Panel,  Itamar Arel of the University of Tennessee announced the founding of a wiki at agi-roadmap.org that will serve as a supplement to the creation of an AGI Roadmap.  Taking as examples several previous, related technology projects, J. Storrs Hall made mention of work conducted by the Foresight Institute on the Technology Roadmap for Productive Nanosystems and Ben Goertzel discussed his participation in the writing of the Metaverse Roadmap.

The following transcript of the AGI-09 AGI Roadmap Panel has not been approved by the speakers.  Video is also available.

Ben Goertzel:  I think Tom did a thorough job of reviewing the various human, social, organizational factors that have been affecting, and in some cases impeding, the progress toward general intelligence.  Itamar is pushing for something I think is also valuable, which is the construction of a roadmap for AGI development.

I just wanted to mention a couple other roadmap projects that I have either been involved with or have watched progress. One is a nanotechnology roadmap, which Josh has had a bunch to do with. The other is more recent, which is called the Metaverse Roadmap, which had to do with virtual worlds technology.

I was more involved with the Metaverse Roadmap, although I was not one of the key architects.  What we did there was pool together people from various companies, some academics, and tried to get some sense of what is the end goal.  In that case, with the  Metaverse Roadmap, the end goal is a virtual world that is an analog of the real world, but in a computer.  This would have different towns, cities and countrysides, where anyone can log in and control a human avatar, buy and sell things, teach each other and get married, everything that you can do in regular life, in the virtual world.

They set out in ten, fifteen years to be at one place with the metaverse, and looked at what steps needed to be taken to go from our current technology and the current economics of the virtual worlds industry to that state of a richly featured virtual world. In terms of artificial general intelligence, there is nothing remotely like that for which the AI community agrees on.  I think that would be worthwhile and does not require a consensus on “fundamentally, what is general intelligence?”

An analogy I look at there is biology.  It’s not like biologists start with everyone agreeing on a precise definition of “what is life?”  There are various conceptualizations of life and there is a general, common understanding of it.  Trying to formalize the question may actually be a valuable thing, but I don’t think everyone needs to agree on that to agree that the human genome project is an interesting idea. I do think defining things precisely has a valuable role to play, but I do not think it is a prerequisite. Also, I don’t think it is a prerequisite for everyone to agree on a common architecture or even to agree on common terminology for everything that happens inside the mind.

I think that a group of researchers could agree on a roadmap for development, in spite of having a very healthy disagreement about a lot of other things that are involved in carrying out the roadmap.  What Itamar called “axioms,” I prefer to think of as functional requirements, to use software engineering terms, although I don’t necessarily want to say that the solution is software rather than hardware.  I think if the community could agree on what functional requirements we would like AGI systems to fulfill n years from now, and agree on a series of incremental milestones, that would be very helpful for a number of reasons—one being better coordination of work among different people, another being making the field seem more substantial to funding sources and people in other fields.

I think the roadmap project is an important one.  What I saw from the Metaverse Roadmap is that it was not terribly easy for them to put together.  It was over a couple years through in-person meetings with various key people in the field that they finally hashed out something that they all thought was meaningful.  Can you say something about the Nanotechnology Roadmap project?

J. Storrs Hall:  Sure.  I am the president of the Foresight Institute. Foresight and the Battelle Institute, which runs the national laboratories, collaborated to produce the Technology Roadmap for Productive Nanosystems.  The idea being, unlike the vast majority of the current nanotechnology research, which is really focused on objects that can be measured in nanometers, we had a vision for producing machines which can produce objects at the atomic scale, in particular to atomically precise specifications.

One of the major things that came out of that roadmap was to have a fairly distinguished panel of scientists and others in the field agree that it was a worthy goal and useful to pursue.  Another one was to get as many researchers together as possible and have them point out places that we could get to, short of the goal.  There was a certain amount of ontology building: we decided what sort of things we might actually be able to do, as well as reviewing our current state of knowledge. We also tried to assign some figures of merit and ways of measuring how far we had gotten along.  The result is a nice big document that you can get from the Foresight website.

Hugo de Garis and J. Storrs Hall at the 2009 AGI Roadmap Panel

Goertzel:  Do you think this is actually going to be valuable in getting work done?

Hall:   I certainly hope so.  I think the fact that it is there legitimizes some of that research a lot.  The other thing is that it does in fact give us things to shoot for.  They can point to the roadmap and say, “This is how far I’ve gotten. I’ve advanced this figure of merit this far.”  My experience has been that it is very difficult to get these figures of merit out of researchers.

Goertzel: You mean metrics for evaluating progress?

Hall: Right. How far you’ve gone and so forth. At the same time, once you have done it, it is the most valuable part of the whole effort.

Goertzel: In terms of AGI, I feel like first there is the question of “What is the goal?”  I mean, are we looking for something that holds a humanlike conversation, like the Turing test?  Is it a humanoid robot that acts somewhat like a human given certain situations?  There’s that question.  Then there is the question of how to make metrics to evaluate incremental progress toward that goal, which is probably a lot harder.

Hall:  I think it’s clear that if you get the right core of a system that is robust and learns from its own experience, that you can adapt it into any of the kinds of roles that you are talking about.  I think that one of the things that we should look at in terms of the roadmap is looking at what the core of that is.  What is it in building the AGI that we cannot get away without?  That’s sort of the ontology.

I have an interesting story.  A couple years ago I was at a AAAI fall symposium and at the reception I was talking to someone who was a mainstream university AI researcher, and the DARPA Grand Challenge had just been won.  Actually, several of these cars had completed a 130-mile course, where the previous year they had not been able to get more than one mile.  I was saying,”Wow, that’s a big advance in just a short amount of time.”  While he was saying, “Nah, that’s nothing.  They didn’t discover any new techniques.  They just took all the stuff that they knew how to do and put them together to make them work.”

To me, that’s progress.  The fact is, if you look at the field of AI as a whole, as much as it’s fractionated if you go all the way from mathematical machine learning to data mining, it may well be that we are getting closer to a point where we are on thin ice and it might collapse, the same way that we had all the techniques for automated driving.  When I say “collapse,”  I mean finding out how to integrate all that.  It might just be susceptible to a process of integration that we saw in the DARPA Grand Challenge.  In basically the course of two or three years people took all the stuff that they actually knew, put together working teams, and built the cars that could win the prize.

I think it is obviously going to be a bit harder in the case of AGI, with all the pieces of knowledge out there in the field of AI and computer science in general, as well as related fields such as neuroscience.  I think one of the keys is just to identify all the stuff that could go into a solution that we already know, and try to understand how hard it would be to put any two of these together.

Goertzel:   That is actually kind of a different approach.  Laying out a roadmap, as I understood it, would be to identify more functionally what needs to be done and what are the incremental stages.  There are so many different opinions on what stuff needs to go into it in order to achieve machine intelligence.  Getting agreement on that is going to be much harder.

Hall:  For a map you have the towns, but you also have the roads that connect the towns.

Itamar Arel:  I want to try to be pragmatic about this, because I think that is important.  If we want to make progress, we have got to have some charter here.  I think what Josh was alluding to is that there are really two steps here.  There is the first step of agreeing what are the axioms or core functional AGI attributes—and once we agree on those, the next step would be to create this roadmap with milestones leading toward human-level intelligence.  That, I think, is the overall goal.  I would argue, let’s first discuss these core attributes.  I feel that if we agree on those and make progress on them, then that’s a good first step.

Stephen Reed:  Just to back up what Itamar says, why don’t we just consider those goals, and I would add natural language understanding to those axioms, and leave it at that.

Arel:  I’m sure some people would disagree and say natural language understanding is not necessarily the first, second or third goal on this AGI roadmap.  I’m not saying I agree with that.  Maybe the first step should be to define whether an AGI system must have it on a functional level.

Reed:  I’m not proposing ordering the goals, whatsoever.  Just to have a bag of goals.

Mark Waser:  I think we need to commit to a consensus in a fairly short time frame.

Goertzel:  Getting people in this room to commit to a consensus does not necessarily make that much progress.  Itamar has created a wiki at agi-roadmap.org.

Arel:  That may be the first step, agreeing on these core functions for AGI.

Christine Peterson is the founder and vice president of public policy for the Foresight Institute  and co-chair of the Convergence 08 Unconference.   For the February 13, 2008 lecture for the Stanford Law, Science and Technology Colloquium at Stanford University, she presented on the long-term potentials and unprecedented challenges inherent in nanotechnology.

The following transcript of Christine Peterson’s Stanford Law, Science and Technology Colloquium has not been approved by the speaker. Video is forthcoming.

Long Term Potentials, Unprecedented Challenges for Governance

This is a huge topic, nanotechnology. It is an absolutely immense topic. The reason for that is that it overlaps with the future of physical technology in general. If it has to do with the physical world, including your body, nanotechnology is going to be a major part of its future. Of course, that covers an immense space. Let’s just jump in and see what we can do in one hour.

You have to have some kind of definition to get started. The U.S. government definition is “the understanding and controlling of matter at dimensions of roughly 1 to 100 nanometers, where unique phenomena enable novel applications.” Now, “novel applications” is an understatement, as you will see.

Is this a useful definition? 1 to 100 nanometers is a linear dimension, a factor of a hundred difference, in length. In terms of area, that is 10^2 times 10^2, which is a variation of 10,000 in surface area. This table is a surface area. Then, if you think about three-dimensional volumes, that is another 10^2. So it is a factor of a million in actual size. So, although up at our macroscale we think 1 to 100 nanometers is not that different, the fact is that if you think of them as three-dimensional objects, we have a difference of a factor of a million in the size of these objects.

If you picture that up at the macroscale, this difference between a factor of a million in volume is huge. Take this object [a water bottle] and then something that is a million times the volume of this. Now, is there any connection between these that is useful to think about? Not really. It is a huge difference in size, even though it is down at a tiny scale. It also, as you notice, says nothing about what it is made of in terms of atoms. Not only do you not really know the size of these things from this word, you have no idea what they are made of.

It is not really a very useful word, especially to the legal profession. As you will see as we go forward, maybe now and then I will remember to say, “Think of how useless that word is.” In the companies and the governments you work for in the future, non-technical people are going to come to you and say, “Why can’t we regulate and patent nanotechnology?” And the answer is—because that is not a useful word, especially to the legal profession.

You will hear, people will sometimes use the word “nanotechnologies.” The plural form is just to remind everyone involved that this is not one technology. You will also hear the term “the nanotechnology industry.” There is no nanotechnology industry, and there never will be a nanotechnology industry. There are nanotechnology-based industries.

So when you hear that term, there is a number of things that could be going on. The person you are talking to is using this term “nanotechnology industry” to try to get funding. They are trying to force through regulation, and so they are using this term to try to get government action from people who often do not have a clue about what is going on. The third possibility is the person using the term is not trying to do anything in particular and is in fact clueless. If you use this term “nanotechnology industry,” be ready to be challenged on it, and be ready to back it up.

Why are people excited about the field? They are very excited. Huge amounts of money are going into this, not just in the U.S., in places like Japan, but even in some of the smaller countries. If you want to be a player in the science and technology of the future, everybody feels they have to have a nanotechnology program of some kind. Take any country that wants to be more advanced scientifically and technologically, type the name of the country and nanotechnology into your search engine, and boom. Generally, they will have some kind of program.

With the countries you named as we went around the room, probably your country has a nanotechnology program. It is worth checking. How much are they spending, where is the money going, when did they start the program, who is involved? These are some things that you might want to be aware of.

This is just a tiny fraction of the applications that nanotechnology will have going forward. You noticed, I just used the single term. It’s a short form—I’ve briefed you on that. Here are just six off the top.

Clean energy is critical. We all want that. It is very important. Clean water is a very big issue for a lot of the countries that you folks mentioned. Huge medical applications are coming. They are coming quickly, initially in the cancer area. But we can expect many, many other applications.

Healing and preserving the environment—environmental cleanup is going to be a big application here. It is already a big part of information technology, and that will continue. Some of the countries that you mentioned, like Japan, have a space program. This will play a big role in that as well.

Anything that involves physical technology in the future is going to have nanotechnology involved. There is just no question about it. To the extent that you go back to your home countries and you are involved with manufacturing, with any kind of industry, anything that is not pure software, it is going to have a nanotechnology component. That is why governments and companies are investing huge sums of money in this. There seems to be a race going on between the U.S., Japan, and Europe to see who can spend the most from the government level, but many of the other countries that you mentioned going around the room are also spending.

The question is will these countries get value for that. In such a huge space, it would be easy to take whatever money that your country is putting in and spread it too thin, and not really gain world leadership in any niche market at all. That could easily happen. When you go back home, and you find out where is this money going, that is what you might find. Now, some countries are realizing this and are saying we are not going to do all of “nanotechnology.” We are going to home in on one specific area or set of areas where we have some inherent strengths already, perhaps based on our natural resource base in industries we already have. That would be something for you to check. Are those synergies operating? Or has the money been siphoned off in not very helpful directions?

I know that Russia, for example, is putting in a lot of money. But I was told frankly while I was over there that about half of it will be siphoned away into corrupt uses. This is just because they do not have a good governing system in Russia. There is a lot of corruption at all levels in the country. You can vote to put the money in, you can be trying to help the country, but the money tends to flow away. This might be something to watch for when you go back home and you are looking at your countries programs. Where is this money going?

I know some of you will probably go on to work for companies, some of you may go into private practice, some of you may go into public policy or government eventually. This is just something to be aware of.

When you think about nanotechnologies, it is really important to think about timeframe. This set of technologies is going to change drastically as time progresses. Right now, in the current phase, mainly what we are doing is coming up with new materials. Many people, when they hear the word “materials,” they fall immediately to sleep, and I understand. It sounds like a boring thing. In fact, there are some amazing materials coming—the kinds of things that can give you really cost-effective solar power, for example. That is going to be critical for all countries that you mentioned, whether developed or developing. Cheap solar power is going to make a huge difference for us. It is important for the environment. We are starting to do things like coming up with coatings that will keep things really clean. Again, it sounds really boring until you realize that saves huge amounts of water and energy.

Even though materials does not sound exciting, that is what is happening right now. In the countries that you are from, when you go back, that is mostly what you are going to find. Don’t be put off by it. There is plenty of profit to be made in those areas and some very exciting applications also. To find out about it, you can use standard industry information sources. It is not that hard to find out. Compared to some of the other things I am going to tell you about, the benefits and the challenges of the materials timeframe are relatively modest. There still are huge benefits and big challenges, including legal challenges.

The phrase you will hear is “nanoparticles.” These are chunks of matter of some kind, which is in this roughly 1 to 100 nanometer size range. These are called nanopartilces. They really do act differently from larger chunks of stuff. They have new physical properties, and they need to be treated as new entities in terms of regulation, and that is not happening yet. The process of figuring out how to regulate these things is just starting, but it is starting. We are doing better on this than we have in the past on some technologies, but not really well enough.

That is the near term, what is happening right now. Then we move out into the mid-term. Now we are not just looking at simple materials—we are looking at devices. Sensors which can sense the environment in many different ways. Actuators, which are things that move. Active materials—a material which does things, which changes shape, for example. Many projects are involved with trying to read DNA. That is going to be a big application, and that is coming for sure. It is just a question of who is going to win that race. Targeted drug delivery, so that things like chemotherapy agents go where they are supposed to go.

If any of you have had relatives who have gone through chemotherapy, you know it is a miserable, horrible process, because the poisons that are going into the body are attacking healthy cells as well as the cancer cells. That is why things like your hair falls out—the whole body is under assault. When you do chemotherapy, the hope is that these toxic agents will kill more cancer cells than healthy cells, but it barely works. That is why it is such a miserable process. If we could find a way to make sure that those agents only killed the cancer cells, it would be fantastic.

That is coming along. I have seen some impressive results. How would this work? There are a lot of different mechanisms being explored. I’ll just give one example. You have a nanoparticle of silica, which is basically sand—silicon dioxide. It is coated in gold, and then they functionalize it—they attach to it some kind of biological molecule that likes to attach to cancer cells. Throw those in the body, they attach to the cancer cells. Now what? It’s very clever. Someone realized if we shine near IR light on the person’s skin near where the tumor is, it will go through the skin, heat up the gold, and the heat alone will kill the cancer cell. It’s technically elegant—a very clever thing.

That is the kind of thing people are working on. So, how do we get these active agents to only hurt cancer cells? My guess is, given the age range of the folks here in this room, it is very likely that although today in the U.S. cancer is the number two killer, by the time your generation is hitting those ages, it may not be the case. Something else will come along, I’m sure to fill that place, but it may not be cancer anymore. That is a huge change. It is something very exciting and hopeful to look forward to in your generation.

I think these are coming along fast enough and are looking exciting enough that it would not surprise me to see one of these approved in the U.S. within ten years. Sometimes they will fast track something that is looking really exciting. Well, these are looking pretty exciting, and a huge amount of money is going into this. Now, there’s a change.

If you are interested in this timeframe and want to know what kinds of things other than targeted drug delivery might happen, it is hard to find predictions in this timeframe. You have to be gutsy to make them. One of the few organizations that does make them is the Institute for Soldier Nanotechnologies at MIT. This is not offensive military, this is defensive military. It is not classified work—they’re pretty open about what they are doing. They are doing things like working on uniforms that will help a soldier who is injured. They talk about boots that you could wear that would let you jump twenty feet in the air. They don’t talk about what happens when you fall down twenty feet to the ground.

This is when we talk about active materials. These are materials that have energy stored in them that can be released. It sounds like science fiction, but these types of applications can give science fiction-type results. If you are interested in this timeframe and want to get a feel for what kinds of things are coming, you can look up Institute for Soldier Nanotechnologies on the MIT website and read about the kinds of things they talk about.

It is very hard to predict what is going to be the order of innovations that arrive in this timeframe. This timeframe will see important economic and strategic changes. You can see that the kinds of things I have been describing from this institute are defensive technologies, but you can imagine offensive technologies as well. You can imagine disruptions to the world’s global balance of power once countries start developing offensive applications. It is not a pleasant topic, but something that could very much affect the countries that you are coming from.

That is the mid-term. Then we get into the longer-term. Now what we are looking at is the original vision of nanotechnology, which is being able to precisely control every atom in an object. This is a very ambitious technical goal, but this was the original dream of nanotechnology when it was first envisioned. This gets very dramatic in terms of results. One of my favorite applications for this is the fact that if you can manufacture things using this kind of technology, there is no excuse to have waste anymore.

Today, when we manufacture things, and this is going on in all of your countries right now, there is something left over at the end of the process. It gets dumped in the air, the water, and the landfills. There is leftover stuff of some kind that is not recycled. The goal is to get away from this. When we have this level of technology—when it is cost-effective, as it will someday be—there will be no excuse for it technically anymore. No more dumping in the air and the water.

For those of you with an environmental orientation, which I would bet is everybody, that is a very inspirational dream. Some of the countries we are talking about here are in the developing phases. Is there a way for them to leapfrog away from this dirty phase of development. Can we skip that and go right to something cleaner? This would be the vision and the dream.

Other countries you mentioned are more developed. There are often in that case problems with environmental degradation. With this level of technology, you could go in and clean that up. You could say, Not only are we not going to pollute in the future, we are going to go back and clean up what’s been done in the past. To me this is a very inspirational vision and this is what drove me to take an interest in this field from the beginning.

Another consequence of this in the long-term, and I think this will be true in your lifetime, is what would happen if we have atomic-scale control of our bodies. In the mid-term timeframe I was talking about nailing cancer. No more cancer cells, so that people do not die of cancer anymore. With this level of technology, you move on. What’s left? What kind of disease could a human being have that could not be addressed with this level of technology eventually? It’s hard to say.

I suppose a massive accident—you’re hit by a truck. That could be the end. But in terms of disease, where there is time, it’s not clear what kind of disease you could have that would not be treatable with this level of technology. Basically, you want to put the atoms of your body back in a healthy arrangement.

What about aging? Aging, as we know, is a very complex process. There are many things that are all going on at one time, but all of them are misarrangements of atoms. In principle, if you could figure out what a healthy pattern of atoms and molecules in the body is, you could then restore that pattern. The goal, of course, is that you not disrupt the structure of the brain, because that is where your memories and your personality are. I don’t mind if everything from [the neck] down is rebuilt—start from scratch, make it all perfect. But up here [the brain] we need to keep the patterns.

Ultimately, I think aging is addressable. When that happens is very hard to say. Even before nanotechnology tries to take on the question of aging, there are biological approaches, that my colleague Susan [Fonseca-Klein] can tell you about, since she works with the organization that is doing these. They may nail the issue prior to the arrival of this more ambitious technology. If they cannot really solve all of the problems, eventually some day this type of thing should do it.

This is a picture of something that has not been physically built already—this is a model using chemistry software, showing a type of machine. This is a very simple bearing. We make these all the time at the macroscale. In this third stage, we are building actual machines down at the molecular level. There would be things like this bearing down there, working away.

This is called a planetary gear. For those of you who know something about cars, I have been assured many times that we have planetary gears in our automobiles. This is a cut-away version—you are seeing half of this gear—and when I click on it, you will see it moving. These things are rolling. Again, these things have not been built at this scale—this is a model using the best chemistry software. We think, based on that, when we are able to build these things, that they will work. This is this third stage. I think you will see this sometime in your lifetime based on the age range in the room.

This schematic is trying to get across the idea of building with single atoms. The green atom is supposed to be a hydrogen. It is being transferred from the upper tool piece down to the final product on the lower level. You can imagine this as a factory system. It is like a little assembly line. Again, a very long-term goal, but as far as we can tell, there is no reason why it should not work.

For you, as lawyers, whether you are working in companies, for government, or on your own, there are many legal issues raised by these types of technologies. I have a list of five here. The first one and the last one are immediate. When you go home, these are probably happening in your country right now. You will have to be prepared if you are going to be addressing these. The other three are things to be aware of. You will see them eventually.

The one big issue that is happening right now with these nanoparticles and nanomaterials right now is that there are concerns and issues regarding environmental health and safety. These materials act differently from what we are used to. You can take the same atoms, you make them into a particle at this size range, and it acts differently. That is often a good thing, but it could be a bad thing. Often, we don’t know. We have some preliminary testing data that indicate there is some potential for environmental or health concerns. We have other test data that seems to show no problem. So we have a mix of data.

When you buy these materials, here in the U.S. and probably in your countries, it comes with a safety data sheet that tells you how to handle the material. If you buy a nanomaterial now called carbon nanotubes—these are tiny tubes as you can tell from the name, made of carbon. They are a little bit like graphite in their structure, but they act completely differently from graphite. There is almost no connection, but if you buy it in the U.S. now, the safety sheet you are provided is for graphite, because they are both made of carbon. This is not just misleading, it is completely wrong. We are not seeing the level of attention for this from any of the entities in this process that we need to see.

Are there consumer products that have these in them? There are consumer products that have these in them, but they are bound. Things like sports equipment. They are not loose—they are bound into a matrix. The issue there is not that they might get into your body. The issue there is what happens to these things in the landfill. That is more of an environmental issue.

The public expects government protection. Both in the U.S. and in your countries, that is what the public expects. The companies who are making these products want the government to come up with some rules that will give them a safe harbor. You, being attorneys, understand the concept of safe harbor much better than I do. They want to be protected so that they know, here are the tests that we have to do, and if we do those tests and they all come out well, there will not be huge judgments against us, because we have followed the proper procedures. That is what companies want and feel they need. The governments are not providing either the protection the public wants or the guidance the companies want. They are not giving either one.

Unfortunately, some companies are not acting responsibly. We see already the big insurance companies, especially the reinsurance companies, are very concerned about this. Currently, they are not excluding these products from their insurance policies. If you are working for a company, you have some kind of insurance policy that is trying to protect you from these kinds of lawsuits and the reinsurance companies that are ultimately going to pay are nervous about this. They are trying to figure this out and are trying to nudge the governments into stepping up and providing the kinds of guidance that the companies need, but it is not happening, and they are getting very nervous.

I’m going to give you a couple of examples. These are the kinds of things you want to keep an eye on in your own countries when you go home. You do not want to work for a company who does this kind of thing. There is a company in Asia—I did not list the country because I thought maybe someone here might be from this country. What they are doing is they are selling a washing machine product that puts silver nanoparticles in the wash water to kill bacteria. Now, there might be benefits to this. You can use lower temperature waters, save energy—that’s a good thing. The problem, of course, is that then you have these silver nanoparticles in the water, and the water does not stay in the washer. It goes out into the sewage treatment plant and eventually gets discharged into the natural environment.

What happens to these bacteria-killing particles? As we hopefully know, many bacteria are good bacteria. We don’t want to kill all the bacteria—that would be very bad. So, what about that? The company has been asked about it, and they say, “We did testing, and it’s fine.” But if you look on their website, you cannot find anything about it. Frankly, until I see better data, I’m not buying it. It would have to be pretty thorough data, because when you think about where these particles are going to go in the natural environment, they are going to go everywhere—just as water does. Once it gets out in the oceans and the lakes, it is just going to be everywhere.

This product has already been withdrawn from the company in Sweden based on some consumer concerns. These concerns are spreading. I believe that the U.S. recently decided to not crack down on this washing machine product in a regulatory way. I would have to question that decision, but this is the kind of thing when you go home, your countries are all going to run into this. Even though you might not be technical folks, you have friends who are. You have consultants you can call in to advise you on these things. You should listen if they say this is not clearly a good idea. The less developed your country is, the more you are going to have to watch for these things. You do not have the infrastructure in place to keep an eye on this stuff. The U.S. is having trouble. Sweden is the only one that has really caught this one.

Another example was a European firm. I mentioned carbon nanotubes in a tube shape. These are carbon in round shapes. It is basically sixty carbon atoms in a soccer ball shape. They are called buckyballs after Buckminster Fuller, who had to do with these kinds of shapes. They have powerful properties. For example, they are great anti-oxidants, which can be very good things. However, they are very strong. Some data has shown that these things may have health concerns. There is a company in Europe which is putting buckyballs in face cream. You may not be aware that cosmetics are poorly regulated, certainly in the U.S. Apparently, that is also the case in the country that this is being made in. The benefit of whatever this face cream may be doing does not sound big enough to balance the potential risk of these powerful chemicals. Let’s say a woman puts this face cream on her face, she has a baby, the baby is getting this in its mouth… is this really a good trade-off?

There are concerns about this product. Can these buckyballs cross various biological barriers? There are some data that seem to indicate that some nanoparticles can cross the blood-brain barrier. That could be a useful property if you are trying, for example, to treat brain cancer. It is hard to get things into the brain. That barrier is pretty good. If you want to get a chemotherapy agent in there, you have got to get something that is going to get past that blood-brain barrier. These may do the trick. In general, we do not want things to go past that barrier. Certainly we do not want buckyballs to do that. There are some nanoparticles that seem to be able to do that. Is it worth the risk to put this in a face cream? I would say absolutely not. It took awhile for scientists to start to say that, but they have started in this country to ask why this product is on the market. Meanwhile, it is on the market. You can buy it right now.

This is the kind of thing that is happening because companies are expecting government to regulate, and government has not stepped up yet. The reason is that there is so much uncertainty on the technical side. Let’s say you go home and take a position in your government. They call in the technical experts, and they say it is too early to know, what do you do? It’s a very difficult situation.

We can come back to that, but I want to move on to a mid-term concern. Right now there is a lot of surveillance happening. It is mainly video cameras and sound equipment. These kinds of technologies can be used to control populations by governments. That can be a concern in countries where the government is not representative and may be oppressive or authoritarian. I am not going to speculate as to whether any of you are from those kinds of countries, but they are around and already using surveillance technologies. What is the next step? With nanotechnology you can do more advanced chemical surveillance. You can detect single molecules with these sensors. That is the best chemical detection you can ever have, and they are already building these things.

The military of any country would probably be the first to get these. Then, non-military government, like police. Then commercial, and then individuals eventually, as the prices come down. That is a speculation on the order in which these things might roll out. Who will be allowed to have these devices? Some countries have rules about what individuals are allowed to own. In the U.S., I was shocked to find, there are long-distance audio recording devices that individuals are not allowed to own. You can buy them in England. It’s legal there, but not in the U.S. I was surprised to hear that. You can ask what will be allowed to be sensed and recorded. Where will these things be allowed to be used? For example, in the future, ten or twenty years from now, if we have this class then, everyone in this room may be wearing these recorders and picking up all kinds of things. They could tell, if we were to shake hands and used my recorder, I could say, “Did you have fun eating that pizza last night? And what were you smoking?” These kinds of questions come up.

Who owns this kind of information when the government collects it? What about when a company collects it? When an individual collects it? How long can it be kept, how can it be used, can it be used in court? This would be an issue for you. Often there is data that is known, but it cannot be used in court. Could it be used in job screening? All these legal issues are going to have to be resolved one way or the other. They may be resolved differently in your country, compared with the U.S. I mentioned food sensing. What have people been eating, what drugs have they been taking. DNA sensing—many people would like to think their DNA information will be kept private, but it is important to realize that you leave your DNA all over the place. When I drink from this water bottle, I’ve left DNA right here. When I leave the room, I may leave this behind. I’m certainly not going to keep it with me for the rest of my life. The data is there, and whoever has one of these DNA readers in the future will have access to that data.

If people still have this idea that DNA information is going to be deeply secret, I just do not see how that is possible. We leave it everywhere we go. How will society decide what can be done with that data is another issue. Each of your countries may decide this question differently, but decisions are going to have to be made. You may very well be involved in those decisions.

In the longer-term—I believe this is within your lifetime, but it is not near-term. We have to ask ourselves how will these powerful technologies be used in offensive military use. The way to think about it, when you ask what would an offensive nanotechnology weapon be like—think of it as being harsh and tough like a chemical weapon, which are awful things, but controllability and targetability. You could imagine it combined with a DNA readers, for example. Controlling these, arms control of this type of technology, would be very, very hard. We have been fortunate with nuclear weapons because much of the arms control surveillance can be done from space. You don’t even have to go into the country to do this kind of survey. You can do a great deal of it from space, it is non-intrusive, and we have been fortunate for decades without having more nuclear wars happen.

That is not going to work for nanotechnology, or biotechnology-based weapons. To detect this kind of weapon, you would need very thorough inspections. Look what happened the last time we tried to do an inspection on a country that did not want it done. There are approaches for this that have been discussed that we can get into. What does it mean for today, and for your countries when you go back home? We need to practice on this and start to get better at regulating and controlling chemical and biological weapons. These things are out there now. We are not controlling them now, anywhere. If we want to have any hope in the future of these more challenging control problems, we should practice right now and get good at these types of controls because they are similar. We need to be able to work out how countries are going to reassure each other that they do or do not have these things, that they are or are not being properly protected, and that they will not fall into the hands of terrorists, or maybe will not be built at all. We can practice right now on these existing weapons because the problem is very similar.

Sometime in your lifetime I expect these technologies to be used for human enhancement. Someone mentioned, what about aging? If aging could be addressed, that would be an example of human enhancement. It is important to realize, we already do enhance human beings. There are many pharmaceuticals that already change personality. We change our physical appearance already. I think that you can imagine extreme changes—improved senses, maybe new senses, better memory, recording abilities. After all, if you can have a recorder of this size, you can have a recorder inside the human body that could do all the same recording. How does that change the legal situation? We come up with legal rules that define how government can get a hold of the data in my personal computer, and they generally can do it. There are many circumstances when they can get the data. What if it is inside my body? How does that change things? I think it does change things.

Control of emotions and personality, increased concentration, you can imagine all kinds of changes. Some people want this type of thing to be illegal. They point out, for example, that if it is legal you will have increased inequality. That is certainly true. Of course, it is unlikely that all the countries represented in our room here will agree on what should be legal and not legal. For example, the religious backgrounds of our countries are very different. There are already countries that are taking a different position on things like cloning and other early procedures that are likely to propagate into the future. Countries are going to have different laws, which means you are going to have citizens traveling to have procedures done in other countries. We already see tourism for cosmetic changes. That will certainly continue. We have to ask, how effective can these laws be to make it illegal if you can just go overseas and come home? Are these countries going to say you cannot come home? I would assume that is a ridiculous concept, and yet, without that kind of rule, this idea that it is illegal is not going to fly.

I would argue that the effects of trying to make these procedures illegal could be worse than the effects of just saying this is not something we can control. We will see how people will feel about that. I think there are big cultural differences represented around this table, and I for one am not going to try to take my preferences and enforce them in your country. I hope everyone in my country will take that position. It is not our job to go around and tell other countries what they can and cannot do with their bodies and the bodies of their visitors. I hope that other countries would take that position as well, but I don’t know. I think this is going to be very controversial, and I think there are going to be a lot of fights about it. Someday, some time in your legal career, this may be coming up. People may be coming to your country for enhancement tourism. I hope you make a lot of money off it.

In terms of being responsible about this kind of technology, we have come up with some scorecards—voluntary guidelines so that government, industry and individuals can get a feel for how they are doing. This is something that when you go home, if people ask for your assistance in being responsible in their nanotech development, whether it is government or industry looking for help, you can say—this is a place to start. These scorecards are a self-assessment tool to get some ideas on how to move forward toward safer development. Your comments on improving these would be helpful, especially if you, in the future, you are involved in this professionally. At that point, you may have some very helpful comments about this.

Some of what I have talked about sounds like science fiction. My point here is that I am not trying to talk to you just about what you are going to find when you go home at the end of your program. I’m trying to talk about what is going to happen throughout your career. You are going to be doing this for decades. So I am trying to give you a feel for some of the things that could happen. I, in my own career, have seen the world change from no internet to today’s internet. That has been a huge change and it has had a lot of legal consequences. I was very fortunate, because in the early days of our organization we looked ahead and saw something that looked like the internet at that time. You can see that in the book that founded our organization—it is described there. When it arrived, we were ready. We said, “Wow, here it is.” We jumped in and took advantage of it. You can do that, too. You do not need to focus too hard on this—just have it in the back of your mind that we had that discussion at Stanford, and here it is. Here is what we expected.

If it seems like science fiction, if you are looking far ahead, it has to seem like science fiction if it is going to be true. It is hard to realize for people who are younger, but the internet sounded like science fiction not long ago. Don’t let that bother you—you have got to get past that.

One last issue—patents. This is a near-term one. This is one you will hit as soon as you go home. If you look at the major technological revolutions that have come through—information technology, biotech, in the early days of those there were some innovations that were not patented or that were patented and opened up somehow. That is not true for nanotechnology. By the time nanotech started, the patent system and the patent procedures were very tightly in place. Pretty much everything has been patented from day one. We have fundamental, basic building block technologies which in other industries may not have been patented. They are patented in this set of industries.

When an innovation is made by a professor in a university, which is where much of this work is being done, there is an act of legislation here in the U.S. that assigns the patent rights to the university. That may or may not be the case in your countries, but many countries have copied this practice, and more and more are starting to do it. Regardless of what is happening in your country now, this may be the way it is done in the future. What this means is that these patents end up under the control of a university technology transfer office. These folks are not necessarily good at finding ways to most efficiently license these patents. They may not get licensed at all. They may take a very broad patent and license it exclusively to a narrow application, and the company they license it to may not have the time, especially if it is a start-up, to look for other applications or even deal with inquiries from other people who may want to use that patent.

You can see that although many people you talk to say that the system is working, that is not clear. If you get the opportunity to participate in shaping your country’s patent system, and you may, I would not assume that the received wisdom of just doing it the United States way is necessarily the best way. It is not clear that any country has found a better way, but I am pretty sure that this is not optimal. I hope you folks will innovate and take whatever system that you think is the best and then continue to evolve that, because I think this is not working optimally here in the U.S.

There is another problem of overlapping patents. The patent offices are supposed to try not to do this, but the fact is that it is quite common for patents that have been granted to overlap. It happens all the time. It can get to the point where, for a company to try to make a product, they have to try and get through a patent thicket—a bunch of intermeshed patents where it is just impossible to license everything you need to do. This is going to slow down innovation, it is going to cause problems for all the companies involved, because it can be hard even just to figure out what is out there.

The Bayh-Dole legislation here in the U.S. allows the government to “march in” if you have a problem of this kind. But this march in right, I have been told, has never been used by the government. If this interests you, and you want to try and prevent it in the country you are going back to, there is an excellent paper by a Stanford law professor here by the name of Mark Lemley. If you ever get the opportunity to do some innovation in this area back home, pull out that paper, call up Professor Lemley, and get some insights into what you can do.

Other problems in patents in all areas, but certainly in nanotechnology, we have got patents that should not have been issued. We all know that patents are only supposed to be issued for things that are non-obvious. It happens all the time that something that really is pretty obvious gets patented. Here is how I think about the problem. You have government patent offices and they have patent examiners. These are folks with law degrees who do their best to try to prevent bad patents from going through. On the other side, you have extraordinarily well funded law firms with very well paid attorneys, very large numbers of them, who can put in immense amounts of time to try to get something through which perhaps should not be gotten through. It is either too broad or too obvious, but there is tremendous resources on one side and on the other side you have an underpaid patent examiner who has four hours per patent application. It is very clear that this is not an even situation.

There is an arms race going on and there is one side that is terribly crippled, especially since if someone is a really good patent examiner, and he or she would like to move forward in their career, what are they going to do? They jump ship. They are very valuable people, because they know how the system works—they have been inside. If you are running the patent office, either in the U.S. or in your own country, how do you hold good people? I don’t think you can. They have to be very public spirited to keep working under tremendous pressure for poor pay. So there is an arms race going on with one side crippled. How do we even out the sides so that the patents that come out are not so poor quality?

There is some hope, fortunately. There is something called “peer-to-patent project” that you can type into your search engine. There is a project to pull in outside eyes, because the patent examiner needs some help. He or she is overwhelmed, but there are a lot of folks outside the patent office who have a financial interest in preventing the bad patent from issuing. These are all the competition to the folks who are submitting the patent, these may be professionals in the field who are irked by the fact that patents are so poor quality, who would love to participate in the process and try to look at patents that are about to be issued. This peer-to-patent project has already been instituted and is already an experiment within the U.S. patent office. If it goes well, I hope it will be put through as a standard process. You might keep an eye on it. If you think it is working in the U.S., you might consider instituting something like that when you go home, or at least suggesting to the patent folks in your country that you are never going to win against the kinds of odds you are up against—we need to get more eyes on these patent applications.

As AI developers are convening in San Francisco this week for the Game Developers Conference, another artificial intelligence conference is wrapping up in Arlington, Virginia, a short walk from the Pentagon. AGI-09, the second conference on artificial general intelligence, brings together researchers attempting to create learning, reasoning agents with broad, humanlike intelligence.

Organized by Dr. Ben Goertzel, chief science officer of Novamente LLC, the AGI conference series is a motivated effort to steer research back in the direction of the original intents of AI, namely to make a thinking machine. Goertzel’s plan is to inch up the cognitive ladder by incrementally developing more cleverly adaptive pets in virtual worlds and massively multiplayer online games.

>> GameSetWatch: The Merits of Novamente’s Parrots and the Arrival of Advanced AI

When cheap, advanced sensors give rise to ubiquitous monitoring technology, there will be the potential for what David Brin in The Transparent Society and others have called “sousveillance” to become universal. One could envision a future in which everyone was monitoring the activities of everyone else.  At the AGI-09 post-conference workshop, Ben Goertzel presented on a paper with Stephan Bugaj on various scenarios resulting from a future of advanced artificial intelligence that includes sousveillance technologies.

The following transcript of Ben Goertzel and Stephan Bugaj’s AGI-09 presentation “Is Sousveillance the Best Path to Ethical AGI” has not been approved by the speaker.  Video and audio are also available.

Is Sousveillance the Best Path to Ethical AGI?

I am now going to give a kind of science fictional talk moving out even further into the hypothetical future.  I have given talks before on similar issues to what the last two talks were about.  Stephan and I gave a presentation at last year’s AGI conference on stages of ethical development for logic-based AIs.  I did not think of anything new to say about that topic for this year’s workshop, so I decided to share some thoughts we have been pursuing about a related topic, which is the intersection of advanced AGI technology and other possible technological developments.

Often when extrapolating what will happen in the hypothetical future where we create very powerful AIs, we tend to ask “What if everything else was the same?”  That is a sensible way to investigate things, varying one parameter and seeing what happens. There are a lot of other interesting changes that could happen in the future of humanity as well.  It’s interesting to look at some of the intersections, what happens when advanced AI is combined with other things that could occur.  What we are talking about here is the intersction of AI and what David Brin and other thinkers have called “sousveillance.”

This is a variant of the traditional concept of surveillance.   The Big Brother is Watching You-type future scenario is one where the government or the galactic megacorporation is watching everyone and everything you do, whether for your own safety or for their own ends.  Sousveillance is a tweak on that idea, where basically everyone is watching everyone else.  David Brin, in a book I think is really excellent, called The Transparent Society, posits that given the development of surveillance technologies, pretty much the only options we have are the powers that be watch everyone or everyone watches everyone.

The argument he puts forth is that privacy is going to be dead, one way or the other.  He figures we are better off with everyone watching everyone than just having the government watch all of us.  If everyone watches everyone, then you can watch the watchers. There is a bit more safety potentially there.  If you go even a little further out into science fiction, you can talk about really strong sousveillance, where you watch other people’s mental states as well.  That gets far out with humans.  When you are talking about AGI systems it is quite possible that you could demand that any AGI created make its mental states open to inspection by other AGIs.

That raises a lot of interesting questions.  Even if you can see into its mind, can you understand what is happening?  I am going to talk a bit about what may happen when you put AI together with this sort of universal surveillance that may be part of the future.  One of the interesting things that comes up when you think about this in a psychological sense is, even setting AGI aside, what would this do to people?  Imagine, hypothetically as a thought experiment, you had a brain chip (a cranial jack, like in William Gibson‘s novels) that could read what thoughts were in your brain and project them to other people.  What effect would that have on us?

What if you came back from a late friday night out and you wouldn’t let your wife tap into your cranial jack?  Is she going to become suspicious?  What if you won’t let your boss look at what you are thinking?  How much more efficient could a software team be if you could share thoughts of a certain nature?  Maybe you would want to filter out certain thoughts and share other ones.  You could make an argument that the nature of the human mind would change dramatically once you had this kind of technology that the construct Thomas Metzinger calls in his book Being No One the “phenomenal self.” This is the image we make up of ourselves and guides our interactions. Maybe that will either go away or radically change with this type of technology.  Of course, we don’t know.

There is a possibility that you could have a new kind of cognitive agent, which I’ve called a “mindplex.” This is a group of minds, maybe human minds, that are very tightly coupled together. Each has its own individual consciousness, maybe with some kind of collective consciousness at a higher level, something that we cannot have now because of the different technological substrate. What I have just said about humans you could envision with intelligent AI systems all the more readily, because the cranial jack essentially is already there. You can read out the internal representation of an AI system to other ones. You get into interesting questions there about sharing cognitive content between AIs of similar versus different designs.

What happens when you put AGI and sousveillance together? The first, most obvious point is that AGIs may be necessary to enable really effective sousveillance. We already see the inklings of this in what is facing the U.S. intelligence community and other intelligence agencies. They are gathering a lot of information about all of us right now. Part of the reason Big Brother is not here yet is that mining all the information that they are gathering is really hard. Look at all the security cameras watching us everywhere, all the information about our credit card transactions and on Google’s databases, which is most likely being piped to other repositories run by the government. Why are they not doing more with that? Part of the reason is that they may not be as evil as some people say. Part of the reason is surely that mining through all that knowledge is really hard.

We all know enough about AI here to know how hard it is to find relevant information in a huge, heterogeneous knowledge store. Speech-to-text does not work that well, search engines do not work that well—we are still stuck on keyword search—image search does not work that well. If all those things really worked, Big Brother might be here already with the existing surveillance technologies. Arguably a powerful general intelligence, or even the right kind of combination of advanced narrow AI technologies, could enable both surveillance and sousveillance.

We can see the start of a phenomenon with things like Google Street View. There are cameras and you can go to Google Maps and see who is walking on the street in some places, and I think they did something to obscure people’s faces because it seemed to be an invasion of privacy to watch people walking around. You could counter-argue—is it really an invasion of privacy? If you are walking down the street, everyone can see you. How is it different from seeing you through a camera?  That is just going to get worse and worse.

Other interesting options are, with the cranial jack scenario, you have the possibility to plug into Google, plug into Mathematica  (which would make theorem proving a lot more fun). If you have an AGI theorem prover, a mathematician will be impaired if they cannot hybridize into a cyborg mind with the AGI theorem prover. That makes for AGI + human mindplexes.

Then there are all these questions of AGI ethics. What happens when you make smarter and smarter AIs? Could better surveillance technology help with this? You could inspect the AGI’s mind as it gets smarter and smarter and develops the desire to annihilate you, then perhaps intervene and protect against it somehow.

When you follow this train of thought a little further, it gets kind of interesting. The most interesting scenario to think about, consider the case where there is sufficiently advanced sousveillance for a bunch of minds to see what each other are doing, and there is relatively symmetric practical power among the different agents.  You come to some interesting conclusions. If you have a situation where one agent is massively more powerful than everyone else, it does not particularly matter whether you can see into their mind or not. If I am much stronger than you, and I have a huge weapon and you don’t, and you can see that I’m coming at you, and can even see in my mind that I’m thinking of clobbering you over the head, if you do not have the power to stop me it does not help you very much. If you have a collection of agents that are roughly equally powerful, you get potentially some interesting dynamics, given universal surveillance.

I think that there is a real issue here in terms of conformism versus innovation. You can see a bit of this in the financial markets. If you have a situation where everyone can see what everyone else is doing, if there is no one who is too different from anyone else, it is probably a fairly tractable problem to make sure no one gets too far out of line and does something nasty. If you imagine a community of 10,000 very smart AGI systems that can monitor each others’ cognitive states to some level, if they are operating fairly similarly they can probably tell if one of them is getting out of line and starting to develop dangerous tendencies.

On the other hand, if one of those guys is thinking according to very different algorithms and is just behaving in a way that is unpredictable relative to the other ones, the other guys are not going to be able to tell what it is doing. I would say sousveillance combined with conformity potentially could provide a measure of safety. This gets back to these evolutionarily stable strategies. If you have a bunch of guys who are fairly similar and they all understand each other, if anyone gets too weird you clobber him. Monoculture survived a really long time before we developed advanced technology.

It is true that stability is not always the optimal state. Innovation generally results from non-conformity. Hyper-conformist societies can stagnate. There is a kind of give and take here between safety and innovation. This gets into the economics of future societies. The financial markets are an interesting analog here. I spent some time doing consulting work with hedge funds and other financial actors. I should say as a preface to this that I do not believe in the efficient market hypothesis. There are a lot of inefficiencies in markets, which pop up, and smart people exploit them. The markets are close to efficient, but not fully efficient. If you buy that, which is the attitude of most traders and quantitative finance people, the next step along that line is to gain an advantage in an environment where everyone sees the trades that everyone else is making.  Here it is important not just to be smart, but to be smart in a weird way… what I call “peculiar cleverness.”

Support vector machines are pretty smart, but there are a lot of people trying to do trading using support vector machines now, so the benefits to be gotten from that method are largely priced into the market. Feed-forward neural nets were priced into the market a long time ago, and before that linear regression was priced into the market. It is not enough to be really clever. In a case where everyone can see what everyone else is doing and copy it, you have to be clever in a weird way so that people will take a long time to copy it. In a situation where there is so much mutual observability among actors that are roughly equally powerful, what you need to prevail is to be smart in a way where no one else can understand what you are doing for a while.

That would be even more true in a community of general actors that were doing something besides financial trading. Imagine that all of us in this room were competing at some goal, like AGI research, and we could see what each other were thinking. Let’s say we were competitive and wanted to beat each other to the goal. How would we get there? If we thought in such a convoluted and weird way that even when the other guys could see our thoughts they would not understand what they were seeing, we would have a chance of getting there ahead of anyone else. Whereas if you were really smart but everyone else could see what you were thinking, the odds of beating everyone else would be lower. Peculiar cleverness is a big advantage in the face of sousveillance. On the other hand, non-conformity has to be squashed because it could lead to someone causing damage.

You have a really interesting dichotomy between freedom and security. Of course, we have that all the time, but it becomes much more poignant when you have this type of universal surveillance technology. Similar issues come up when you talk about the notion of mindplexes. That makes things even harder to track because, as I said before, the most interesting situation is when you have actors that are roughly equally powerful. If you have a mindplex where 500 people meld together into a Borg mind to think better, predict the future better and be peculiarly clever better, then what is the individual actor? Is it the individual guy, or is it the Borg mind? It’s hard to know which individuals seek an unfair advantage when you cannot tell who the individuals are. You come to the conclusion that mindplexing, fusing into group minds, would need to be restricted in this kind of future to prevent anyone from becoming more than x times smarter than everybody else.

There are a lot of potential future scenarios that could come out of this. The next few slides run through a few fairly obvious possibilities that could come about. I do not claim to know what the probability weighting is of all these different science fictional possibilities. One simple possibility is that we get to watch each other, and everyone gets kind of embarrassed, and we become a Puritanistic society without much innovation. I don’t think this is too likely, but you can see it as a possibility.

One possibility is that all the inhibitions and nasty impulses we have get dampened out because everyone can see what everyone else is doing. We realize that we can waste less resources by not worrying about it and see through each others’ games and ruses. Then we could get a sousveillant utopia.  That also probably is not very likely, but it’s interesting.

The panopticon, where basically sousveillance fails, is where large organizations can get superior monitoring resources. Basically sousveillance gives way to surveillance, which I am afraid may be a more likely possibility than the previous two. However, I do not place too much weight on my probability estimates.

A mindplex utopia is where basically individuality goes away and it becomes to everyone’s benefit to link into the global brain. This is an interesting scenario, which I also think is not completely unlikely. We think of it as giving up freedom to link into the Borg, because that is how Star Trek portrayed it. Ultimately, from a neuron’s point-of-view it may not have its freedom restricted by being part of a brain. We may be able to link into some higher-level global mind and still have a sense of individuality, freedom and personal satisfaction.

Of course we have the Borg collective, which is well known from science fiction. The mindplex level, in order to achieve advantage, becomes more and more restrictive of what the individuals can do. I would say we actually don’t know whether in a competition between mindplexes either the more Borglike mindplexes or the mindplexes allowing more internal freedom to their members is more efficient. We don’t really know that. It may depend on a lot of other factors.

Instead of the Borg collective, we could have the “bored collective.”  This is sort of like the first one but without all the Puritanism. The singularity kind of throws a wrench into this whole thing.  It could be that once we achieve a certain level of intelligence, whether individually or through the mindplex, as Vernor Vinge says, once something gets enough more intelligent than we are now, we are idiots to think we can predict what is going to happen.

Of course, another scenario that I’m afraid is not that unlikely is that something goes wrong with all these dynamics and we can watch very closely what some other agent is doing while it prepares to annihilate us, and then does so.

In conclusion, sousveillance is an advanced form of collectivism that we cannot understand very well right now.  I think it is a form that is fairly likely to happen, just as I think totalitarian surveillance-based society is also fairly likely to happen.  I’m not sure which one is more likely.  I tend to buy David Brin’s argument that with the advent of monitoring technologies, either sousveillance or surveillance is likely to be the future, unless we bomb ourselves to oblivion first.  If it is the sousveillance-based future, there are going to be real issues with preserving the right to innovate and non-conform.  This is an issue that we all should be thinking about.

There are huge benefits in terms of possible increased cooperation and harmony between people and also huge risks to the individuality and diversity.  Certainly in terms of the safety of advanced AGI systems, sousveillance provides no guarentees, but it does change the dynamics of the scenarios.  It is a different scenario in terms of AGI safety than the scenarios in which sousveillance does not exist.  The dynamics are different and they are very complex. When you spend time thinking about them, you run into an awful lot of uncertainties, which is hardly surprising considering that we can just barely think through scenarios where we introduce AGIs into current society without any other intervening variables, such as sousveillance.

 Ben Goertzel and J. Storrs Hall at the AGI-09 post-conference workshop

Following in the footsteps of AGI-08, the Future of AI workshop was held in conjunction with AGI-09. This year’s workshop, held Monday, March 9th, 2009, at the main conference venue of the Crowne Plaza National Airport in Arlington, Virginia featured a slate of invited talks as well as contributed papers and posters. The event was hosted by J. Storrs Hall, president of the Foresight Institute, and introduced the topic of the economics of advanced AI.

The following transcript of J. Storrs Hall’s introduction to the AGI-09 post-conference workshop has not been approved by the speaker.  Video is also available.

Welcome to the Future of AGI

I’m Josh Hall.  I’ve been doing futuristic stuff for awhile and have written a couple books, including one called Beyond AI.  I’m now president of the Foresight Institute, which I am hoping to steer in the direction of looking at the future of AI.

Without further ado, let’s look a little tiny bit at the subject of what the future of AI might bring. One of the things is that the same worries that people have had about automation ever since there has been automation is that if AIs get as smart as we are, and continue to get cheaper the same way that Moore’s law has made computers get cheaper, they will economically crowd us out of the labor market, even for intellectual labor.  People look at this and they say this is a horrible thing; no one will have jobs anymore.

On the other hand, if I built a robot to do my own work, I would want it to do my work.  The whole point would be so that I did not have to do my work.  Why can’t we, the human race as a whole, do the same thing?

If AIs are smarter than we are, won’t they take over and end the human era, as Vernor Vinge puts it?  On the other hand, if they are really smarter than we are, shouldn’t they be the ones in charge?  Why have dumb humans in charge if we have smart robots?

To look at it from another point of view, if you can build a machine that is smarter than you are, why can’t you build a machine that is morally better than you are?  As Ron Arkin put it last year at this conference, “It’s a low bar.”  If we can in fact build machines that are morally better than we are, don’t we have a moral duty to actually do that?

A few other things that we might try to think about today: are the robots themselves going to have rights? After all, we are talking about machines that are as intelligent as we are.  By owning them, are we not just reintroducing slavery?  If not, should they be able to vote?  After all, they are going to be getting cheaper and cheaper, and more and more numerous.  Near-term concerns such as legal issues: at what point do you start assigning blame to the actual robot, as opposed to the manufacturer who built it?  That makes a big difference.

People claim, for example, that the German companies that had the self-driving cars back in the ’90s never followed them up and introduced them to the public at large because they were afraid of being sued for every accident that the cars might have.  Even if the rate of accidents on the whole was considerably lower than the rate of accidents by human drivers, the company would stand to have its socks sued off.

Finally, the public perception of AI and robots has been very strongly influenced by alarmism in this movie industry.  You see these horrible, evil scientists with these giant labs and so forth.

Marcus Hutter:  If only!

J. Storrs Hall:  That is the same question with the robots.  If only we could have robots that can do this stuff that they do in the movies, when in fact the reality and the perception are quite different.  In order to get any good public decision-making on the issue, a certain amount of education is necessary, I think.

With that, I will turn the stand over to  Jim Albus, who is one of America’s leading roboticists.  We have two really excellent speakers on the economics of AI this morning: Jim Albus and Robin Hanson.  Let’s give a welcome to them.

The world’s democracies are set to face their gravest challenge yet as viable and ongoing political options. George Dvorsky, who serves on the Board of Directors for the Institute for Ethics and Emerging Technologies and Humanity+ while bloging at Sentient Developments, presented at the Global Catastrophic Risks conference in Mountain View on how given these high stakes situations, democratic institutions may not be given the chance to prevent global catastrophes.

The following transcript of the presentation “Risks Posed by Political Extremism” by George Dvorsky has been cross-posted from Sentient Developments.  Video and audio are also available.

Risks Posed by Political Extremism

We have had some great ideas today.   Jamais, Mike, Martin and others have put together some concrete ways in which we can use the institutions that we have at our disposal and go about dealing with existential risks in a way where we can still live as civilized human beings and not have our lives diminished appreciably.

What I am going to be speaking about today though is another path that we could take.  What if we do not set up a resilient civilization?  Aside from the obvious result of there being a global-scale catastrophe or outright human extinction, there is the path down to political extremism, which might be a natural consequence of the emergence of existential risks in the first place.  There is a double-edged sword here.

That is essentially the theme that will be at the base level of this discussion.  Throughout the 20th century there was a lot of political perturbations and restructuring that happened, largely driven by the maturation of the nation-state and industrial economies.  These states had to figure out very quickly how to manage a civilization and redistribute wealth.  You had a number of ideological forces coming into play to argue this exact point.  It explains a lot of the tensions that did occur in the 20th century, most dramatically in the form of totalitarianism.  Meanwhile, the democratic nations, who resisted this radicalism, were working to develop the welfare state and put Keynesian economics into practice.  Not everyone had to fall into these radical patterns.

Looking ahead into the 21st century, the politics and restructuring of our political institutions will be driven by the demands of mitigating existential risks.  In particular, managing the impacts of disruptive technologies and the threats posed by apocalyptic-scale weapons and ongoing environmental degradation are subjects we absolutely do have to talk about.

This restructuring is already happening.  We are living in a post-9/11 world.  That was an example of “superterrorism,” where it was a rather devastating event that impacted not just those immediately involved but on our psychologies and societal sensibilities.  We have seen what has happened in the seven years since then.  A reaction is happening.  Looking into the future, we can certainly anticipate that there will be more of this.

Even the term “existential risks” is slowly starting to seep its way into the popular vernacular.  Some of you might have caught during the second debate between John McCain and Barack Obama, McCain actually used the term “existential risks” during the debate, which caught me by surprise.  He was not speaking of course of humanwide extinction.  He was speaking specifically about the state of Israel and its current situation as it is being confronted with what it perceives to be an existential risk in the form of a nuclear attack from a rather belligerent Iran right now.  This I expect not only to enter into our parlance more frequently but to seep itself into public policy in a very real way in terms of our institutions and the ways in which we react to these threats.

This is a three-part presentation.  Existential risks will change the political landscape.  There is an ever-growing multiplicity of threats.  The next generation of threats are on the horizon.  Today, many of us have spoken about what exactly those are going to be.  What is worse is that there is an increased chance that there is going to be an increased chance of unchecked development and proliferation.  Given the nature of information technology today and the access to information, there is this increased threat that anybody, given the right information and the right resources, can put these threats together.  Further compounding that is an increased sense of motivation among some groups, whether done at the individual level or state level, to put these weapons into practice.

We are collectively speaking here about global catastrophic risks.  It does not result in humanwide extinction but it is an event of such devastation that each person on this planet will be impacted in some way.  When that happens we will be overnight put into a reactive state.  That will be an order of magnitude beyond anything that has happened since World War II, in terms of our entire structure having to be based around this reaction.

Another issue as we are dealing with these catastrophes, maybe politicians will lose faith in the kinds of remedies we are trying to articulate today.  Perhaps they may think that we are naive and that a rather more heavy hand is required. They may lose faith in the ability of democracies to deal with these particular problems and look for more extreme measures and more draconian answers to these problems.

The future may unfortunately not be exactly what we had hoped for.  At the end of the Cold War, you had this rather optimistic sense that things were finally going to change for the better.  You had this feeling that Western liberal democracy was in the process of triumphing and that free market capitalism was about to envelope the world, you had all this talk about the end of history, and a new world order.  However, this has not been the case. The last fifteen years have been replete with violence and has resulted arguably in a more unstable world from a geopolitical perspective.  We have hardly reached the sense of a new world order or an end of history.

In the 21st century, with the introduction of apocalyptic threats, it will not be politics as usual.  These times are going to call for more drastic measures.  The mere presence of existential risks will result in there being more political extremism.  This is a negative feedback loop.  Those who feel persecuted will have an added impetus to use weapons of mass destruction.

What kind of challenge does this pose to democracy?  Democracy is still the exception and not the rule.  Right now, according to some surveys, as little as 45-48% of the world can be classified as being truly free. This was a figure that was down to around 35% as early as 1973.  Speaking about cognitive biases today, we may actually be the victims of an ideological bias.  We have this idea that democracy is here and here to stay.  This may not in fact be the case.

We may actually be living in a rather extraordinary time in human history where the social and technological situation is such that we have reached an equilibrium where we can have the strong democracies and freedoms that we have.  We may be entering into a period of social disequilibrium where democracies simply will not be able to withstand the pressure of the threat of existential risks.  A big part of what I am speaking about today is the perceived need for extremism.  There will be an unprecedented need for social control.  That does not mean merely looking at people and wondering what they are doing, but getting them to do something or not to do something.

It can also be not only in anticipation of a disaster happening and getting individuals to work to prevent it from happening but it can be in response to an actual catastrophic event on the scale of World War II, where suddenly everyone in society is mobilized to  participate in disaster recovery.

The second part is defining and anticipating political extremism.  What do we mean by political extremism?  It is a relative term with no fixed political baseline.  It can be used to describe the actions or ideologies of individuals or groups that are outside of a perceived establishment.  The views and actions of perceived extremists are typically contrasted with what we would consider moderate opinion.  It is also used to describe those groups who violate the sense of there being a common moral standard.  Again, it does not have to be a group of individuals outside society, but could be members of your own government.

Extremists can direct their angst either internally, within their own group, or outside of the confines of the state, or both.  They will often be accused of advocating violence against the will of society and the actions are often considered beyond what is necessary.  Some might consider the actions of the outgoing administration of the United States in the introduction of the Patriot Act and warrantless wiretapping are extreme measures beyond the call of what is necessary.  The term is almost always used as a pejorative.  No one declares themselves to be an “extremist.”

What will give rise to various forms of extremism?  When times are good, you are not going to have agitation.  You are not going to have calls for more radical political action.  When you have economic, environmental and civil strife, that is when things get churned up.  There is an old revolutionary credo that the worse things are, the better.  That is the only time when people are going to be willing to do something about their situation.  Reasons for extremism at the state level are when the ends are felt to justify the means.  There is seen to be just cause to implement policies that reduce our civil liberties.

Radicalism begets radicalism.  When you had the emergence of fascism in 1930′s Europe, arguably the reason for it was that it was a reaction to Bolshevism and the threat of Collectivism.  You had radicalism establish itself in one spot, and others freaking out about it and deciding to embrace radicalism to counter it.  Given prescriptions for the 21st century on how we will address existential risks, you could see the same bipolar stratification happening with different forms of radicalism.  I cannot speak to them specifically right now, but perhaps later when I go over particular threats themselves.  You could envision a radical progressive group countered by a radical luddite group, for example, as they gain political power.

There is the issue of future shock, as well.  The idea that accelerating change will upset a lot of our psychological sensibilities is something that Émile Durkheim referred to as “anomie.”  Because things are changing so quickly, the public does not really know what to grab onto.  Their footing is lost, social norms are changing.  Today, of course, we are feeling it through such things as gay marriage.  Social changes are happening with greater rapidity than they have in the past.  I look at things going on at Google with a certain degree of reverence because I simply cannot imagine how they are piecing things together.

How the Nazis took advantage of this in the 1930s was by appealing to people’s sensibility and nostalgia.  While on the one hand they would tap into what technology had to offer, they would offer them comfort in values that they could relate to.  Accelerating technological change can also give rise to a call for radical action.  You also need a psychologically primed populace.  A catastrophic event will create a populace that is hysterical or primed for a strong central authority figure to tell them what to do and how to do it.  The regime will take advantage of that and scare the populace with threats in hopes of keeping them in control that way.  This is a common political tactic.  A psychologically primed populace will be both welcoming of and supportive of a regime that comes in and take away those civil liberties.

Despite the previous slide, where I went over conditions, there are drivers that will speak to 21st century politics.  How are we going to restructure our politics such that we will avoid wisespread catastrophes and human extinction?   We are going to need our political institutions deal with the managing of ongoing disasters.  Right now what comes to mind are the environmental disasters that appear to be looming on the horizon.  However, one could imagine a pandemic of some sort, a nano-disaster, not at an existential level.  Meeting the demands of managing these disasters and dealing with disaster recovery are going to be cost prohibitive to say the least.  As usual, the economy is going to be a huge issue in the 21st century.

There are some secondary drivers as well that may again lead to some more sense of instability.  That is the emergence of disruptive technologies.  That will have profound socio-economic effects that have been somewhat outside the bounds of this symposium.  There is nanotechnology, for example, and what it will mean to the manufacturing sector.  There is robotics and what it will mean to unemployment.  Then there will be dealing with the restructuring of society that will be on the scale of the previous industrial revolution that will happen because of AI.

Another possibility, one that could be considered extreme, is do-nothingism.  That is denial, underestimation and circumvention.  Today we are seeing corporate interests interested in obfuscation and disinformation.  For their own selfish interests they will see the world burn.  Political self-preservation, scared politicians that are simply afraid to do anything, may only be interested in maintaining the status quo.   It also might be an issue of human nature.  We are victims often of our own psychologies, denial or our inability to grok probabilities, and so on.  We may fail to realize a threat is on the horizon.

There is also the possibility of isolationism.  Countries that do not want to deal with this will not follow the path of meeting this threat.  Also, there is backwardness, simply not comprehending the issue at hand.  Look at Africa over the past thirty years now largely being in denial of the AIDS epidemic.  That is a massive disaster in its own right.

Moving on to assessing the various threats, I broke them down into authoritarianism, totalitarianism, paramilitary groups and radical social groups.  What are the drivers for an iron-fisted government?  I cannot stress enough both the positive and negative injunctions for making people act in a certain way.  A state that distrusts and is fearful  of its own citizens will pick quick, easy and lazy ways to deal with crisis situations, such as circumventing due process by tweaking the Constitution or putting it aside.

This type of regime can manifest itself through an existing democratic regime like the United States or Canada.  It is just a matter of putting the right tools into action to make it happen.  For instance, the Nazis were voted into power.  It could be a coup, a junta, an occupying force through the establishment of police states and dictatorships.  There was a coup d’etat in Pakistan in 1999 with Musharraf taking control of a nuclear-capable country.  What can an authoritarian state do to deal with the threat of there being a disaster?  It can declare a state of emergency, suspend elections, dissolve the government, ban all criticism and protests, reduce privacy and mobility rights, conduct illegal arrests and torture.

This would certainly be a threat in terms of the gross diminishment of our rights and civil liberties.  It would provoke reactions internally and abroad, working to destabilize the situation even further.  I think you could devote an entire symposium to the future of totalitarian threats. It is important to address, simply because it is an existential risk unto itself.

Why would an authoritarian threat want establish totalitarianism? It is the need for absolute social control, to mobilize the people and get them to think the way that is in the state’s best interest through the imposition of an ideological imperative.  This could be a religious imperative, or it could be regime based on getting itself back on its feet after a global catastrophe.  I think some of the same technologies that would work to enable totalitarianism in the 21st century would also work to undermine the instantiation of totalitarianism.  Namely, communications technologies could be use to hack the system.

This could manifest itself through an existing regime.  It might not be through the radical left or right as we know it.  It could be the emergence of neo-totalitarianism under new conditions.  The political tools would be the same as authoritarianism, but other things that they have at their disposal could be a monopoly on all political activity, on the ideology, on the means of coercion and the means of persuasion.  All economic and professional activities of the state would become subject to the state.

Paul Saffo is a forecaster and essayist with over two decades experience exploring long-term technological change and its practical impact on business and society. He teaches at Stanford University and is a Visiting Scholar in the Stanford Media X research network. He was the founding chairman of the Samsung Science Board and serves on a variety of other boards including the Long Now Foundation. At the Convergence unconference in November, he delivered a keynote presentation on the differences between forecasting and advocating for potential future outcomes.

The following transcript of the Convergence08 keynote presentation by Paul Saffo has not been approved by the speaker.  Video and audio are also available.

Mapping a Cone of Uncertainty

Welcome back from the break-out sessions.  I certainly enjoyed the ones I was at.  I had originally planned to offer my perspectives on what I thought was happening in this space, as I have been a friendly bystander to the whole converging areas of nano and bio for a long time.  I was just remembering when I ran into Chris this afternoon, I think the first time we talked about this stuff was we once spent a long weekend with a bunch of nano-heads up in the Sierras way back in 1983.  Ever since it has fascinated me.

I realize however that it is sort of an abuse of privilege on this stage to offer you my opinions when there are so many wonderful opinions in the group. I am going to step back and instead touch on some of the things that are happening in the short-term, but use them to talk about how I think about forecasting and offer some advice about how you all may want to consider thinking about this.

Let me begin with the question: Are you a futurist or a forecaster?  I draw the distinction thusly.  A futurist is generally someone who is an advocate about the future. They want to see a particular outcome.  In my work, I am generally a forecaster, consciously a bystander, stepping back and saying, “What is the full range of possibilities?”

Now, in my personal life I most assuredly also have the instincts of a futurist, because like anyone else I have some pretty strong opinions about how I would like things to turn out.  However, I can tell you from long experience, as a forecaster you have to be explicit about when you are thinking like a forecaster and trying to understand the full range of possibilities and then those moments when you say, “Okay, I am going to be an advocate here.”  The road to ruin in forecasting is to allow your opinion of what you wish would happen to interfere with your judgment of what you think will happen.

Especially in this field, which is so exciting, it is real easy to slide into that advocacy role, when it is probably a good idea to be an analyst.  Instead of being a participant, it is a good idea to be a bystander.  Keep that in mind as you engage in your conversations later today and tomorrow.  When is it appropriate to take your understandings as a forecaster and say, “This is the direction I think we all should go.”  Of course, at the end of the day, people who are not advocates for the future live the old parable: If we don’t change directions soon, we are most assuredly going to end up where we are headed.

The other aspect of being an enthusiast about the future is there is an odd tendency, call it Hitchcock’s law of futurism, to make the future more dramatic than it might be.  I invoke Alfred Hitchcock here, who said, “Movies are reality with the dull parts edited out.”  I can assure you after nearly thirty years in this business, the future will have long stretches of dullness in it.  The exciting parts will be exciting because they are brief moments, and then things will get dull again.

A danger that I see, and a lot of people do this, is when they look into the future they compress all the exciting stuff together, forgetting that in fact it will be dull.  Remember back in the early ’90s when the web arrived and digital technology was taking off?  The word Bill Gibson wrote about in his 1984 novel Neuromancer was cyberspace. This was invented by a novelist typing on a manual typewriter, of all people.

We were just giddy at the prospects of “cyberspace.” It was going to enlighten and ennoble mankind. We have a long history of this.  Around the turn of the century, in 1900, the phrase “air minded” was very popular among aviation enthusiasts. Somehow flying in airplanes was going to enlighten and ennoble mankind. It would be the end of mankind, bring world peace and… well, you get the idea and you know what happened.

Cyberspace is still around, but let’s face it, what we really got was not the sweeping exhilaration of cyberspace changing all our lives. I must confess parenthetically, at Stanford I am so jealous of my students, because those little twerps have Wikipedia and the web. They are so cunning at faking the fact that they have not done their homework. I am from the generation where we would plead with the librarian to leave the library open. Anyway, but I digress.

We thought this vast new cyberspace would be wonderful, while some people said it would be the end of civilization as we know it. What did we get? We got cyberbia. Cyberspace kind of looks like the real world, all the way down to crab grass. Some things are worse and some things are a little better. The future is probably going to look a lot like today, except we have a little bit better gizmos. Remember Alfred Hitchcock and his rule: there will be dull parts in the future.

Now as a forecaster, I think about the future as mapping a cone of uncertainty.  As I am standing in the present moment, witnessing some event that has just happened, I say, “How much uncertainty is there and how broad is that cone?  What lies inside it and what could change?”  Apropos of that, I encourage you to challenge all of your assumptions, every single last one of them.

For example… convergence?  Are we really, really sure it’s convergence?  New industries emerge from the intersection of old industries.  Maybe it is the cross-impacts of industries working together, or maybe it is the convergence of technologies leading down to singularity, but what happens afterwards?  Maybe things are in fact diverging, and that in this convergence there is also a lot of divergence.  Remember the last time people talked a lot about convergence, in terms of media convergence in the 1990′s?  They talked about how the whole media industry was going to converge into one super-industry.

What happened?  Convergence at a narrow technical level led to divergence in terms of markets and products, which is why newspapers are going out of business instead of controlling the world.  If it had been media convergence, the New York Times would own everything and record executives would not be in trouble.  In fact, things diverged in terms of the specific industries.  Keep in mind that it may not exactly turn out the way that we think.  A big reason is this other rule of thumb from my forecasting experience: change is never linear.

Nothing interesting is ever linear.  We all know S-curves.  The mother of all S-curves is Moore’s law.  Here we are in Silicon Valley, and what better place to talk about it than here?  The thing I see again and again, especially here in Silicon Valley, is people who live and die by the S-curve really don’t get it.  They really do not understand what it is like to be on an S-curve and what it is like not to be surprised by an inflection point.  The problem is that we are linear thinkers.  We tend to look into the future by taking a ruler to the past, turning it around, and drawing a straight line.

Inevitably, what happens is there are two kinds of people.  There are those who are surprised when the inflection point hits.  Like “Wow, where did the world wide web come from?”  Then there are technologists who pay the price of being wrong not once but twice, because they stand at the flat spot of that curve, long before the inflection point, and they say, “The future is so obvious. It’s just around the corner!”  Then they stand around waiting twenty years for it to happen.  Just before it arrives they go, “It will never happen at all!”  Then they walk away just as the fortunes are being made.

For the most part, these days one wants to make people understand just what inflection points feel like.  However, there are parts of this audience who have been “Waiting for Godot.”  Back in the 80′s, nanotech was just around the corner.  We have been on the edge of an AI revolution for 50 years.

The important thing to pay attention to is that flat spot.  If you want to look for a short-term success, look for something that has been failing for twenty years.  My rule here is “cherish failure,” especially when it’s someone else’s.  When you look at that S-curve, it implies more than technology.  Where did Columbus fall on this S-curve?  He was at the inflection point.  To put it more precisely, Columbus was not the first European to make it to the New World, he was the first European to make it back. That flat spot is paved by one interesting failure after another, and it turns out there were countless expeditions.  They never got a holiday named after them because they did not make it back.

Just as an example of cherishing failure and showing you that some things really do look like the curve of Moore’s law, some folks have talked about robotics today.  Robotics is one of those things where you can hear the Doppler whistle of the inflection point.

Here is a picture that I shot in May of 2004 at the first DARPA Grand Challenge.  Some of you who were there will remember it was a 150-mile course and there were 21 teams.  I was an optimist, part of a medical team sitting next to a helicopter halfway out on the race.  We felt kind of stupid because the thing looked like a Monty Python sketch. This robot in particular was very sweet.  It came out of the barrier, got to the exit point where the race started, looked left and right, thought for a second, hung a hard left and drove into a coyote bush where it died.

Here is the robot who did much better.  This is Sandstorm.  The answer to the question “Why did the robot cross the road,” is “To drive into a fence, of course.”  This is what Sandstorm did just shortly after that.  The robot that got farthest in the May 2004 race got exactly seven kilometers into the race before it slid off the road and died.  Everybody shrugged their shoulders and felt stupid.

18 months later we had the second Grand Challenge.  What happened in that one?  A very different result.  Not coincidentally, it was one doubling period of Moore’s law.  In October 2005, twenty-two teams got farther than Sandstorm got in the first race and five finished.  I’m also pleased to say Stanford won… though the truth is that Carnegie Mellon lost.

The real difference in philosophy—Red Whittaker, who is the head of the Carnegie Mellon team, is a former marine.  Let me correct that.  There’s no such thing as a former marine.  He likes hardware.  If you have a dollar, you are better off spending it on hardware.  The leader of the team at Stanford, Sebastian Thrun, who we stole from Carnegie Mellon, says superior software will always compensate for inferior hardware.

The Sandstorm team was crushing the Stanford team until a bolt came loose and the LIDAR unit on top of Sandstorm started bobbing, which caused Sandstorm to crawl to a walk. Stanley, the Stanford bot blew past because, let’s face it, software rules.

The third Grand Challenge is especially interesting.  That was in November of 2007.  It was 96 miles, 11 teams and five finishers.  This was the urban Grand Challenge, and it was a success.  We demonstrated that while primitive, robots understand the vehicle code in California better than most Californians.

To put this in context and to apply another rule that I follow, look back.  A lot of people will tell you “so and so” is looking into the mirror through a rear-view mirror.  I’m hear to tell you rear-view mirrors are good forecasting tools, as long as you use them the right way.  The wrong way would be like the gentleman who back at the start of the Iraq war came to the Secretary of Defense and said, “Sir, I have all the files and lessons learned from Vietnam,” and the Secretary said, “I’m not interested.  We lost that war.”  That’s the wrong way to use a rear-view mirror.  That is confusing being a forecaster and a futurist.

The right way is to look for the general patterns, not the specifics.  Mark Twain allegedly said the future does not repeat itself, but sometimes it rhymes. What you want to do is look for the rhyming, the patterns that are similar.  My rule is always look back at least twice as far as you are looking forward in order to pick up that pattern.  If we do look back twice as far and we look at fundamentals, there is an interesting pattern that reinforces why robots could be taking off.

About every decade we have an enabling technology that arrives that sets the competitive landscape for entrepreneurs here in Silicon Valley. That technology in the late ’70s was cheap microprocessors.  The revolution it triggered was a processing revolution, and the poster child was the personal computer.  You know it was a big deal because the pimply faced geeks on the cover of Business Week and Time were people like Steve Wozniak and Bill Gates, though I don’t think Steve Jobs ever had acne in his life.  You get the idea.

That decade was completely and utterly preoccupied with processing.  The next decade, the ’80s, was shaped by a fundamentally different technology.  That was the communications laser, which arrived and gave us fiber optics bandwidth, CD-ROM, and all that stuff.  It triggered the access-centric decade of the ’90s, and the poster child was of course the world wide web.  Remember back in the ’80s when people said, “What are we going to do with all that bandwidth?”  We could have done the web much earlier, except that we did not have bandwidth.

We had this shift from the processing decade to the access decade.  Of course, PCs did not become irrelevant.  They just changed in function from being defined by what they processed for us to being defined as connectors. What is the big technology of this decade?  It ain’t software, and it ain’t Web 2.0.  It’s cheap sensors.

This revolution on little cats’ feet has been sneaking past us, and we have been so transfixed by other thing that we have overlooked the fact that sensors are coming into our lives everywhere. Whether it’s having more cameras built into our cellphones than are sold separately, or RFID transponders proliferating everywhere, this is a decade being shaped by cheap sensors.

The sequence is this:  in the ’80s we invented our computers, in the ’90s we networked them together, and in this decade we are hanging eyes, ears and sensory organs on them.  We are asking them to observe and manipulate the physical world on our behalf.  All you need is to give them a way to roll around—give them wheels, darnit, they don’t even need feet—and they’re robots.  The poster child of this decade is going to be robots.

The short answer to the big question of where the big fortune is going to be in the short run, forget about saving the world by reinventing humanity.  The next Steve Jobs is laboring away anonymously in some garage, and we are this close to somebody figuring out how to turn these technologies into a compelling robotic product. Maybe it will take two years,  maybe it will take three years, but that is going to be what takes off in consumers’ lives.

We have evidence of this.  Let me give you another rule of thumb: look for indicators—look for things that don’t fit.  Good forecasting is the opposite of good research.  In good research, you hold off on your opinion, look at data and carefully develop your theory, then you pursue doggedly research evidence that supports your theory.  Good forecasters do the opposite.  You come to a conclusion as quickly as you can, then set out systematically to demonstrate that you are wrong, and then look for weird little things that just do not seem to fit that might be important.

For instance, there was this thing back in 2003.  I remember when the Roomba arrived, I had a whole bunch of geek friends here in Silicon Valley, engineers, who were totally stoked about having this robotic vacuum cleaner, and I thought this was really odd.  These are engineers.  I don’t recall them ever having an interest in having a vacuum cleaner at all.  Then when I started asking questions, I noticed they were giving their robotic vacuum cleaners names, and I thought this was very strange.  When was the last time anybody gave their vacuum cleaner a name?

Then I talked to the folks at iRobot and they said “Yeah, you know it’s really weird, we’ve discovered that two-thirds of our Roomba owners give their Roombas names, and one-third confess to having taken their Roombas on vacation with them, or to their friend’s house to show off.”  As a forecaster I went: A. that’s important.  B. it has nothing to do with cleaning floors.  This is scratching some deep, emotional human itch.  After I took one apart I realized these aren’t really even robots.  They’re a pile of transistors posing as robots. If you take one apart, it’s a fraud.  Still, it is an indicator that you can hear that Doppler whistle of the inflection point coming.

Another indicator, the first human was killed by a robot in 2002 in Yemen.  We have something like 20,000 robots flying around Iraq today.  They are primitive and ugly, and there are people who will say, “Those aren’t really robots.”  Still, they are autonomous enough to count.  Here is another indicator that is a little farther afield.

What could an automobile wreck possibly have to do with a robot revolution?  This particular wreck took place 250 miles north of the airbase where the third Grand Challenge was held.  It took place thirty minutes before the urban robotic race began.   A thick fog appeared over highway 99, and people were driving their cars with the instinct of salmon going upstream.  118 cars smashed into each other.  The paramedic working at the front said he could hear them smashing into the back. 118 cars, something like 18 big rigs—this is proof.  People shouldn’t drive, as Brad Templeton writes about.

You should not be afraid of the robot drivers. You should be afraid of the humans. When I look at this picture, my only thought is why do we have to wait that long? What are we going to do with all those parking lots next to buildings when we have robots? After all, a robot could drop you off and it could go four miles away, have a cigarette with its robotic friends, then come back and pick you up.

Having talked a little bit about robots, one other thing I would mention is that maybe like that Roomba that is not really a robot, the robot revolution may arrive without robots. Maybe they are stupid. How many people here have read Daemon by Daniel Suarez? He actually originally wrote it under the pseudonym Leinad Zeraus. That’s “Daniel Suarez” backwards, you get the idea. If you have a copy of his self-published book, don’t sell it. They are already going for $100 a copy on ebay.

He depicts a dystopian future of software bots that are dumber than your average Congressmember and completely autonomous. They create massive havoc and chaos.  It is one of those wonderful science fiction novels with a concealed forecasting message: while we’re all waiting for the Robert E. Lee of big, clanky robots to arrive, maybe they are sneaking in underneath us and we’re in for a surprise.  The robots are already here, and they’re really, really dumb.

I am almost at the end of my time, so I will offer one last suggestion.  Those who think the farthest win.  I should not have to say that to this group.  For you all “a thousand years” spins off your tongue.  How many of you are proud of the fact that you think long-term?  Well, guess what?  You are not the best long-term thinkers.  Now, I sit on the board of the Long Now Foundation, where there will be a ten thousand year clock in the second-tallest mountain in Nevada.  We like to think that we are pretty good at it, too.  However, we are not the best people.

I would suggest to you the best long-term thinkers in terms of conceiving things long-term and also making things happen long-term would be religious fundamentalists.  Actually, I think what is going on today is that there is a sort of race.  You know how there are two types of fools, one who says “this is old and therefore good,” and the other who says “this is new and therefore better.”  They are saying it is a race between those who love technology and those who hate technology.  I think the race today for civilization is a race between people who think the farthest.

You all should keep that in mind, because so far, you are not the ones who think the farthest.  Let me just mention one more benefit to thinking farther.  We assume that when technologies come they revolutionize our lives, they change things forever, and they enable new possibilities.  As a technology forecaster, I am a historian of technology who happens to spend most of his time looking at technologies that do not exist yet.  I am here to tell you that for the most part we use new technologies to ossify old habits.

Think about DOS.  DOS was just a simulacrum of what we were doing with time-sharing, except there was no remote access. We have this example again and again.  My favorite one was the first thermoset resin invented by Leo Baekeland in 1907. It was called “Bakelite,” the first plastic.  What did people do with Bakelite, the first plastic?  Why, of course, they spent their whole time trying to make it look like wood and tortoise shell.  It took people about twenty years before they realized that it made for really cheesy wood and tortoise shell, and then let plastic be plastic.  Then things got interesting.

Well, the same thing is going on today, whether it is DOS imitating time-sharing, or email imitating what was done with the postal service.  Then there are some things that are more profound.  How many people here think our social security system is a great idea and works just fine?  Don’t all raise your hands at once.  Well, it’s your and your antecedents’ faults.  We have the social security system because UNIVAC arrived just in time to save the system from collapsing.  The arrival of computers ossified an arguably obsolete system.

The last piece of advice I would leave with you is: the person who thinks the longest wins, not only in the long-term future, but also in terms of the short-term opportunities that really will change tomorrow.  Thank you for listening.

Though some areas of SENS, such as stem cells and amyloid immunotherapy, are sufficiently mainstream not to need funding, most are still relative backwaters that rely on the Methuselah Foundation to progress. As a result of the great generosity of donors, the non-profit organization trebled the diversity of its research in 2008. At the BIL unconference in February, Chief Science Officer Aubrey de Grey gave an overview of the research projects that the organization is now funding, their significance to SENS, and their potential to lead to accelerated progress towards the defeat of aging in 2009 and beyond.

The following transcript of Aubrey de Grey’s BIL unconference presentation “SENS Progress Worldwide” has not been approved by the speaker. Video is also available.

SENS Progress Worldwide

For anyone who has seen me talk before, you have seen this table.  This is a one-slide summary of how I think we are going to fix aging.  Essentially about ten years ago I realized that the weak links in aging, the types of molecular and cellular damage that accumulate late in life and eventually cause age-related diseases, decrepitude and so on, can all be split up into these seven major categories.  This is really good news because for each of these categories we know more or less what to do about them.

I do not mean to slow down the progression of these things and the accumulation of these types of damage.  I actually mean to get rid of the damage, after it has happened.  I have listed them there, and if anyone would like to know more about that, obviously you can read my publications, especially a book I had out about a year ago called Ending Aging.  It has an entire chapter dedicated to each of these things, which was mainly written by my unbelievable research assistant Michael Rae, a fantastic writer for a general audience.  There is a paperback edition to the book that came out four months ago, and that is my first piece of good news for you.

The paperback edition was a year later than the original.  So much had happened in the meantime that we had to write an entire new chapter to cover all of the wonderful progress that had been made in these various areas.  That is all definitely good news.  The problems have not got harder as time has gone on over the past year or two.

However, the Methuselah Foundation is still pitifully small.  Out of these seven things, the only two that we were actually putting serious money into were these two here: mitochondrial mutations, a big problem in aging that we think we can fix by essentially making the mitochondrial DNA superfluous, and the accumulation of garbage inside cells, which is the main focus of the group that we support in Arizona, and also the group that we support in Houston, Texas.

We have been funding both of these things for at least a couple of years.  You see here that in red are the areas that have been funded by the Methuselah Foundation since 2006.  There are none of those.  That is because we had not been able to expand the diversity of our research for the previous couple of years.  Now I will show you how it all changed.

This man, Peter Thiel, who you probably recognize, first of all gave us a fair chunk of change towards the end of 2006.  In the subsequent three years he has committed to giving us a 50% supplement to any donations that we get from anyone else for research, so a one-for-two challenge pledge.  The first year of our pledge, which was the year 2007, we did not get the maximum amount he committed to giving us.  (This was a million dollars maximum per year, if we got two million.)  We did not get that much—only about a third of that much, and yet he gave us the whole million anyway.  What a complete stunner!

I always tell everyone that everybody can do something to contribute to the mission:  If you’re a journalist, you can get me on the television.  If you’re a politician, you can start changing policy.  If you’re anyone at all, you can engage in advocacy… but if you’re a billionaire, there’s something very specific you can do, and this man’s done it.

Anyway, this is really the main reason why things have changed somewhat in the past year.  The good news is that in this year, since the beginning of 2008, we started funding quite a lot more. We have been able to more than double the diversity of our research support in universities around the world.  This bottom one, which is the approach that I have proposed for really, really curing cancer is something that has lots of components to it, but we have started funding a couple of them.

These are the four things that are really new projects in the foundation.  These are things that I really wanted to spend a few minutes summarizing, especially for those of you who are bored to death about all the other stuff that you have heard me talk about a dozen times before.

A fantastic, originally Serbian immunologist called Janko Nikolich-Zubich, who is a prominent gerontologist and works in Tucson at the University of Arizona, has become very interested in the possibility of being more ambitious about repairing and rejuvenating the immune system than anyone has previously been.  There are two major things that go wrong with the immune system during aging and they fall into two of seven categories that I always talk about.  People have been exploring these things in isolation in a somewhat halfhearted sort of way for quite some time, but no one has had the balls to do them together.

I have managed to persuade Janko to do this.  He is basically applying a combination therapy to mice whose immune systems are going downhill because of aging and seeing whether the immune systems can be really rejuvenated so that the mice are better at resisting infection, getting back to where they were in early adulthood.  It is a reasonably long project, as is more or less any project involving the aging of mice, but it is already underway.  It is being funded by the Methuselah Foundation and we are extremely happy about it.

Jan Vijg is a professor at Albert Einstein in New York and another very prominent gerontologist. He is another strong supporter of the general principle that I have been putting forward for the past two years. His main theme is the accumulation of nuclear mutations. (This is not mutations in our mitochondria, but mutations in our chromosomes.) He has for some time taken the view that the accumulation of non-specific mutations, mutations that randomly disable some aspect of the cell’s function, may actually be a major driving force in the rate of aging. I think he’s wrong. If I thought he was right, then I would be much more pessimistic about our ability to do much about aging anytime soon, because let’s face it, it’s pretty tricky to mend mutations in the nucleus. We are probably going to need molecular nanotechnology before we can do that.

I think we are lucky that actually the only thing that really matters as a consequence of nuclear mutations is cancer.  As many of you who are familiar with my work will remember, I have a specific approach to dealing with cancer.  My take is, if we can really get that working, then all the other things that the accumulation of nuclear mutations might cause will not actually matter for a long time—several times a currently normal lifespan.  We can afford not to worry about that for a little while.

Jan, as I said, disagreed with me on this.  He claims the mutations may matter in a normal lifespan, but he is such a damn good scientist and such a nice guy that he is doing an experiment that he did not want to do, and he’s doing it for me.  Basically, he is having a look at the brain to see whether the brain accumulates what we like to call epimutations.  These are not changes to the DNA sequence, but changes to the decorations of that sequence: things like methylation of histones, methylation of CpG dinucleotides.  These are things that cause changes to which proteins are actually expressed from the genome, as opposed to which proteins could be expressed.

Jan found some years ago that in the brain actual bona fide mutations do not accumulate at all in mice during the whole of adulthood.  They accumulate during growth, but after it the mouse gets to full size and nothing happens.  Of course, if nothing is happening, then it cannot matter in aging.  For this reason, it is very important to determine whether the same is true for epimutations.  I think it probably will be, and he is having a go.  If he finds that there is some change, then of course we have to find out whether there is enough change to actually matter.  That is a whole other set of experiments.

That, again, is underway now. I should say that for both these projects, the people actually doing the work are not the professors, but the people working for the professors.  In both cases we have an absolutely splendid person that each of these people have brought in.  I am really happy that two accomplished and talented post-doctoral fellows are actually doing the experimental legwork here.  I am delighted at how these projects have got going over the past few months since we started funding them.  It really is happening.

Now, demography is not usually listed in my seven strands of what we need to deal with in order to defeat aging.  However, as all of you know who have tried to talk to people who are not terribly persuaded that this would be a good idea, it is really important to think about the social consequences of defeating aging so as to be able to call the bluff of the idiots who actually say that aging is a good thing.  Of course the biggest reservation that people normally have when you talk about defeating aging is “Oh dear, there will be lots of people, won’t there? That will be terrible.”

I have lots of fairly sarcastic answers to this, of course.  However, it would be nice to actually have some data.  What we have decided to do is create a really authoritative study, along with some software that can be used as the substrate for creating other studies, that will allow us to see what the demographic consequences would be of developing therapies that generally knocked aging on its head.  This would be on the basis of various other permutations and assumptions, like how rapidly technology accommodates increasing population, how rapidly the birthrate declines, how rapidly these therapies spread around the world when they actually arrive, all those things.

The Gavrilovs, Natalia and Leonid, are professors in the University of Chicago.  They are, again, very good supporters of this whole mission.  They are some of the most pro-anti-aging, so to speak, demographers in the world, in marked contrast to other demographers.  They are doing this work for us, and furthermore, it is not very expensive.  It does not involve test tubes.  I put this down on the bottom rather cheekily, because we have not strictly speaking signed the contract yet.  We will probably do so next week.

Lenhard Rudolph is a professor in Ulm, a city in Germany.  He is one of the experts in the manipulation of telomerase in mice, the enzyme that maintains the length of chromosomes.  He has done some good work in this area over the years. In particular, he is the world’s leading specialist on the manipulation of mice with respect to the blood.  Now, the blood is a tissue which is going to suffer if we do what I think we need to do in order to really defeat cancer.  It is going to have side effects on the ability of stem cells in the blood to actually continue to work indefinitely.

The way that I propose we are going to get around that is by periodic replenishment of the bone marrow through the stem cells to the blood, but we need to determine that this is actually going to work.  This is the world’s best person to do this experiment, and he is going to do it.  I am pretty happy that all these things are happening, and it is mainly because of Peter Thiel.   I bow down to his generosity.

I bet a lot of you know undergraduates, so you can talk about this to them.  This is an initiative, the Methuselah Foundation Undergraduate Research Initiative, which was put together pretty much single-handedly by another of the mindblowing people that I have had the great good fortune to be able to attract into my team.  This is a guy named Kelsey Moody, who is an undergraduate at Plattsburgh University in Upstate New York.   He has basically discovered that if you are an undergraduate, you can do lab work and it will be really well subsidized, you will get credit, and all that sort of stuff.  He ended up putting together a very well organized structure for us to attract people to do such projects, and also he is running the whole thing in his spare time.

This is a really new venture that is moving forward fast.  Essentially, this is a description of what’s going on.  We are doing things if you are an undergraduate outside the U.S. that is related to this.  Within universities in the U.S., independent research can be done at no cost to the students, so long as they find a lab to do it in and a faculty mentor.  Using this mechanism, this organization that Kelsey Moody has put together utilizes undergraduate research to perform research that benefits the SENS agenda.

Of course, undergraduates are fantastically cheap.  They are almost as cheap as graduate students.  We have a scholarship program, and grants applied to cover the basic costs of the lab work.   Kelsey Moody is orchestrating the whole thing, but visit mfuri.org and you can go from there.  I am really excited about this initiative.  It is going to make a big difference.

There is some bad news, of course.  There are still masses that we are not able to fund yet because we are still far too small.  I should point out that all the red ones above are not actually funded.  They are really only partly funded.  In each of these categories there are many types of things that go wrong.  In each case we are just analyzing one example of that category.  In other words, there is lots and lots that we want to do in each of those categories that is not within our current budget.  However, it’s better than nothing.

There is another piece of very bad news. Nobody has got any money to give us anymore.  That is not completely true, but it is not looking very good.  I have spent a lot of my time over the past few months talking to people who have been wringing their hands a great deal in response to the situation at the moment.  This obviously has hit a lot of charities very hard.  I think it is going to be a challenging year for pretty much every non-profit in the world.

This is a shame, but we are working on it.  We have a few leads that may lead to a substantial increase in funding during this year.  Of course, that is what I spend my time rushing around the world trying to make happen, so wish me luck.  I’ll stop there.

Convergence08, the interdisciplinary unconference, continued with a panel discussion on life extension. Moderated by Christine Peterson of Foresight Nanotech Institute, the group of biotechnology and health experts included Aubrey de Grey of the Methuselah Foundation, Terry Grossman of the Frontier Medical Institute, Bruce Ames of the Children’s Hospital Oakland Research Institute, and Gregory Benford of Genescient Corporation.

The following transcript of the Convergence08 Longevity panel has not been approved by the speakers.  Video is also available.

Longevity Convergence

Christine Peterson: We really have four ambitious individuals up here in terms of their longevity goals.  Why would you get someone up here that does not have ambitious longevity goals?  Our first panelist will be Professor Bruce Ames of molecular biology at UC Berkeley, very well known to many of you. He is the inventor of the Ames test. If any of you learned about it in school, this is the Ames of the Ames test.  His research is focused on cancer and aging.

Bruce Ames: I was a professor at Berkeley for many years, and now I am at Children’s Hospital Oakland Research Institute.  My interests have gotten more medical.  I am interested in preventative medicine and worked for many years on DNA damage, how to detect it, and what is important in humans for its prevention.  Then we got interested in aging because all of our medical expenses are from these degenerative diseases of aging that come at the end of your life, such as cancer, heart disease, cognitive dysfunction and stroke.  If we really want to have an impact on that, we have to work on the mechanism of aging.

We did a lot of work on mitochondrial decay.  I am too old to wait around three years while a rat gets old.  Mitochondrial decay is something you can get your hands on, and we published 35 papers or so on mitochondrial decay in aging.  A terrific post-doc and I came up with normal biochemicals used in the mitochondria that delay mitochondrial decay and made the old rats more like young rats.  I founded a company called Juvenon that sells these pills and put all my stock in a foundation so that I have no financial interest.

In the last few years, another thing going on in my lab is work in micronutrients.  I am interested in what is optimal nutrition, and we stumbled on something that has great relevance to aging.  The principle is that there are 40 micronutrients—about 15 minerals, 15 vitamins, about eight essential amino acids and two essential fatty acids.  If you do not get any of these 40 substances, you die.  What nature has built in, we suddenly realized, is a system for dealing with shortage.  Nobody worries that 60% of the country is really low on magnesium.  Magnesium is in the center of the chlorophyll molecule, and every time you eat a big plate of spinach you are getting your magnesium.   60% of the country is really low in it.

It turns out that what happens as you get low in magnesium is that you get rid of a lot of your metabolism, all the long-term stuff.  Basically, it ages you faster.  Through all of evolution, animals have been running out of nutrients, so it is a trade-off between survival and long-term stuff.  Whenever you are short of any micronutrient, it ages you faster.  We are talking, I think like 90% of the country.  There are not too many people eating a balanced diet, and the big disaster was sugary soft drinks, potato chips and all these high calorie foods that have no nutritional value.  We are not getting enough Omega-3′s or enough magnesium.  Not many of us are eating a perfect diet.

Peterson:  Our next panelist, Dr. Terry Grossman, is a medical doctor.  He is the founder and medical director of Frontier Medical Institute.  Most of you know his name because he is the co-author with Ray Kurzweil of the book Fantastic Voyage: The Science Behind Radical Life Extension.

Terry Grossman:  Thank you very much.  I am a practitioner of anti-aging medicine, so the disease that I treat is the disease of aging.  Most people are accustomed to thinking of aging as a natural process, but I like to differentiate between aging the disease and growing old as a natural process.  All of us grow old, and we are a few minutes older than when Bruce started speaking, but we have not necessarily aged during that time.  It really has to do with the lifestyle choices that we make on an ongoing basis that determines whether or not we age as we grow older.

For instance, if we helped ourselves to some of the foods that were on the snack tables earlier this afternoon, we possibly underwent some aging.  For instance, I spoke with someone today who had a sugary breakfast, some bacon and eggs, whereas I had some fruit and green tea for breakfast.  I think that by making proper lifestyle choices it is possible to grow older without aging.  It is possible to do that at many ages.  We can look at children between the ages of 5 and 15, and they have aged ten years between that span, yet 15 year-olds are in many ways healthier than 5 year-olds.  They are ten years older, but they have not experienced much in the way of aging.  15 year-olds have much higher levels of anti-aging hormones like testosterone, estrogen, DHA and growth hormone.

It is much more difficult to do the same thing between the ages of 65 and 75, or the ages of 75 and 85.  In fact, the aging process is largely due to lifestyle choices that we make.  I feel that by making proper lifestyle choices it is possible to grow older without aging at any age.

Aging has one purpose, and that is to destroy our health and to result in our death. The thing about aging is that if we look at it as a disease, it is unique in that it afflicts 100% percent of us and has so far proven to be 100% fatal.  It is well worth our attention.

The fact is that aging has been introduced at an earlier age as an elective.  If we look at our sperm cells and our egg cells, they are actually immortal.  Sperm cells and egg cells do not age.  If we keep these cells in a petri dish, provide them with nutrition and remove their waste products, they will live forever.  As soon as they join with egg cells and form a new life, then the aging clock begins to tick and mortality begins.  It really is an elective process that has been imposed upon us.

My philosophy in respect to longevity and aging is expressed I think in the titles of the three books I have written. The first was called The Baby Boomer’s Guide to Living Forever, the second one with Ray Kurzweil was called Fantastic Voyage: Live Long Enough to Live Forever, and the third one, which is going to be coming out in April, is called Transcend: Nine Steps to Living Well Forever. There’s a common theme here.

None of these books is called “How to Live Forever,” because the bottom line is that we do not know how to do that. However, what Ray calls “the law of accelerating returns” means that technological change is happening very rapidly, and biological change happens linearly. By linking technology and biology we can essentially reprogram our biochemistry such that we are able to take advantage of that rapid technological change. Our genes today are essentially the same as they were 40,000 years ago. They were well suited for a world of scarcity, where starvation was constantly on our doorsteps. Whereas we live today in a world of abundance, although looking at what has happened with the stock market in the last couple months we may be half as abundant as we were.

Nonetheless, things are better than they were in the past.  I think now we are at the inflection point.  Exponential growth looks flat for a long time until you reach the knee of the curve and then you think it is exponential, but it has been exponential all along.  I think right now, with respect to aging and longevity, we are right at the knee of the curve and dramatic increases in the acceleration of lifespan will be available to us in the years ahead.

Peterson:  Our next panelist you will remember from this morning, Dr. Gregory Benford of UC Irvine.  A physicist originally, many of you have read his fiction.  He has moved into a field related to treating longevity and is the founder of Genescient.

Gregory Benford:  Yes, I am not actually a biologist, I just play one on panels.  I spoke this morning about the general strategy of my company Genescient.  I thought today I would talk more generally about what you do when you have acquired a lot of genetic information, as Genescient has.

We read the genomics of these long-lived fruitflies that lived about four and a half times longer than the control flies.  The biologists thought we would get about fifty useful genes, SNPs.  We planned on carefully parsing through all of them.  When we ended up with around 750, the plan went out the window.  We realized that the whole process is far more complex, with multiple pathways, multiple disease targets and overlaps that we had not suspected.

I mentioned this morning that we were very surprised to find that Alzheimer’s genes in many ways overlapped with diabetes Type 2 genes.  We went to talk with the clinical people and they knew that diabetes Type 2 was predictive for Alzheimer’s.  That was news to us.  We discovered it in reverse by going through the genomics.  To me, playing a biologist on a panel, this was a real surprise, to discover something from the raw data instead of from the clinic.  The real message was with regards to the real use of this information.  We started out thinking that we would begin with a genetic diagnostic.  It turns out that it is not as broadly applicable or as profitable as people on the street would like to have.

What people want is something they can take to make their lives better, to make them live longer, to augment their genes in the defense against these results of aging.  That is why we chose this path.  At the same time it also means that you have choices of how to filter the information.  One strategy is to screen people’s aspirations, things that they want to test.  You can provide a service for telling people what pathways to go down to and therefore intersect their interests.  After all, we have this curious habit where diseases are kind of broken up into interest groups.  There are whole foundations for diabetes, Alzheimer’s and so on.

Many of you know that Terry Pratchett was diagnosed with Alzheimer’s last year at the age of 57, and he has been going through this process of documenting publicly what it is like to feel yourself accelerating along this curve.  He turned around and gave a million dollars to the Alzheimer’s Foundation in the UK.  Later he asked me, what impact did I think it would have on his life.  I said, to tell you truthfully, I’m not sure it will make a difference because their plans are long-term.  I think what everyone really would like to see is a faster path than “Let’s do some more R&D.”

I am glad to see us actually have an open discussion on what to do.  Moore’s Law and the advance of genomic readers has given us more and more information, and I am beginning to regard this field as an information management problem.  What do we use it for and what is the best path?  Honestly, one of the reasons I am at this meeting is to find out how to use it.

Peterson:  Great, thank you, Gregory.  Our fourth panelist, Dr. Aubrey de Grey, is best known for his work on SENS, which is Strategies for Engineered Negligible Senescence, and his involvement with the Methuselah Foundation.

Aubrey de Grey:  Thanks very much.  As Christine said at the beginning, I guess I cannot claim to be the most ambitious of the people up here, but in a way that is because I am the least optimistic about the less ambitious approaches that are being pursued by my colleagues here on this panel and by many other people.  In Terry and Ray’s book Fantastic Voyage, there is what they call three bridges to the comprehensive defeat of aging.  In a way, there is a bridge zero that they did not mention, which is the approach that is taken by the enormous bulk of the medical community who are interested in addressing the problem of aging.

Of course, one of the biggest reasons why Terry Pratchett’s million pounds is probably not going to make much difference to his life is not simply that the research that we have funded is not long-term, but that there is actually already an enormous number of millions of dollars already being applied to Alzheimer’s research specifically over the developed world, added to all the other diseases of aging.  The difficulty with that approach to combating these things is that by the time these diseases have gotten far enough along to actually be called diseases, things are getting pretty far out of hand and chaotic, especially since many of these diseases may be happening simultaneously, exacerbating each other and generally making the whole thing pretty intractable.

The real reason why what I call “the geriatrics approach” to combating aging is intractable is simply because it is not applying the really rather reliable principle of “prevention is better than cure.”  Essentially the targets for these interventions are consequences of stuff that is going on throughout life and progressively accumulates to result in various types of molecular and cellular damage that are side effects of our normal metabolism.  As people get older, those side effects continue to accumulate.  If you are attacking the side effects of those consequences, namely those particular diseases, then your job is just going to be getting harder and harder as time goes by. This geriatrics approach is therefore going to be a short-term approach, and a losing battle.

Of course, Ray and Terry wrote very openly about how we can apply the principle of prevention being better than cure by identifying changes to the diet, taking supplements and so on, that may, if you like, clean up our metabolism—stop it from laying down these various molecular and cellular side effects quite so fast as it normally would.   That is very much the sort of approach that people in academia who study the biology of aging also tended to gravitate to, and the work that Bruce mentioned is a fine example of it.

I feel those approaches are very valuable in principle, but my sense is that they are not going to give us very much.  Essentially, the body is very well tuned, and with the exception of the work that has been done more recently that Bruce mentioned, the compensation for magnesium deficiency, it is going to be pretty difficult to get all that much extra healthy life for people who are already doing pretty well.  I think it is going to be hard to improve those people’s lives by the gerontology approach of trying to clean up people’s metabolism.  For this rather pessimistic reason, I gravitate myself to the view that we need a third approach that is more preventative than the geriatric approach, but somehow avoids the enormous amount of ignorance that we still possess with regards to how cells work, let alone how whole organisms work, and how these various forms of damage accumulate.

That was what led me to believe that actually we might be better off trying to apply regenerative medicine to the problem of aging.  In other words, let these various forms of molecular and cellular damage be laid down at the rate that they are naturally laid down—so that you do not have to understand how they are laid down—but then clean them up before they start to lead to diseases of aging.  This can be called a regenerative approach because it is essentially restoring the structure of the body at the molecular and cellular level to something like what it was at a younger age when less damage had accumulated.

Various things that I think we are going to need to do to make this sort of thing work involve what would be called classical aspects of regenerative medicine—things like stem cell therapies, tissue engineering maybe, certainly a variety of different applications of gene therapy.  There will be other things that you may not ordinarily think of as regenerative medicine, but would certainly come under that strict definition that I gave a moment ago.

I guess the main reason why I have become quite optimistic, indeed why Terry and Ray recapitulated a lot of my work in Bridge Two of the book that they wrote, is because I think that regenerative medicine has now come far enough, and also our understanding of these various types of damage has come far enough that they have become practical—not yet, but in the foreseeable future—to apply pretty comprehensively the regenerative approach to the concept of aging.

That is the work that I focus on, and that is the work that the Methuselah Foundation focuses on.  I think we have a pretty good chance of developing proof of concept experiments in the laboratory in mice within the next ten years that will demonstrate that approach is feasible by taking middle-aged mice and rejuvenating them thoroughly enough to give them a couple extra years of healthy life.  I do not know how long it is going to take to translate that technology into humans because we are talking about the relatively distant future, where technology is hard to predict, but I think we have a 50% chance of getting there within 25 or 30 years.

Because these therapies are bona fide rejuvenation therapies, repair therapies that actually restore people to a younger biological age, that means that all of us in this room have a fair chance of benefiting from that approach.  So, I’ll stop there.

Peterson:  Great.  Thanks, Aubrey.

Ames:  Maybe I’m prejudiced, but I think there is a tremendous amount we can do by improving our diets.  Micronutrients are dirt cheap.  They really do not cost anything.  Minerals and vitamins are cheap, and even Omega fatty acids in the form of fish oil are cheap.  If we had the right information, we could tune up everyone in the world inexpensively.

Already I think we know enough to help the poor, who are eating godawful diets.  They are going to get cancer and die early of diabetes.  Obesity is a disaster for the health.  There are forty or so different diseases tied to it.  At some point thin people are going to rebel and not want to pay for all these medical costs.  Obese people are amazingly deficient in everything.  Their diet is sugary soft drinks, donuts and things like that.  They are hungry all the time, but we are working on that.

Peterson:  Okay, so, take micronutrients? Eat as healthily as you can, and if you need to lose weight, lose weight?

Ames:  Exercise, don’t smoke…

Peterson:  Wear your seat belt.  Terry, what would your average middle-class person do?

Grossman:  Well, begin with the things that are free.  Like Bruce said, it does not cost more to eat healthy.  Eating more vegetables or to exercise does not cost more.  You can get a gym for ten dollars by buying rubber band.  That is what I bring with me when I travel.  They weigh a few ounces, and instead of spending $90 a month to join a gymnasium, you can walk, jog or ride a bicycle.  Something you can do that will actually save you money, so it is a positive for your budget and will extend your life, is to reduce your calories.  Caloric restriction has been found to extend life in animal species, and there is a lot of suggestion that it makes people healthier.  You will save money by eating less.

Cutting calories even ten to twenty percent makes a big difference.  I think most of us over the age of thirty should at least take a multiple vitamin/mineral, should take some supplemental fish oil, and should have our vitamin D levels checked.  Many of us are low on vitamin D above and beyond our multiple vitamin/mineral.  As far as testing goes, the number one cause of death in the U.S. at least is heart disease.  There is a test now available called coronary artery calcium score, a CAT scan, which costs between $250 and $300.  You do not need a doctor’s prescription to get it.  It will screen you and tell you if you have any amount of build up of this cholesterol plaque in your coronary arteries.

You need to know that.  If you have not had a coronary artery calcium score done yet and you are a woman over the age of fifty or a man over the age of forty, give serious consideration to getting one of these as a baseline for a few hundred dollars as part of your budget.  Also, follow the appropriate cancer screening.  Colonoscopies after the age of fifty, either thermography or mamography for women, and PSAs for men.  That is really the low-hanging fruit that does not cost a lot of money and I think will translate into some significant benefits for the majority of people.

Peterson:  You did not mention this but I have a couple of friends who actually go to you as their physician in Denver.  If you do not have a longevity doctor now, you can go.  I don’t know if you have any space on your schedule, but Terry is in Denver.

Grossman:  There are several of my patients in the audience.

Benford:  I will just make one observation.  The faculty at UCI, particularly the physics department, there is a window that people go through between forty and fifty when they essentially give themselves permission to grow older, get fatter and they really go away.  It is a psychological state.  The game’s not over, but you can just sit and watch it. I have found in some of my friends, one of whom died very recently, that it is a phase transition they do not survive for very long.  They really go downhill quickly.  It is a psychological barrier, not a physical one.

Peterson:  Great point.  My guess is Aubrey is not going to tell us not to do these things.  Of course he is going to tell us to do these things.  He has a somewhat distant perspective on how do we take money and convert it into longevity.

de Grey:   I would absolutely like to applaud all the things that were just mentioned.  I think that some of these things cost a little bit of money, but if you are a bit creative you can make sure that you do not, as Greg just said, end up giving up all your intellectual activity, which is an extremely bad thing to do.

When you ask what can a middle-class person do to change the game, so to speak.  I tend not to think of these things personally, but think in terms of how many lives I can save by hastening the defeat of aging.  However, to answer the question, I would say that there is an awful lot that can be done just to raise awareness of the potential for major breakthroughs in combating aging in the foreseeable future, in terms of making serious benefit to the healthy longevity of people who are alive today.  I think that even though most people are not scientists, are not biologists, or not in the position to directly get involved in the research, there are plenty of ways in which people who are in the middle income bracket can get involved in advocacy, especially in raising the legitimacy of discussion about the possibility of doing something serious about aging.

The sorts of things that I tend to emphasize are to make sure that people don’t change the subject.  Make sure that people don’t carry on thinking that this is all about longevity for the sake of longevity, when in fact it is all about keeping people healthy and not getting sick, which is something rather hard to argue against.  Advocacy does to some extent cost money if you want to make a big difference to a lot of people.  I think this is something that young people in the audience would have the highest probability of benefiting the most from.

Coming back to the things that were mentioned earlier, there are all manner of creative ways of making a difference to people’s lives, such as spending a few hundred dollars a year buying multivitamins for a hundred or a thousand people in your local poor community, for example.

Ames:  Costco sells 400 multivitamin/mineral pills for eleven bucks.  It’s really cheap.  However, a multivitamin is not quite good enough.  It does not have enough magnesium, calcium, or potassium, which would make the pill much too big.  It does not have Omega 3s, which is in fish oil.  It does not have fiber, which is important for gut health.  We made a bar that I think will do all those things, has very few calories and tastes good.  It took us a couple years.

A lot of it is motivation.  It is easy to not exercise.  My wife is Italian and she feeds me a wonderful Mediterranean diet, but she kept nagging me that I should get more exercise.  One day I told her that I get plenty of exercise: I run my experiments, I skip controls, and I jump to conclusions.  I like that joke so much I must have told it fifty times.  She said, “I’ve heard enough of that damn joke—I’m taking you to a personal trainer.”  Now I go a couple of times a week and lift weights, row and do all the things that I know are good for me.  Single men have an eight year shorter life expectancy than married men.  Both men and women know why.  Single men self-destruct.

Convergence: Longevity Panel

 

Peter Voss started his career as an entrepreneur, inventor, engineer and scientist at age 16. After a few years of experience in electronics engineering, at age 25 he started a company to provide advanced custom software development and information-technology services. Seven years later the company employed several hundred people and was listed on the Johannesburg Stock Exchange. Having recently taken his artificial intelligence company Adaptive A.I. Inc. (a2i2) out of stealth mode, he presented at the BIL unconference in Long Beach, California in February on the prospects of creating artificial general intelligence, or “Real AI,” in less than a decade.

The following transcript of Peter Voss’s BIL unconference presentation “Toward Real AI” has not been approved by the speaker. Video and audio are also available.

Toward Real AI

Thanks everyone for coming to this talk.  After seven years I am very happy to be able to talk about some of my work.  I am not going to be able to talk in much detail, but we have been in stealth mode for seven years and we have just come out of it.

I have been instructed to make this a firehose presentation, that you guys are all very smart, so I am going to give you a lot of bullet points.  I am going to be covering a lot of ground on my personal experience working on AI for so many years now.  As some of you might know, we launched a commercial venture recently to automate call centers.  These things are also called IVRs: interactive voice response systems.    We have basically been using our AI brain to do that.

What is artificial general intelligence, the field that we are working in?  It can also be called “Real AI.”  It is basically getting back to the roots of AI.  What AI used to be fifty years ago when people started it was to build computers that could think and learn like humans.  That dream had been abandoned with the AI winter, but artificial general intelligence is getting back to that vision to create machines that can think and learn.

If we look at intelligence as knowledge and skills, then AGI is about building machines where acquiring the knowledge is more important than having it.  The system’s ability to learn is the important part, not some programmer sitting down and coding information.  General knowledge relates to general AI, as opposed to domain-specific knowledge in narrow AI.  Things are learned in the abstract, where possible, and the specific context will guide the application of those skills. That is the idea.

Another detail is that it is self-directed, ongoing learning as opposed to a programmer or user deciding, “Hey, we need some new feature,” and then externally deciding to build that feature in.  AGI is about this ongoing, interactive learning process.  It is a system that has to interact with the world.  It has to have input and output, dynamically learning from interacting with the world.

Talking about the longer-term, more ambitious goals of AGI, people often think in terms of human-level intelligence and beyond.  What in my view would qualify for AGI?  How would I recognize it if I saw it?  We might build something only for people to say it is just technology, not really AGI.

My view on that is AGI will not solve all possible problems.  This is a theoretical view of AGI that some people have—something with infinite computing power that can solve any kind of problem you can throw at it…  I think that is nonsense.  To me, if AGI can solve problems that humans can solve, that is perfectly adequate.  To make it simpler, let’s exclude the problems in arts and music and let’s just talk about technical problems.  That to me would be the domain of AGI.

If you have a machine where you can tell it what the problem is, or it can read up about the problem, think about it, or do research, that to me would be AGI.  It does not need to have all human senses.  It does not need to have the sense acuity and the dexterity of a human.  Those are actually very difficult to engineer and computationally expensive.  I think they are actually not necessary.  You guys are all smart, so just think “Helen Hawking.”

Switching over to intelligence augmentation, there is a lot of confusion as to what intelligence augmentation entails.  I put it into three different categories.  One is the use of technology.  Wearing glasses is intelligence augmentation by that definition.  Using a calculator or the internet, to me that is mundane and does not qualify for intelligence augmentation in the realm of AGI and future enhancement.

The second category is the real thing, upgrading our wetware: whether through drugs, implants or some kind of engineering, really upgrading our brain capacity.  That to me is real intelligence augmentation, but it is really, really difficult.  This is why I believe AGI is going to happen way sooner than this kind of intelligence augmentation.  The third category then is uploading and upgrading—placing our brains into another substrate.  That is an even more advanced step and would give us more cognitive power, but I see that as being way, way more difficult.  I believe in fact it is going to require advanced AGI to achieve that.  There are people who disagree with the view, but this is to give you my view on that.

When do I think full-blown AGI will happen?  Some of you may have heard my talk some 18 months ago.  At the time, I was pretty sure the timeframe was less than ten years and as little as five.  Today, I think it is going to be around eight years.  I know that is way optimistic and a shorter period than pretty much anyone else gives. Perhaps other people are just pessimistic on this.

I will give you three different angles.  The one is, practically no one is working on AGI.  Is that cause or is that effect?  Do we think it is not going to happen in the next fifty years because nobody is working on it, or do people really have a good reason not to be working on it?  I will argue that people are actually blindsided in many ways.  The second thing that I like to remind people is that things really have changed since the last attempt at Real AI, which was probably thirty years ago.  Lastly, I would like to talk about the personal experience from the work that we have been doing over the last twelve years and give you some perspective on why I am more optimistic about the time table.

Getting back to the first point, 80% of the people working in AI do not believe in general intelligence.  They do not believe there is such a thing as “g,” in psychology… I think they’re wrong. Out of those remaining, many do not believe that human-level AI is going to be possible in the next hundred years, if ever.  Therefore, they do not attempt it.

Out of the remaining people, 80% are actually working in narrow AI.  Maybe they would like to work in general AI, but they actually work in narrow AI. This is because commercially it is much more viable, you get short-term results, while AGI is very long-term. There is a very strong bias to be working on narrow AI. In academia there is very little support, little funding and it gets no respect. We are only now having the second AGI conference, and it is a tiny subfield. It is really hard to get any support to work on AGI, so people work on narrow AI.

Out of the remaining people, 80% of them do not have a good theory.  They might want to work on AGI, but they really do not have an idea of how to solve the problem.  That does not leave a lot of people working on AGI. That is both a cause and an effect of estimating how soon Real AI can come about.  It was something that really troubled me when I entered the field some twelve years ago.  I was puzzled: “Why is nobody working on this?” Then I actually did spend a significant amount of time analyzing that.

As regards things being better, I have been around long enough in the tech industry to give a personal perspective on this.  On the hardware side, 25 years ago when there was the last real attempt at AGI, machines were a million times slower than they are now.  A one-second response time to test my computer today took twelve days to get that same answer.  If you can imagine experiments where you are testing thousands of hypotheses, which is what we do now routinely, it makes a real difference.

We also spent an awful amount of time writing programs in 64k—that is “k” as in kilobytes.  In fact, we were writing in 48k often.  It wasted an enormous amount of time.  The software side is also very significant in providing advances that shrink down time.  Over the last seven years the chances of being able to find a program on the internet, some tool that previously would have taken months or years to develop that moves us on to the next step, are really substantially better.

This is a quick timeline before switching over to my personal view of working on this.  I spent about five years learning, educating myself and experimenting.  We then started the company a2i2, and for about three years we did pretty much pure research.   We tried to put a lot of these things into practice to find out what worked and what did not work. We then got to a point where we believed we understood it well enough to focus on commercial development, and spent almost a year putting together a business plan, making presentations, raising money and all of that.

It took a lot of time putting together demos and presentations over the last seven years. It has definitely bitten into the AGI work significantly.  We then spent about two years with about fifteen people doing some solid AGI development with some good funding.  Over the last year we have really focused on our commercial track.  We are still doing about a third of our work in pure AGI, but a lot of our work is now getting our company off the ground.  You know, we have to pay the bills.  Fortunately, our AGI system is now getting smart enough to earn a living.  That’s going to help.

What are the other lessons that I have learned?  One needs a clear theory.  To try and do what our brains can do is very intimidating.  All of these issues of what is consciousness, free will and the complexity of vision, learning and epigenisis, these are really complex.  You really need to simplify the problem as much as you can so that you can focus on it.  Having that clear goal and vision is absolutely key.

My view is that all the pieces of the puzzle are out there.  No fundamental technology needs to be developed to achieve full blown AGI.  Over the years that we have improved the intelligence of our system and developed it, we have hit no brick walls that we could not break through in a period of six months or a year.  I do not anticipate that changing.  The problems are hard, but I believe they can be solved by putting effort, thought and good engineering into it and focusing on the issue.

Another thing I have found is that it is very hard to find AI psychologists.  This is a profession we invented.  It is basically the skill of having a good software designer and cognitive psychologist in one person.  Seeing both perspectives is really hard.  Most psychologists hate computers and most computer geeks really do not care for psychology.  To find people who have the skills to think of building a system that thinks, learns and interacts with the environment and has layered learning is really difficult.  It has been hard but we have put together a good team of people to do that.

This is why I also believe that a large company throwing hundreds of millions of dollars at this problem may not make any progress at all.  That is because their mindset will be a software engineering mindset.  Unless you can get the right leadership and get into the right groove, I think you are actually not going to make much progress.

The next three problems that I want to talk about deal with an open-ended general AI.  As the system improves in capability it is difficult to find the right level of problems that you pose.   If you make things too hard, you cannot succeed.  If you make things too simple, it becomes trivial and you do not make progress.  Getting the data into the system, kids and animals can run around in the wild and get all of this feedback, but with a simple AI it is really hard to get it the right training and testing data.

Funding is obviously very difficult.  AGI is a very long-term thing and gets no respect.  What helps us is that we have found a way of having a parallel path of AGI development and a highly viable business model that we could actually marry synergetically.  There are hazards and benefits to marrying commercial goals with AGI.

What will it take for ongoing progress in AGI?  It will take a workable design theory to cut through all the clutter and make this huge problem more manageable.  It takes a workable AGI platform, which is also not easy.  There are not tools out there that allow you to experiment and develop AGI stuff.  There are very few people working on that.  When I started on the project, we spent several years developing this platform.  This is important to have for ongoing development.   Money, focus and real-world feedback to serve as a reality check to your AGI design are very important.  It is very easy to subconsciously dodge some of the problems that real-world constraints impose on you.

We have an IVR system with an AGI brain.  The brain improves the capability of the IVR and the IVR gives us both practical feedback and hopefully lots of money.  This allows for improving the AGI.  There are human operators with different levels of skills in call centers.  The very lowest level, touchtone IVR—press two for one option, press seven for another—is very simple technology.  Voice activated IVR is slightly higher up in the food chain, and we are aiming at the upper end of that.  As the brain gets better, we move up the food chain with our hopefully virtuous cycle of AGI and business model interaction.

Speaking to the risks of commercializing, if we take narrow AI such as a chess-playing program, over time it becomes slightly more capable, but all it will do is play chess.  It will not direct traffic, do medical diagnosis or anything else.  Up until now, IVR has been a narrow AI application.  AGI, as it gets smarter, encompasses a wider and wider range of applicatons.  That is basically the huge difference.  It is much harder to get to some useful application, but once you get that intelligence, it solves a whole bunch of problems.  It’s general.

If you focus on one particular area, you run a real risk of putting all of your resources into solving one problem.  The risk is that in order to get some sales, you put all your efforts into making this IVR the best possible.  That is a risk we run as well.  We have got ourselves against it by having lots of applications that we are pursuing in parallel.  Our business model and ongoing AGI development aims at lots of applications.  We are just choosing one of them at the moment to commercialize because it is an existing market that is lucrative, but one does run the risk.  One needs to mitigate that to really have the effect of being able to continue to fulfill the AGI’s potential.

What is the inflection point?  How do we know when AGI will take off?  It is really hard to say.  With hindsight analysis has been done to see what was the inflection point for making companies like Amazon successful.  In AGI, what will it be?  I believe it is very likely to be the first real successful commercialization of an AGI engine that will have that virtuous cycle.  It may be ours or someone else’s, but I believe that will quite possibly be the inflection point for AGI.

Switching gears back to AI versus intelligence augmentation, since there is a question of which one is likely to happen sooner.  It is really much harder to upgrade wetware than it is to deal with hardware.  The FDA approval alone is probably worth thirty years of development.  Another thing is that with AGI the design information is available to the designer.  With wetware, we don’t have any blueprints.  AGI development, even though it is complex and emergent in many ways, we still at least have some design information.  Once AGI becomes smart enough, it can use that design information itself.  That is Seed AI potential.

The singularity is another controversial topic.  That is when AIs become smarter than humans.  I am quite happy to work on making the dumb AI that we have right now less dumb, for them to earn a living and for us to move the technology forward so that ultimately these AGIs can help us solve the problems that I think most of us want to see solved, whether they are disease, aging, over-population, resources or pollution.  I believe we need better intelligence than we have in humans to be able to solve these problems, and I think AGI will help to give us that.

Will it be a hard take-off?  I do not believe so—not in the sense that it will wake up in some garage or some basement somewhere and overnight take over the world.  There are many physical limits that have to be dealt with, physical processes that work in real time.  Even though your computer could be solving problems in minutes or seconds, there are things that have to be tested in the real world, and then you are hitting the time limitations and dynamics of real-world problems.  For example, if we wanted a computer to upgrade our wetware, it takes a long time for anything in our brains that we implant to heal or to grow. That timeframe I believe will make it a firm take-off rather than a hard take-off.

Convergence08, the technology unconference, began with a different kind of AI debate: not on whether to create AI, or which technical path will work fastest, but how we can use AI technology to build the world we want to live in. Jonas Lamis of SciVestor moderated the panel of artificial intelligence experts, which included Barney Pell of Powerset, Steve Omohundro of Self-Aware Systems,  Peter Norvig of Google and Ben Goertzel of Novamente.

The following transcript of the Convergence08 Artificial Intelligence panel has not been approved by the speakers.  Video is also available.

AI Convergence

Jonas Lamis:  We have a great group of speakers here.  Barney Pell was the founder of Powerset, which was acquired by Microsoft not too long ago. He is now an evangelist, along with holding other titles at Microsoft. We have Ben Goertzel, who is the director of research for the Singularity Institute for Artificial Intelligence and also is the founder and CEO of his company Novamente. We have Steve Omohundro, who is one of the big brains behind the world of AI. Currently he runs his own company, Self-Aware Systems. Peter Norvig is director of research at Google and is another of the great brains in the computer science world.

I will be letting each of these guys tell you a little about themselves.  The way I am going to do that is by asking them a simple question, and we will start with you, Peter.  Please take no more than two minutes to answer the question: “Why are you here?”

Peter Norvig:  I am here because you all are here.  This is an interesting group to talk with and understand what is going on in the world.   I want to understand better the future and where we are going, and you guys are the ones that are doing it.

Steve Omohundro:   I think this is an amazing moment in history, where we have a convergence of these new technologies that are arising.  I think there are amazing things happening in the short-term.  Intelligent technology has the opportunity to revolutionize many of our political and economic systems, ways that we gather information and act in the world. Then there are longer-term things that are coming up that we are just dimly beginning to see, and we can foresee a lot of challenges along that path. We need to start thinking about a roadmap for deciding what kind of a future we want and figuring out a way to get there in a safe and positive way.  I am hoping that this group will contribute to those big questions.

Ben Goertzel:  “Why am I here” is a big question which you could interpret a lot of ways, but I’ll try to take the simpler interpretation.  I have been thinking about these sorts of issues that we are gathered here to discuss for quite a long time, as I imagine many of you have.  I am 41 years-old, and I guess for at least 35 of those years I’ve been thinking hard about all these things, in terms of AI, immortality, space travel and nanotechnology.  I started seriously trying to design a thinking machine when I was maybe 16 or 17 years-old.  It has been a lot of work and I think I have made a lot of progress toward those goals.

One thing that has been very interesting to see in the last five years or so is that a smaller percentage of people think I’m insane for thinking about these things.  I don’t live in San Francisco, I live in Maryland.  It’s a lot more conservative, but even there now and then you run into people who have heard what the singularity is, and if you tell them you are working on AI and on an AI that thinks like a human, they say, “Wow, good luck,” instead of “Wow, there’s the mental hospital.”

I think we are seeing a change in the attitudes of a lot of people toward this kind of work, which I think is a very important and very exciting thing.  It not only makes those of us who are working on it feel better, but it can actually accelerate progress.  Even though I am crazy enough to think that I know how to make a thinking machine, I don’t think I have every bit of the answer in terms of rolling them out in a way that benefits society.  We can all collaborate to make the singularity.  Who is going to do it is in large part the people in this room.

Barney Pell:  My thoughts would echo most of those that were said before.  I am here because I care passionately about these kinds of topics—not just artificial intelligence but in general, technology as a means for transforming the human condition.

I have big goals in my lifetime.  I would really like to see intelligent machines, I would also really like to live forever and travel in space, to be able to communicate with anyone, anywhere, at any time and in general have all kinds of superpowers enabled through technology.  I have had these kinds of feelings ever since I was a kid.

One thing that I think has just been echoed here is that we really are in a special time.  I feel really lucky that we are now at a point where technology is making some of the skeptics have to reconsider—and making some of the futurists feel empowered to unite together and make things happen.  I would like to do everything I can to connect with like-minded futurists and support this emerging community.  I look forward to all of your inputs, and when the computers look back in 25 years, these kinds of conferences are going to be seen as seminal events.

Lamis:  25 years…

Goertzel:  He’s a pessimist.

Lamis:  Barney mentioned better living through Microsoft technologies.  One of the themes that I hear a lot from the AI community is improving the human condition.  One of the questions that always comes to my mind is, do we have a growing gap between the “Haves” and “Have-Nots”? If we do, should we be looking towards our AI development activities to try and correct it?

Omohundro:  Certainly there is a growing divide between the “Haves” and “Have-Nots.” The “Have-Nots” are getting wealthier, but the “Haves” are getting even wealthier faster.  What we do about that is a sort of political question.  I think intelligent technology can help us aggregate the whole world’s opinion about that sort of question.  As nanotechnology and AI come together, we will be able to have far more productive manufacturing, which has the potential to make everyone much wealthier.  How we distribute that wealth is a perennial question.

How do we balance fairness with individual freedom?  I think that’s one of the difficult challenges that humanity as a whole is going to have to deal with as these new technologies shift what the political and individual landscape looks like.  I think the potential to eliminate poverty is huge, but doing it in a way that really reflects everyone’s human values will be one of our challenges.

Peter Norvig:  —And I think it is as much a question of values as technology.  If you have technologies that allow exponential growth, then you are going to get more divergence, because exponential curves which have a different exponent diverge faster than linear curves, which have a slightly different slope.  That means there is going to be more disparity.  The moral question is whether we are interested in equality of opportunity or in equality of outcomes.  Different societies have different takes on that.

I think the way Western society is organized, most of it is toward equality of opportunity, and I think we are on a pretty good track with that, where we can take the billions of people who are not participating in the current economy and give them the ground level of support that allows them to have that opportunity.  Then it is up to each individual, what their specific exponent is and how far off the curve they are able to go.

Lamis:  Ben, what do you think?  Equality of opportunity or outcome?

Goertzel:  The main point I would want to make is it is not us directly that are making that choice.  I’m very aware of the differences between the conditions of various people in the world.  I just arrived yesterday on a flight from Brazil, which is where my company’s main software development office is and where my wife is from.  Certainly in the outskirts of Brazilian cities you see people who have never had the opportunity to learn about any of the ideas that we are talking about here because the public education system isn’t good.  Of course, they are far better off than people in many parts of Africa.  My heart goes out to these people in these situations.

In terms of my own efforts, it seems like the greatest leverage that I can have, since no one has elected me president, and that’s not too likely to happen for various reasons, is to work in advanced AI technology.  As Steve said, as general intelligence technologies get to a certain point, the age of scarcity is going to be gone.  I don’t think any of us can hardly understand what that is going to mean, to live in a world where material scarcity is not one of the defining characteristics of life.  Not that anyone can have everything, but having a molecular nanotechnology-based synthesizer to make physical objects according to our desires is going to make a big difference.

After that point, the difference between “Haves” and “Have-Nots” will be a matter of psychology and culture more so than opportunity. I think we can get there within decades, and that is one of the most important motivations to me for working on this, to be able to uplift everyone and live more according to their choices than their circumstances.

Pell:  From my perspective on this question, I am honestly a bit torn about the extent to which the AI technologies and advances in this kind of field directly impact global living conditions, poverty and so on.   I was just at the San Jose Tech Museum’s Tech Awards for humanitarian, social entrepreneurship companies, and the things that were coming out there were like better processes to do farming and bringing light to people’s houses. I do not think that artificial intelligence will be competing with a whole slew of much earlier technologies in bringing the world out of poverty, but I do think that improvements in information technology in their time will have a totally transformative effect on people’s lives.

If you look at search, it is not necessarily changing everyone’s living conditions.  Actually, I think search benefits most of the world rather proportionately.  I do not think that it is people endowed with great resources that take advantage of search a lot better than others.  Once people get access to computing, they can access all the educational resources in the world pretty much as well as most people can.  I guess overall it is kind of like steel.  Steel helped everyone in the world.  These fundamental advances will help everyone in their right time.

Lamis:  Clearly we are all here to talk about the convergence of all these interesting technologies that we all believe are going to be accelerating in power, scope and capability over the decades ahead.  In my mind that triggers an entrance in the broader community and involvement from the political sphere in the years ahead.

We just had an amazing election that is likely to bring about some significant change in technology policy for the country.  We have heard that Barack Obama is going to potentially appoint a CTO for the United States to help advise technology policy.  Imagine if you will that each of you has been selected as a candidate for that CTO position and you are sitting down with the president.  I want to know what advice would you have on these kinds of technologies, in particular artificial intelligence.  Over the next couple of decades, how should we be thinking about U.S. technology policy?

I am going to start with Peter because if you have not been to Norvig.com, he has done an amazing level of depth of research and writings about the election. I know you have been thinking a lot about this stuff.

Norvig:  I certainly have.  That was one of my hobbies over the past several month.  As addictions go, it’s a great one because on November 4th, you are cured.  It is great to be here in this room, because we actually had on election night a “Silicon Valley for Obama” party right here in this room.

My first advice for the president, were I in that hypothetical situation, would be: “Don’t choose me, choose somebody else.”  Most of my advice would be things that I think the president is already doing.  The most important one is to believe in reality.  You’d think that would be something you would not have to tell to a president, but apparently it is.  I think Obama does, so that’s off to a good start.  I think the next thing is to invest in R&D.  His job is somewhat to lead the world, but mostly to lead the United States, and it is important to reestablish the United States as a leader there and make us a country that can step forward in terms of development.

In terms of our policies in immigration and being a destination for the world’s students to come to learn and then go off to become entrepreneurs, everyone can benefit from that.  We have slipped over the last eight years or so in terms of funding research and creating that environment, so it’s time to restore that.

Omohundro:   I see two areas.  One is using intelligent technology to make better decisions in our society, and the other is new forms of research and investigation into the future of artificial intelligence.  In the first end, our voting system is really antiquated—both at a mechanical level in how we do ballots, but also in its structure.

There is a very nice website, rangevoting.org,  where a friend of mine has analyzed a number of voting systems, and ours is by far the worst at  reflecting what people’s actual values are. I think there is huge opportunity for getting better ways of aggregating the beliefs and desires of the population. There is something I call “semantic voting,” where instead of voting for this candidate or that candidate, or voting “yes” or “no” on this issue, you should really be able to express much more nuanced opinions about something. There should be a way to aggregate that so that we can guide decision-making.

The bailout was a perfect example, where the population as a whole was hugely against it.  I and many people I know wrote to our congressman about the structure of the bailout, which I think was 99 to 1 against, as it was proposed by the Secretary of the Treasury, and yet they ended up passing it.  I think economists as a whole view that as very bad, and yet our political structure allowed it to go forward.  I think that is an example of something where we can bring greater intelligence.

I think we desperately need greater understanding of the potential pathways as we move into the future and new technologies come to the fore.  Right now it is just a smattering of little organizations, while it would be really nice to have a countrywide analysis and planning for what this future pathway looks like.

Goertzel:  To make detailed recommendations of what the president could do would obviously take longer than a few seconds.  In reality, Obama like any other leader is subject to all sorts of constraints in terms of Congress and so forth.  The basic message that I would get across if I were in that position is that we are completely insane to put a trillion dollars into corrupt banks and incompetent auto makers.

If we want to inject a large amount of government money into the economy, we should put a half trillion dollars into AI research and a half trillion dollars into life extension research, put a half trillion dollars into nanotechnology research, and don’t put it in by having government labs do it but have the government invest in private enterprise, though with a timescale that is not what typical VCs do. Let the government invest large amounts of money in companies that understand the payoff to be ten, twenty or thirty years out. Something like that could make a huge difference in the progress we have toward the singularity. I do not see Obama or anyone else thinking in those terms right now. I think that we will get there anyway, but it is too bad that there is not more of a Singularitarian vision governing the stimuli that we have decided to inject into our economy.

Pell:  It is pretty hard to advise the president.  There is a lot to say.  I agree with what Peter said about reestablishing our leadership in R&D.  I would also add education.  Future leaders are not being given the kind of education that they ought to have, and I think there are tremendous opportunities for improvements in education.  Like Steve was saying, I think we should be transforming the nature of discourse. Our government is run and our people are elected using methodologies that are two hundred years old.  With all the transformations in society, these things should change.

There is no reason why individual voices cannot be heard at all levels of government and why information cannot be shared in tremendously new ways with new levels of transparency. In terms of specific R&D, I would point to aging.  So much money is going into health care and treatment of symptoms of aging, rather than the causes.  If we could make some progress in that, it has the potential to increase our lives and reduce our costs.  The other one would probably be energy.  Nanotechnology has the ability to transform a lot.  We could invest in nanotechnology and its applications for manufacturing and miniturization, and we might be able to both generate more energy and use less of it.

Convergence: Artificial Intelligence Panel