Future Current

Steve Jurvetson is a Managing Director of Draper Fisher Jurvetson, a leading venture capital firm with affiliate offices around the world. He was the founding VC investor in Hotmail (MSFT), Interwoven (IWOV), and Kana (KANA). In September of 2007 he presented a talk at the Singularity Summit in San Francisco hosted by the Singularity Institute for Artificial Intelligence called “Dichotomy of Designed and Evolutionary Paths to AI Futures.” There he asked how the first general artificial intelligence that exceeds human intelligence will be built. Some technologists advocate design, while others prefer evolutionary search algorithms. Still others would selectively conflate the two, hoping to incorporate the best of both paradigms while avoiding their limitations. But while both processes are powerful, they are very different, and they are not easily combined. Rather, they present divergent paths.

 

The following transcript of Steve Jurvetson’s 2007 Singularity Summit presentation “Dichotomy of Designed and Evolutionary Paths to AI Futures” has not been approved by the author. An audio version of the talk is available at the Singularity Institute website.

Dichotomy of Designed and Evolutionary Paths to AI Futures

Before we start, I’m just curious about the audience’s predisposition on this topic. I will be talking about what I think will be two fundamental paths to AI futures, once of which is designed, one of which is built or evolved using simple iterative algorithms. By “evolution,” by the way, I don’t mean biological evolution, per se, but evolutionary algorithms: simple selection and variation, selection and variation, over and over again. And I’m curious, if you think about the first artificial intelligence that is human-capable, let’s say it passes the Turing test, a crude measure… how many people in the room think that that solution will be primarily designed without any benefit of an evolutionary algorithm in the development or training? Maybe a quarter of the room. How many of you think you were designed? Same hands. I was just wondering if there was a philosophical overlap. And how many people, back to the Turing test question, think that whichever approach “wins,” if you will, the race will be fundamentally based on an evolutionary approach? I’m with that second camp. And then there’s a fused methodology, Brad Templeton’s bonobo uploads, which is my favorite of the possible scenarios, which we’ll get to later. Nevertheless, I think this is an important topic not just for AI but for a variety of fields that pertain to how you build complex systems that are robust and resilient. So, even though I will be speaking mainly about AI, the thought started when we were looking at nanotechnology and trying to figure out which technologies would be commercialized first, and we had a similar series of epiphanies on why we think the biological approach will win.

Here is one slide on us. If you’ve never heard of DFJ before, we’re an early stage venture capital firm. So, for context, I’m going to be mainly speaking about stories and anecdotes that I learned from brilliant entrepreneurs, not so much from big companies. We have offices all around the world and manage about $5 billion, but we fundamentally invest in startups that are just getting started and have big dreams and aspirations and want to change the world. I’m going to add some hasty generalizations from what we’ve learned from entrepreneurs. So, when we first started investing in nanotech, we had to, as with any new emerging field, ask the question when will these technologies develop? What’s the pie-in-the-sky science project? What is a near-term commercial opportunity? And to make a long story short, the framework that sort of shook out for us is that there are two fundamental approaches to nanotech visions of the future. When I say “visions of the future,” I mean big, bold visions, where everything costs a dollar per pound, pick your favorite analogy. The top-down approach I would call the semiconductor path. That was what Intel and IBM would talk about, the 20-30 year march where you would build a business from the top-down. You could think of the miniaturization of the semiconductor industry and the MEMS devices that they are producing as steps in that direction. It’s slow, steady, somewhat predictable.

Much more radical, and a bit unfamiliar for the semiconductor industry was the bottom-up approach: using self-assembly, (dare I say) bio-inspired models for how you build semiconductors or how you build micromachines. In many cases you can do things today that are much more powerful than what the chip industry can do today, but we lack systems theory and in a lot of cases we don’t know what we’re doing, but I think it’s an area that has a lot more power and potential. In the nanotech area, we have invested primarily in that latter bottom-up category and have found all kinds of opportunities today in energy, electronics, and medical applications that are in some way or another bio-inspired. We’re borrowing components from nature, we’re re-engineering biology. Synthetic genomics is creating artificial life forms. They’ve already completely swapped the DNA from one organism to another. They’ve changed the phenotype, basically changing one type of organism to another. People are doing fascinating artificial evolution work in the labs. In fields like molecular electronics, solar cells and in others, a lot of progress is being made in self-assembly, like ZettaCore’s use of a molecule like chlorophyll to build a better memory chip, for example.

We are seeing current progress in borrowing bits and pieces of a bio-inspired model, what I call the bootstrap model in nanotech. So it begs the question, at least it did for me in the nanotech investing domain, which would you bet on? Would you wait 20 years and just assume that all the radical stuff will fail, or do you try to learn as much as you can about the bottom-up approach from which we know so little? It’s that latter that I at least find more intellectually interesting and more potentially powerful in the near-term. I would sort of posit the supposition for today, and this seems true for artificial intelligence, that if you think about any large, complex system, it’s more likely that some iterative algorithm, some simple algorithm, will be able to transcend the complexity of its antecedents more than we can think in our own hubris that we can design such a thing. When you talk about accelerating change in general and the march to the Singularity, I just want to highlight that the life sciences have just as much interesting stuff going on in accelerating change as do the IT industry. So, the top three would be things we’re familiar with: Moore’s Law, Kurzweil’s version of Moore’s Law goes back a hundred years, and etc. And we’re all familiar with how accelerating change is occurring in the information technology industries. But the bottom ones are in some cases more accurate, like Dickerson’s three protein structures prediction has held remarkably true over 65 years. And, in some other cases, they are accelerating more rapidly than Moore’s law itself. It makes Moore’s law look very flatline by comparison. I’ll use genes mapped as an example, just to remind people how much progress has been made.

This was the march up to the decoding of the human genome, when Craig Venter challenged the U.S. government in this race and did over half the project in the last year. When the human genome was done, the thought was, well isn’t that sort of the pinnacle of achievement? Why would you do more? Aren’t we going to hunker down and understand that better? You might expect that this would become an S-curve of some sort, but of course it doesn’t. The prior graph ends kind of down there, if you look at the next five years of progress. Clearly in the microbial populations there is an enormous wealth of digital data. And this is valuable data, this is one proxy for how much information we have in the visual databases of the life science industry. When you are re-engineering the information systems of biology, this is what you care about, how many books and pages you can pull from, certainly in the field of synthetic biology.

Well, here’s an interesting statement. In the last year alone, one team, Craig Venter’s, has grown this database tenfold. And how did he do it? He went around the ocean and started sequencing the viruses, bacteria and microbes in the open ocean, and found incredible biodiversity. Between the 200 mile span of open ocean he found the genetic makeup of organisms there to be 85% different in some cases. It would be like tundra, savanna, Arctic wildlife, archaea. Pretty amazing. He’s built the largest digital database of biology in the world. Just one example, not meant to be read, all the stuff in red on this chart were unknown to science before he did his very first sample. He then went around the world, but he went to what he thought was the deadest area on the planet, the ocean, and dramatically improved (we believe 100-fold) the number of known genes associated with energy transduction from the sun.

Now, why did I bring that example up? There’s a lot of compounding progress, almost a Renaissance of learning going on in the life sciences, and in genomics in particular, and I think it will shed some light on how we can build complex systems, and these fields will interrelate. I’ve already alluded to the fact that I think all the big, unsolved problems in computer science, in nanotech, synthetic biology and in artificial intelligence are fundamentally, at the core, how can you build robust, complex systems, from a Santa Fe Institute point-of-view? And these two paths, I’ve already mentioned. Now, a very rich debate will be why do these paths need to be different? Can you perhaps in the future blend design and evolution - do a little of one and a little of the other - take the strengths of one, avoid the weaknesses of the other? But I believe they’re diverging, and once you start going down one of these paths, it makes it harder to go back to design once you start evolving, for example.

But I’ll share what I find is a very different point-of-view, and I see this in the writing of many of the luminaries in the field of artificial intelligence. I’ll just pick one example which I found in Jeff Hawkins’ book On Intelligence, which defines in this entire book a pathway to AI development that I would describe as more the evolutionary path through a neural network model and iterative training of these networks. Yet, near the end of the book he makes this zowee kind of claim that we’re going to be able to do what I call “cut-and-paste. ” That we’ll just basically grab like your ability to speak French, or are you a political science professor, and just swap it out - various subsystems of our intelligence and our capabilities, as if it would be that easy, and that is what a designed object-oriented program should be able to do. But I would argue that evolved artifacts are nothing like that. Evolved artifacts are very different. I will be focusing primarily on information systems and networks, as opposed to evolve, say, other organs, and that could be a side debate: Why is it that intelligence and networks this applies to most strongly?

Here’s what tends to happen. You get these emergent layers of abstraction, right? The brain’s great advance wasn’t re-inventing the neuron or re-inventing codons in our genetic code. It was a bigger, more interesting organization of the cortex. With the exception of mirror neurons, there hasn’t been low level innovations to the brain, it’s been more the higher levels of abstraction. Some would argue now that memetic evolution, cultural evolution, is the ultimate extension of the same idea. But it moves up to a different vector of indirection, something Kurzweil has spoken quite a bit about. #2, I think, and others may obviously disagree, biological evolution gives us the only existence proof of a system that can transcend the complexity of its antecedents using a simple algorithm. So instead of just a leap of faith - “I’ll make something smarter than me. I’m just going to be confident and I’m going to do it” - you can look to this existence proof that says a simple iterative algorithm distributed over time and space, evolution, can do that.

There are problems, though. Evolution is not great for everything. The most noticeable one is that subsystems within any evolved system are completely inscrutable to the designe, the person who built the system or the process. There are many examples. Danny Hillis wrote an evolutionary algorithm that then produced a bubble-sort algorithm. It did a pretty good job. It was better than any algorithm that he knew how to write. It wasn’t world-beating, he only did the experiment a bit at a time, but it was completely inscrutable code. He couldn’t figure out, spending quite a bit of time, how it actually did what it did, how it did it sort numbers. Similarly, for anyone working with neural networks, the inner wiring of the network is completely opaque to the outside observer. The subsystems, if there are any, are hard to discern, much less cut and paste. A similar analogy, the wisdom of crowds, teams of diverse numbers are more powerful than any single expert on the team.

The point of all these examples is that Danny Hillis and all these other people working in genetic algorithms and genetic evolution can hone the process. They can learn how evolution works, but they don’t learn about the artifacts that are created. They don’t learn how an artificial brain would work, just by evolving one. In fact, they are as inscrutable as a human brain itself, I would argue, in the future. What they get when they run these processes are these black boxes defined by the interfaces. What were the training sets, the selection pressures, the Io, if you will. The sensory Io, in the case of AI, is completely inscrutable. Now, I would argue, this is sort of a bold claim without any substance behind it, it’s just sort of a gut feeling, that there’s never going to be a solution to that problem. That it’s fundamentally like Wolfram’s cellular automata, the interesting examples, where you have to run the iterations of evolution over and over again to see the results that will unfold, and there is no mathematical shortcut to the output. For those who have read Wolfram, perhaps you know what I’m talking about with some of those rules, where there’s this beauty that unfolds in the interesting cases, but there’s no way to jump to the answer without running the iterative algorithm. It’s kind of like there is no reverse evolution the way there is reverse engineering. You lose information as you go, and you can’t backtrack.

So, what is this dichotomy? To restate some of the points, when you design something you have control (at least you think you do) but your systems are brittle. Anyone using Microsoft code knows what I mean. When you evolve a solution, it’s a little bit out of control. It’s like parenting. But you can, within their accustomed environment - that’s a really important caveat - develop robust, resilient adaptive systems. Now, there may be co-evolutionary islands, if you are evolving something in a computer system, or in a weird testbed, it’s only robust within that testbed. And the idea of differential immunity or taking something out of its accustomed environment can lead to catastrophe as well. So, this brittleness point is only within the accustomed environment. We would jump to the conclusion, by the way, we would want to evolve our AI’s with us, and not without us, because it would be brittle without us.

Design only tackles what I call simple problems. I mean, Word and Windows are simple problems. And the thing - this analogy keeps poking us in the eye, right? - is that the human genome is smaller than Microsoft Office, by far. And it tells you a lot about how they program and how we unpack the code. And I would argue that’s the only, in biology, that’s the only case where you can exceed the complexity of antecedents. The problem is though with the benefits of design is you can understand what’s going on. Modular object-oriented programming, people are familiar with that. The beauty of design is you don’t keep reinventing the wheel and you understand what you’re doing. So you learn about the artifacts. If you build a bridge, you know how bridges work. If you design software, at least if a single person does, they kind of understand how that piece of code works, and, presumably, could use that learning to do better and better design. Design is, in and of itself, a self-reinforcing paradigm of learning. That’s why we learn about how to build things. This is why we learn engineering. This is why we go to design school: to accumulate the learning of our antecedents, or ancestors, and that makes sense. Evolution is not like that. At least what you learn there is process learning: How can I evolve things better? How can I do directed evolution? How can I do all these processes that let things grow without knowing what I’ve grown?

So, what are some impications for AI? First, I don’t think cut-and-paste portability works in the non-design world. If you’re doing rule-based, case-based design, but not in any what I think to be interesting projects going on with Hawkins and others. In the case of Kurzweil and uploading human consciousness, it can’t just be the brain, in my opinion. Problems like phantom limb pain with amputees is a fascinating example of if you are missing part of your sensory context, unbelievably complex things can happen with the brain and rewiring that can go on within them. And uploading a brain in a box could have serious problems, because the brain is just a subsystem of our intelligence. It is not, by any means, the outer boundary of it. The sensory system, the sensory motor cortex, is part of it.

I mentioned co-evolutionary islands. I think this is why robotics is so important. Some people posit that if you don’t have an AI that is coupled to a sensory context that we understand, that interacts with the world as we know it, that it’s going to be alien intelligence. And that might be okay, but we might not even know how to interact with it. Path dependence is interesting. If you evolve a system, and the survival test every time over millions and billions of generations is who survived, or who grew the most rapidly, or propagated against some criteria, one of which is survival, will it have a low-level reflex (almost like a limbic or reptilian brain reflex for survival) no matter how much you try to program it out at a higher level? It’s almost like, in our brains, a higher level control algorithm over a limbic or reptilian brain at the core, might any evolved algorithm, when it gets to AI levels, have a similar problem?

So, here’s an interesting one. This gets to Brad’s bonobos. You’ve got to think, who gets there first? Would ever bother to reverse engineer an evolved artifact? If you look at how much effort it’s taken to understand how the brain works, and the subsystems of the brain, it takes a lot of human effort over a long period of time. Imagine you evolved something comparably complex that looked nothing like a human brain. Would you take out ten, twenty, thirty years to try to reverse engineer it? Or would you run a few more years of the iterative evolution algorithm and make it better, and better, and better? I would argue you would, unless forced to by regulation, why would you ever bother to stop and reverse engineer anything that evolved? You’d just keep going. And if you do that, it’s sort of the first mover advantage of immense proportions there. And if you are trying to augment intelligence, you’re never going to catch up with those that create the alien intelligence and let it run on the rapid rate at which artificial evolution can run computational systems versus a biological substrate of reproduction that’s just deathly slow. Lastly on the uploads, for those who haven’t heard it, before you upload humans, they’re probably going to do animal tests on primates. Make sure they’re bonobos, not chimpanzees. This is thanks to Brad Templeton and his quirky sense of humor. The bisexual sisterhoods of bonobos are just a much more friendly base upon which you want to let these things propagate, unlike chimpanzees which are incredibly aggressive. And once you upload a primate and start accelerating that pace of change, the human upload - if it’s a few years later - might not ever catch up. Something to think about.

So, where are we today? This bottom-up approach isn’t just theory. There’s a single guy, John Koza, running an enormous server farm of low cost computers. Go to the website genetic-programming.org and you can see that he has 23 patented inventions already, 21 of which reproduced patents other people did, but he did it completely in a black box. Two of which, he’s gotten patents on were de novo. What he does is, it’s a complex flowchart but it’s actually quite simple. It’s create a random set of programs, just random bits, mutate them randomly, have them reproduce randomly, and test which does the better job at whatever it is he wants. Analog circuit design and intent design were the two initial areas. His quote at the bottom of the page, he believes he can now “routinely” deliver “human-competitive machine intelligence” in an existing domain, not a general intelligence.

Now, there are some fellows here in this audience who are doing some very interesting work. Todd Huffman, from ASU and Texas A&M, has found a way to scan a human brain very quickly. It used to take months. Under thirty minutes for these mushroom bodies in the brain of a bee. It’s
associated with sensory associative memory. So, we will be able to find a topology and understand the structure of neural circuits a little better than we ever were before, and that can feed into other work. Paul Rhodes, in particular, who’s here at this conference, is doing this fascinating general synthesis work, which is modeled on the human brain, where synapses are forming and breaking thousands of times a second, ion channels are being modeled to incredible detail. In this case, you are looking at a single cortical column, which is the basic repeating building block in the cortex. This is a completely detailed low-level model of how the brain is working, and is being used to self-organize for sensory applications, and olfaction and such. He can model how these things fire and do fascinating work today in a lab using, interestingly enough, graphic processing processors for the CPUs. He uses the NVIDIA cards for the computation, not a Cray or anything fancy.

So, last concept before I finish. I asked in the beginning, is it A or is it B? And I think the most interesting question is could you blend the two? And Jeff Hawkins in a way was saying, “Hey, we’ll just blend the two. Cut and paste.” So far, I don’t see any evidence that anyone has figured out how to do that, except on some interesting corner cases. But, it is a grand engineering challenge that lies ahead of us. Can we blend these? Can we do evolution and put design over on top, like evolve an intelligence then make sure it’s Friendly through a design approach? Can we even control, or bias, the direction which things evolve? If so, to what extent? Do we have a theory, or are we just doing experiments ourselves? D-Wave I won’t go into, but there’s a very interesting, sort of mind-bending, opportunity there to make a great leap forward as well. I think D-Wave is making great leaps forward in computing, which warps Wolfram’s computational equivalence quite a bit. When I asked Wolfram about it, he couldn’t quite reconcile quantum computers with his framework, and that could mean some pretty big advances for how we do computation.

As evolution continues to go, we are, in fact, evolving the capacity to evolve. I will give one last example very quickly, which is a low-level pedestrian example, but it gives you an example how you could blend the two. This company called Genomatica takes the E. coli metabolic pathways and wants to make some biofuel or some industrial chemical. Normally when you do genetic engineering or biotech you splice genes in and the thing evolves away from making the chemical, because its goal in life, to grow and propagate, is not particularly aligned with the goal you have, which is to make the chemical over and over again. What they do is knock-out a bunch of genes in the metabolic pathway, so the organism has to make more of the chemical you want in order to grow. It’s kind of a simple concept, but if you don’t design the answer but just say, “Make some chemical, a little bit of chemical I like, but make it so if you don’t make this chemical you die, and then have an artificial environment where you just keep culling the fastest growing members of this population, in a week or two you start with a product down here in the red, which doesn’t do much of anything. You engineer it, you get the green. But you let it evolve for a few weeks and you get a twenty-fold improvement in the production of whatever chemical it is you want. So, in a sense, you don’t design the answer. No one knows, in the case of the blue bar, how it does what it does. All they know is that they jump-started a process of natural evolution, in the sense of real, living organisms, in a constrained environment.

I’ll end it with this slide then, which is a great quote from Danny Hillis from his computer science primer The Pattern on the Stone that I believe in. This is how I think we’re going to get to the AI futures. If I was to bet on one of the two, and maybe someone wants to make a Long Now bet to make it more fun. I think something that allows us to move beyond engineering, to create something we don’t understand (but we understand the process of its creation) is our ultimate goal as a species, to propagate something beyond ourselves and to fulfill our own desire for symbolic immortality. Thanks.

This entry was posted on Friday, October 26th, 2007 at 7:39 pm and is filed under Singularity Summit, Artificial Intelligence. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.