Accelerating Future

I.J. Good gave one of the first articulations of the Singularity concept in 1960 in his essay “Speculations Concerning the First Ultraintelligent Machine”. He said:

“Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.”

Then, in 1990, Vinge defined the Singularity as:

“the imminent creation by technology of entities with greater than human intelligence”

Nothing about genetics. Nothing about life extension. Nothing about nanotechnology. Nothing about robotics. Nothing about wearable computing. Only smarter than human intelligence. Nothing else. Technology only enters into the picture because it’s the only thing that could actually lead to the creation of smarter than human intelligence, and the assumed means by which this intelligence would influence the world. The involvement of technology is secondary, not primary.

The Wikipedia article on the Singularity has already been updated to articulate the clear division that I’ve always argued for here on this blog: the difference between an intelligence explosion and accelerating change. Notice how the first two paragraphs on the page focus on the concept as introduced by I.J. Good and Vernor Vinge. Ray Kurzweil is not mentioned until the third paragraph. The Intelligence Explosion concept is where it belongs — in the first section. The Accelerating Change concept comes after the discussion of the Intelligence Explosion. When your advocacy of making the ideas distinct appears to be winning, it’s hard to give up.

Vinge’s definition of the Singularity as greater than human intelligence has been around since at least 1993, or about 16 years. In contrast, Ray Kurzweil’s definition of the Singularity, which takes him about three pages to expound and is more of a general description of emerging technologies and transhumanism, didn’t come about until 2001, making it about eight years old. The original, precise definition is about twice as old as the newer, vaguer definition.

What is astonishing is how the latter definition seems to be so ubiquitous that intelligent people can actually read the former definition and it passes right through them without registering, like a neutrino. It’s like they never even read it. For instance, on VentureBeat, Anthony Ha writes,

“The Singularity, for those of you with only a fuzzy idea what the trendy term actually means, was coined by Vernor Vinge, a computer scientist and (excellent) science fiction writer, back in 1993 to describe the point at which rapidly accelerating technology makes the future literally impossible to predict.”

This is false. Vinge described the Singularity as the point at which superhuman intelligence makes the future impossible to predict. Accelerating technology and superhuman intelligence are not the same thing. The former is not necessary for the latter, nor the latter for the former. Superhuman intelligence might decide to invent a bunch of technologies and then just stop, for the radical reason that maybe some people would be better off without unlimited technology. Superhuman intelligence would hopefully be reasonable and moral as well as superintelligent (if we do our jobs right), so why would it necessary keep inventing technologies endlessly and mindlessly, like some sort of unstoppable deranged machine?

The Singularity is not necessarily when we transcend our biological limitations. It could be when a single entity or small group of entities transcends their limitations (biological or otherwise), and then decides that the world would be more interesting if we were all dead and replaced by beings of their design, or nothing at all. The Singularity could be the moment that a group of entities becomes superintelligent and decides that the world is best if the status quo is preserved literally forever. The group that becomes superintelligent potentially rules over all of us. Superintelligence does not necessarily entail supermorality. It could be more of the same. No happy techno-fantasy la-la land. The future may end up looking more like the past than our facile imaginings of The Future(tm).

I think people are inclined to talk more about accelerating technology than superhuman intelligence because the former is easier to imagine while the latter is not. Still, why not try to imagine the latter? Consider working on an incredibly difficult engineering or math problem for years on end, when a universal genius comes along, offers to help you out, and suddenly all the pieces fall into place. You suddenly view all your past efforts, sleepless nights, and brainstorming sessions as transient pieces in a larger vision that was beheld and articulated on sight by the genius. You wonder how many others around the planet are clawing at the darkness of solution space, just waiting to have their search illuminated by the light of intelligence.

The shock of this experience might seem amazing, but it can be instilled by differences in intelligence only within the human normal distribution. Now imagine the same experience, but the universal genius being above you in an inter-species fashion, the way a human is above a chimp, rather than a genius is above an average human. Typically, the thought process of this genius is entirely inscrutable. It only becomes understandable when the genius makes a point of explaining the pieces to you in terms you understand. If the genius is sufficiently smarter than you, it might take them decades to connect all the dots in a way you comprehend, so you only get the salient parts.

There are many reasons why discussing these concepts is somewhat unpopular. First, we like to pretend that intelligence doesn’t matter, or that it can’t be measured. These are fictions. Many academics have political motives to denigrate intelligence research — they consider it necessary for the morale of humanity that everyone be seen as having equal potential, lest the very philosophical underpinnings of democracy be put at risk. Intelligence research is associated with racism (The Bell Curve) and eugenics. If someone is talking about quantifying intelligence, they must only be talking about it because they think they’re smarter than you and want to step on you.

This might be true in some cases, but here, in 2009, understanding intelligence is important, whether or not people put that understanding to unscrupulous ends. The reality of intelligence differentials must not be denied. If there is any magic in the universe, it’s intelligence, and if denying that people are born with more or less of it gets in the way of our responsibly handling the creation of a new intelligent species, then these ideas must be discarded. Our species is on the cusp of an event that’s never been seen before: intelligence building intelligence. If we focus on the shiny glittering toys of technology, we get entirely distracted by the true task ahead of us.

21 Responses »

2 Responses »

There is substantial disagreement between myself and 99.5% of other futurists that talk about human-level AI. The 99.5% talk about human-level AI as if it would be useful like computers are useful, allowing us to have automated secretaries and insurance agents and travel planners and logistics experts. They see AIs plugging into our economy just like humans do today. The AIs are usually used for rote tasks, somewhat analogous to the role of unintelligent software now.

I see scenarios like this as extremely confusing. Even with conservative assumptions, the economic impact of human-level AIs seems to be likely to be much larger than Rosie the Robot type scenarios. Last summer, myself, Steve Rayhawk, Anna Salamon, Rolf Nelson, and Tom McCabe, with help from a few others, sketched out some simple frameworks for modeling possible AI self-improvement speeds, among other things. We used simple variables like “X is how many dollars per hour the AI could earn”, “Y is how many ops/sec is necessary to run said AI”, and “Z is the ability of the AI to acquire physical power relative to its economic productivity”, and saw unanticipated results which led some of us to believe that a typical takeoff curve might be steeper than we previously imagined. Our project was called The Uncertain Future.

Note that simple models of AI takeoff speeds can be useful even if human-level AI is 500 years away. (Although probably not if you think that human-level AI is impossible in principle.) Such models inspire us to think about a variety of questions unrelated to concrete future scenarios, including the usefulness of intelligence, how any agent might improve its own capabilities, and how humans perform on simple metrics of increasing their own efficiency and productivity. Investigating the question of potential takeoff curves is inevitably related to the question of whether we should have the audacity to speculate about futures with human-equivalent AI at all, and for me at least, the answer is yes, we should.

Modeling stuff

Looking at AIs and humans as caricatures in a simple model, the first thing that pops out is the differing modes of reproduction. Humans double their number about once every 40 years. AIs can double their number as long as they have the available computing power to do so.

So, pretend that the first human-level AI created needs 10^17 ops/sec to survive. If the AI or its owners want it to reproduce, they need to obtain another 10^17 ops/sec to create another AI. Because the computational requirements for intelligence are fixed while the cost of computing (currently) constantly changes, the cost of this computing hardware is likely to either be prohibitive (say, tens of millions of dollars) or reasonable (say, tens of thousands of dollars), with a somewhat smaller chance of being in-between (a million dollars). If the cost is prohibitive, then the AI will serve as little more than a curiosity (unless it displays qualitatively transhuman intelligence, in which case even tens of millions of dollars might be worth it for another). If the cost is reasonable, then the AI could be duplicated many times as long as its economic productivity pays well enough for buying more computers. Because of the problem-solving flexibility of human-level AI, much computing power would likely be taken away from conventional software and given to AI, though that is speculation.

Another fascinating element of simple AI takeoff models is the possibility of renting computing power, i.e., from cloud computing. If you rent computers, the barrier to entry on a new challenge is very low. Of course, this assumes that the AI doesn’t take an entire Internet’s worth of computing power to run, but if it’s so computation-hungry, then how did it get invented by anyone to begin with? (Setting aside “Digital Gaia” type scenarios for now.) Anyway, say that I’m an AI looking at craigslist. I see 100 contract jobs that pay $50/hr, all in my field of expertise, and I want to do them all, but if I don’t do them now, the employers will hire somebody else. What to do? Well, if I have the money, I can rent 100 computers to run temporary copies of myself until the jobs are all done. I take complete advantage of the available tasks, and I didn’t have to spend huge amounts of money to buy and cool and maintain 100 me-equivalents of computing power. As long as the jobs I did are enough to pay for the rental costs and then some, I can keep making money this way.

(Yes, these scenarios are somewhat anthropomorphic, that’s the point.)

Another possible scenario that comes to mind, which we didn’t address in our model, was the idea of an AI dividing itself into parts to complete tasks that don’t require its full complement of computing power. In a conservative model, an AI will always require all of its computing power to complete any task, but in the real world, self-division and self-recombination seem like perfectly possible moves. You could even have a collective of AIs that exist as either one big AI or a million little AIs as is required by the challenges that face them.

Once you have a model with 1) how much money the AI can initially make per hour with its skills, 2) computing power required, 2) cost of computers, and 3) how much money the AI can typically make per hour with its skills, there are other interesting variables you can introduce, like 4) will self-improvement make a difference, 5) where do returns from self-improvement level off, 6) will AI(s) acquire independent physical power, 7) will something halt the AI’s progress, how much physical/economic power is necessary for a monopoly on AI research, and 9) how much physical power will the largest AI/human group have at any given time. You can set 4, 5, and 6 to low values, put 7 and 8 at high values, and the takeoff curve is very gradual. That is, if you didn’t already set very high values for 1 and 2 and low values for 3.

The point here is that AI takeoff speeds can be analyzed as a debate about different numbers that make up the latent variables in any takeoff scenario. It’s not necessary to go off the deep end and accuse reasonable people of being Apocalyptic or Millennial, like James Hughes and some of the commenters here have done. What we have here are disagreements in the values of certain numbers. The arguments for the values of these numbers come from places like cognitive science and computer science, though no one can know them for sure in advance. The arguments shouldn’t come from appeals to faith, or social pressure, or being ironically hip or nihilistic.

The critics of fast AI takeoff scenarios act like those who see a hard takeoff as likely are engaging in an escapist techno-fantasy of wish-fulfillment. To the contrary, if we’re so nervous about AI, then wish-fulfillment would consist of a slower AI takeoff that is more manageable, not a fast one that takes us by surprise. Nothing could make me happier than if the challenge of AI could be handled by slow, careful approaches that have worked in the past — introducing or recalling products based on their popularity, usefulness, and feedback from society. But I so happen to think that the first human-level AI created could bootstrap itself to godlike status in a relatively short period of time, making it necessary to focus on that discrete event and not a slow, unfolding process. This belief about takeoff speed comes from the values I set for the 9 variables I listed.

Why don’t we take a look at my personal speculations about values for the variables, to get discussion going?

Variables that matter for AI takeoff speeds

Again:

1) computing power required
2) cost of computers
3) how much money the AI can typically make per hour with its skills
4) will self-improvement make a difference
5) where do returns from self-improvement level off
6) will AI(s) acquire independent physical power
7) will something halt the AI’s progress
how much physical/economic power is necessary for a monopoly on AI research
9) how much physical power will the largest AI/human group have at any given time?

We’d better get started. What is 1? Well, many people believe that intelligence can’t be simulated in a computer. Then, this whole exercise is over. If this is your position, you can stop reading now. Much of the confusion in this whole area of study comes from people who disagree with the entire premise coming in and attacking blindly. If you disagree that building the dam is even possible, then debating you on the details of how much the dam will cost or how many cubic meters of water it will hold back become moot points. In the real world, such a person would realize that they are bored by the discussion and leave the room, but on the Internet, people will offer their opinion on everything even if (especially if) they disagree with the premise.

Continuing, with only the group of people who believe that intelligence corresponds to a certain finite quantity of computational capacity, we have estimates of this value that range between 10^14 ops/sec (Hans Moravec, also Ray Kurzweil using Lloyd Watts’ estimate in TSIN) to about 10^17 ops/sec or even 10^19 ops/sec. There was even one odd paper that came out around 2005 that estimated human brain computing capacity at 10^100 or something like that. (Does anyone know where that went?)

Moving on, how much will computers cost at the time human-level AI is created? I assume that a human-level AI’s worth of computing power will not likely cost more than $10 million, or it would have been too costly to create to begin with, though your mileage may vary on this one. If you believe that creating AI requires unlocking the “secret of intelligence” (and brain emulation doesn’t work for some reason), and that secret won’t be here for centuries, though computers will continue to get faster until they slam into physical limits, then perhaps this value will be only $1 or less. It all depends on when you think AI is likely to happen and whether you believe Moore’s law will continue.

Next is how much money the AI can initially make per hour with its skills. This doesn’t really matter — say $10/hr. It can improve its skills and make more money as time goes on.

Will self-improvement make a difference? This is one of the more interesting questions and where intelligent people who believe in a soft takeoff and those who believe in a hard takeoff legitimately diverge. The soft takeoff thinkers appear to believe that self-improvement won’t make a difference because humans already have “broad-brush” general intelligence that makes us about as smart and productive as it’s possible for any intelligence to be. In that scenario, AIs can only increase their productivity by using up more computing power to make more copies of themselves.

In the model, I wrote the introduction to this variable as follows: “If an AI has human-level intelligence, it will be capable of analyzing its own programming and improving itself to some degree. How much more productive will a self-improving AI become per work-hour of self-improvement invested?” So, say the AI invests 100 work-hours, similar to a series of training seminars (though these might involve the AI actually reprogramming its mind directly), and makes itself 1% more effective. We can translate this as saying the AI makes $10.10 an hour instead of $10.00, or that it can make $10 in 36 seconds less than an hour. It makes little difference to the model. You can measure the benefit in a broader magnification of thinking and abilities if using money as the indicator makes you uncomfortable.

So, you can set any value you want for the difficulty of self-improvement. Depending on how difficult it is, there is an optimal growth-rate-maximizing amount of money/time to delegate to self-improvement instead of other activities, so choosing how much time the AI devotes to self-improvement is somewhat superfluous, though you can add that in too if you want.

Here’s an example of improvement difficulty in the human realm. Say that going to college increases your earning power by 50% (doubtful, but this is just an example). If so, then assuming around 6,400 hours of class attendance and homework, the improved efficiency per hour of investment is roughly 0.0075%. In reality, the difficulty of self-improvement is not a smooth value, it could zig-zag all over the place as you go through successive S-curves of self-improvement. Still, in today’s world, expected self-improvement rates of 0.001% to 0.01% per hour of work seemed typical, based on the examples I looked at. You can use this value if you want, or put in something higher if you think that AIs would be better at finding out ways to self-improve than humans, or if you think that having complete read-write access to every bit of your own mind would make any difference. In a hard takeoff, I speculate that improvement rates of 100-1,000% per hour could be achieved, but the conservative answer is much lower.

#5 asks where returns from self-improvement level off. You can’t self-improve forever. Maybe a superintelligence can program 10^17 of computation as part of a 10^38 ops/sec Jupiter Brain such that it is a million times more effective at using the same computation as humans are. (Here, “effective” can be defined in any number of ways, but for the sake of this model it is ability to get money/computing, which leads to physical power based on another variable we will talk about in a second.) Or maybe humans are pretty much as good as you can get for 10^17 ops/sec and further improvement is impossible. (Yeah, right.) Or maybe we can pick something more in the middle like saying that becoming 100 times more effective is possible but 1000 is not. Whatever you pick becomes the growth ceiling for variable #3 (how much money the AI makes per AI-equivalent of computing power) as it evolves over time due to continuing investments in self-improvement.

#6 asks how many AI-equivalents of computing power will be “exchanged” for human-equivalents of physical power. Again, smart people have major disagreements on this. If you think that AI will always just consist of brains in boxes ready to do our bidding, and programmed not to want power, then the answer is that AIs will not want to exchange their computing power/money for physical power very much at all. Still, this value can’t be zero. AIs with any input into the world whatsoever are certain to have some degree of physical power, at the least by making suggestions to humans about what to do. At the other end of the spectrum is a transcending AI plugged into a gigantic living sphere of active robotics. If the latter could defeat a million humans worth of physical power (your definition of what that is can vary arbitrarily, we only need to agree if we are comparing notes directly), but it only used 1,000 AI-equivalents worth of computing power, then you’d say that there is a money/computing to physical power ratio of about 1,000:1,000,000, or 1:1,000. The ratio could just as easily be 1,000:1 if you want, it all depends on your opinion.

The reason I decided to introduce physical power to the model aside from computing/money is the large variance and disagreement about how readily one transfers to the other. Some models make the mistake of including computing/money only, and part of the reason few people comment on these models is that computing/money doesn’t always translate to power in the real world.

Question #7 asks if something unusual will halt AI progress in general. This question is just a probability. For instance, the probability that all AI research gets banned or that a meteor strikes the planet sometime when the AI is in the middle of figuring out ways to improve itself. I would put this value at less than 10%, but others will put larger values, especially if they think that some other disaster is likely to wipe out civilization before we make it to AI.

Question #8 asks how many human-equivalents of physical power would be necessary to monopolize all AI research and development. This means that the entity/union with this level of power would be able to prevent the creation of AIs they consider dangerous or restrict the growth of AIs with sufficiently divergent goals from their own. I’d hazard to guess that the answer is more than 100 million and less than a billion.

Question #9 asks how many human-equivalents of physical power the largest AI/human collective has at any given moment, as a percentage of total power available. Today, the United States has about 500 million human-equivalents of physical power, related to the fact that about 500 million people live here, which translates to about 8% of global total power, though you can see that this is just a wild estimate because the USA’s power is really greater than just 8%.

If everyone in the United States had a human-equivalent robot that took their orders, then we’d have 1 billion human-equivalents of computing power, which would give us 15% of total power or whatever. The answer to this question is intimately related to what you said for every other question, especially how readily computing power/money translates into physical power, but I’d say that if the takeoff curve is sufficiently sharp, the resulting collective may control a substantial amount of total global power, say 30% or more. (Preferably, this would be a democratic collective consisting of everyone on the planet, but some forms of AI might disagree with my preferences here.)

That’s it! There are other questions we put in our model, but those are the ones relevant to AI takeoff speeds. By doing calculations using all the relevant variables, we can sketch out various takeoff scenarios. Some other time I might discuss how to connect all these variables together mathematically, but for now I just leave you with the variables and thinking of how to put them to use is up to you.

12 Responses »

“How long before superintelligence” is a paper by Nick Bostrom. Here is the abstract:

“This paper outlines the case for believing that we will have superhuman artificial intelligence within the first third of the next century. It looks at different estimates of the processing power of the human brain; how long it will take until computer hardware achieve a similar performance; ways of creating the software through bottom-up approaches like the one used by biological brains; how difficult it will be for neuroscience figure out enough about how brains work to make this approach work; and how fast we can expect superintelligence to be developed once there is human-level artificial intelligence.”

The paper is also updated with several postscripts, including one from 2008, which says:

“I should clarify what I meant when in the abstract I said I would “outline the case for believing that we will have superhuman artificial intelligence within the first third of the next [i.e. the this] century”. I chose the word “case” deliberately: In particular, by outlining “the case for”, I did not mean to deny that one could also outline a case against. In fact, I would all-things-considered assign less than a 50% probability to superintelligence being developed by 2033. I do think there is great uncertainty about whether and when it might happen, and that one should take seriously the possibility that it might happen by then, because of the kinds of consideration outlined in this paper.

There seems to be somewhat more interest now in artificial general intelligence (AGI) research than there was a few years ago. However, it appears that as yet no major breakthrough has occurred.”

Recently on Nanodot, Foresight Institute President J. Storrs Hall said:

“I would guess — and this is blatantly a speculation, albeit a fairly well informed one, that the “secret trick” of AI will fall in the next decade. That means that the 20s will see robots not just as good as humans at specific, well-defined tasks, but able to learn new tasks the way humans do.”

Have we not learned anything? The very idea that there is any discrete “secret trick” is reminiscent of the physics envy that pervades thinking on AI. Fortunately, such beliefs delay work on AGI in general, leaving more time for Friendliness Theory to be developed.

20 Responses »

According to Vinge’s definition, the Singularity is the creation of greater than human intelligence. Vinge says this could happen in four ways:

1. The development of computers that are “awake” and superhumanly intelligent. (To date, most controversy in the area of AI relates to whether we can create human equivalence in a machine. But if the answer is “yes, we can”, then there is little doubt that beings more intelligent can be constructed shortly thereafter.)

2. Large computer networks (and their associated users) may “wake up” as a superhumanly intelligent entity.

3. Computer/human interfaces may become so intimate that users may reasonably be considered superhumanly intelligent.

4. Biological science may find ways to improve upon the natural human intellect.

I personally consider all of these plausible except for 2, which I can debunk in a separate post if there is any interest. I am also skeptical about 4, but I know some scientists that assure me it is plausible in the relevant time frames.

“Singularitarianism”, as defined in 2000, basically just means, “we want to deliberately do that” (create greater than human intelligence). It doesn’t specify why, except saying that you have to want it to benefit everyone, not just yourself. Maybe you want a Singularity to revive your dead father. Maybe you want a Singularity because you decided it’s a better way to pursue humanitarian goals than working on them with human intelligence alone. Maybe you want to become a greater than human intelligence yourself so that you can be a wiser person less prone to human psychological frailty and error. Maybe you think that a Singularity is inevitable eventually, and it might as well be done in a guided rather than unguided fashion. Maybe you think that a greater than human intelligence could wipe us all out, and taking an active role in its creation is necessary to avoid global doom. There are literally hundreds of reasons why someone might want a Singularity, ranging from the banal to the enlightened.

Singularity advocacy is often subjected to accusations of techno-utopianism. In fact, “Singularitarianism” is linked in the “See also” section of technological utopianism on Wikipedia. According to Bernard Gendron, a professor of philosophy at the University of Wisconsin-Milwaukee, there are four principles of technological utopians:

1. We are presently undergoing a (postindustrial) revolution in technology;
2. In the postindustrial age, technological growth will be sustained (at least);
3. In the postindustrial age, technological growth will lead to the end of economic scarcity;
4. The elimination of economic scarcity will lead to the elimination of every major social evil.

As a Singularitarian, what do I think of these principles? Well, 1 seems true. Technological advances in recent decades have been amazing. The world is very different than it was just 15 or so years ago, when I started to pay close attention to what was happening in technology. This wave of technology seems likely to continue, albeit somewhat slowed by the current recession.

What about 2? Seems true as well. It appears that there’d have to be a pretty severe Depression for technological growth to actually stop rather than just slow.

Already, I’m slightly confused. How are 1 and 2 controversial at all? Are they supposed to be?

Alright, on to 3 and 4, the meaty parts. #3 is the idea that technological growth will lead to the end of economic scarcity. Wouldn’t that be nice! Unfortunately, people are never satisfied, so in a certain sense, ending economic scarcity will be impossible forever. By the standards of the Middle Ages, many would say that economic scarcity is already over. Even a not-so-wealthy person such as myself can buy a wheelbarrow full of buttered bread if they want to. The issue is that new, expensive products keep getting introduced, pushing the economic scarcity bar further away. I cannot afford a wheelbarrow full of iPods.

So, keeping in mind the moving goalposts, will economic scarcity ever be solved? Not likely. Perhaps what is meant by “elimination of economic scarcity” is that economic scarcity (by a lenient definition, such as having all the food and water you need) would be eliminated on the entire planet. Surely this will take a while. My concern with principle #3 here is that one can arbitrary define “economic scarcity” in a convenient way such that that goal is never achieved. If one is looking to confirm one’s expectations that “techno-utopians” are idiots, then one will merely define that goal in such a way that even mentioning it could be achieved is silly. Very convenient. Setting up strawmen is an easy way of avoiding debates with real people.

But still, if economic scarcity could ever be defeated, how else could it be done but with technology? By wishful thinking? Magic? Religion? Are you “techno-utopian” if you think that economic scarcity will be largely solved in 20 years? Or 50? or 100? Or 10,000? If you claim that economic scarcity wouldn’t be solved even after 10,000 years of technological progress, then are you being sincere, or do you just believe that eventually eliminating economic scarcity would be a really bad thing, or make technology sound too useful, when we’re supposed to be exclusively focusing on social issues and pretty much ignoring technology? If technological progress continues, it seems that either 1) “economic scarcity” will eventually be eliminated, or 2) eliminating economic scarcity is a receding goal that can never be achieved. Which is it?

Now, on to #4, the idea that “the elimination of economic scarcity will lead to the elimination of every major social evil”. This is so obviously false, I can’t help but consider it a strawman. Did anyone ever make this claim? If so, it’s pretty ridiculous, as having more toys often seems to make people more materialistic, elitist, and competitive, not less. By associating Singularity advocacy with this ridiculous claim, critics seek to demolish our work towards a much simpler goal: creating greater than human intelligence (superintelligence). Yes, if superintelligence were created, it could invent many new technologies, some of which we may find tremendously useful. Superintelligence could even be consulted on how to mediate our disputes and arrange society for the greatest good for the greatest number. Superintelligence could provide help on anything for which intelligence is useful, including logistics, philosophy, economics, aesthetics, and much more. Technology is only a tiny piece of the picture.

Of course, Kurzweil’s viewpoint of the Singularity places extreme emphasis on technology, not the unique benefits of superintelligence in its own right. Because I disagree with this emphasis on technology, I am left but no choice but to reject Kurzweil’s presentation of the Singularity, even if I find his detailed analyses and philosophical viewpoint unique and interesting. I don’t actually think that Kurzweil is techno-utopian in the sense as defined by Gendron’s principles, because he never explicitly says that all social ills will be solved by technology. However, his near-exclusive emphasis on technology and its capacity to solve problems has a techno-utopian flavor that turns off millions of reasonable people. Thus, it’s worth distancing ourselves from.

For me, the goal of greater than human intelligence is great for the same reasons that human intelligence is great: thoughtfulness, reason, creativity, concern, brilliance, spontaneity, unpredictability, character. I want us to take the greatness of humanity and magnify it by attaching it to cognitive mechanisms with superior problem-solving, idea-imagining, mental imagery-manipulating, information synthesizing, and category forming abilities. The notion that superintelligence would also invent technologies that could help people is a secondary effect. The source of the help is not the technology, it’s the intelligence that creates it and deploys it and adds personal touch and care to that deployment. This is crucial, and all the critics of the Singularity miss the fact that that’s what many of us have meant by “Singularity” all along.

11 Responses »

Dr. Richard Jones is Senior Strategic Advisor for Nanotechnology for the UK’s Engineering and Physical Sciences Research Council. In 2008, Jones published a book, Soft Machines: Nanotechnology and Life, that describes why he thinks that advanced nanotechnology will go along more of a biomimicry and organic path rather than rigid structures designed with a mechanical engineering mentality. Soft Machines is published in the UK and the USA by Oxford University Press. For a couple years now I’ve been following Jones’ commentary on nanotechnology, and I respect his viewpoint.

In a recent post on his blog, Jones responds to a response from Robert Freitas and Dr. Ralph Merkle to Jones’ article “Rupturing the Nanotech Rapture”, published in the IEEE Spectrum special issue on the Singularity from June 2008, a paranoid hit piece on the Singularity that editor Glenn Zorpette disingenuously and dishonestly presented as a balanced review. (I made a response of my own shortly after the article was published.) The general gist of Jones’ position is that molecular nanotechnology based on mechanical engineering principles and rigid structures will never be successful, and that organic and soft structures are the future of nanotech. Meanwhile, Rob Freitas and Ralph Merkle have been championing the rigid, mechanical engineering-type approach for well over a decade. This blog post by Jones is the most recent message in a round of debating that has been ongoing between the soft approach and the rigid approach for two decades.

The relevance of the debate between advocates of soft nanotech vs. rigid nanotech (”molecular manufacturing” or “molecular nanotechnology” (MNT)) is that if rigid nanotech is impossible, many of the technologies anticipated by transhumanists and futurists — Santa Claus machines, advanced cybernetics, superabundance, rapid manufacturing, self-replicating nanofactories — may prove impossible or much more challenging than common transhumanist timetables would suggest. Ray Kurzweil’s predictions would be a common example of timetables — in Kurzweil’s books, he basically takes the development of molecular nanotech for granted, anticipating it rolling out in the 2020s. The thing is, while the feasibility of molecular nanotech seems likely to me personally, I consider a 2020s development timeframe to be implausibly soon. 2040s seems more likely, if the technology is even possible at all. This puts me in stark disagreement with many who comment on molecular nanotech, including Ray Kurzweil and the Center for Responsible Nanotechnology, the latter organization writing on their website that molecular nanotechnology “might become a reality by 2010 to 2015, more plausibly will by 2015 to 2020, and almost certainly will by 2020 to 2025″.

(When I say dates like “the 2040s”, note that I’m discounting the possibility of superintelligence accelerating things along or a global catastrophic event halting things in their tracks. Sometimes people call this “CRNS” meaning “Current Rate No Singularity”. Dates like this also discount the possibility of nuclear war, sustained global economic depression, or unexpectedly rapid technological growth.)

On their Nanofactory Collaboration page, Freitas and Merkle point towards 2030 as a likely date for nanofactories if their direct-to-DMS effort has sufficient funding ($1M-$5M/yr).

Besides the back-and-forth between Freitas/Merkle and Jones, another interesting element to the debate has appeared in recent months, with Eric Drexler — the “father of nanotechnology” himself — criticizing the DMS (diamondoid mechanosynthesis) research path of Freitas and Merkle, as pursued by their Nanofactory Collaboration team. In a December 2008 blog post, Drexler called diamond synthesis “a bad approach” and said, “Contrary to popular opinion, diamond synthesis seems almost irrelevant to progress toward advanced nanosystems. At the current stage or research, it is both difficult and unnecessary”. Drexler recommends instead starting with organic “soft machines”, using those to build pyrite, magnetite, and keratin-like structures. He writes, “mechanosynthesis begins with soft machines” and that “there is no gap between soft and hard nanomachines: The technologies form a continuum, and working together, they can form a bridge”. Very poetic!

Back on the Freitas/Merkle side, a 28 December 2008 update to the Nanofactory Collaboration website linked to Drexler’s post and said, “Our assessment is that diamondoid mechanosynthesis (DMS), including highly-parallelized atomically-precise diamondoid fabrication, is the quickest currently feasible route to a mature molecular nanotechnology, including nanofactories. We do not think that DMS is a “necessary first step” for molecular manufacturing, and we wish the best of luck to those pursuing other paths. However, we do think DMS is a highly desirable first step, since it offers a much faster route to mature nanosystems than competing approaches. We disagree with the statement that “diamond synthesis seems almost irrelevant to progress toward advanced nanosystems.” We have a favorable view of the feasibility of the direct-to-DMS approach – a favorable view supported by hundreds of pages of detailed analysis in recently-published peer-reviewed technical journal papers and by gradually-evolving mainstream opinion“. As someone who has followed and admired the work of Freitas, Drexler, and Merkle for over a decade, this is like “Clash of the Titans” to me. What could possibly be more stimulating than considering the merits of each side? Well, a few things, but this ranks up there.

Meanwhile, writing in “Rupturing the Nanotech Rapture”, Dr. Jones scoffs at the prediction of near-term digital control of matter granted by diamondoid nanomachines. Jones laughs at the stance of Ray Kurzweil and other “singularitarians”, remarking, “In 15 years of intense nanotechnology research, we have not even come close to experiencing the exponentially accelerating technological progress toward the goals set out by singularitarians”. Here we have a slightly annoying situation which I keep running into again and again. I consider myself a “singularitarian” by the predominant definition of 2000-2005, the one laid out by Eliezer Yudkowsky, not the new definition presented by Ray Kurzweil in 2005, which defines a Singularitarian as someone “who understands the Singularity and who has reflected on its implications for his or her own life”, a definition so broad as to be meaningless. I can’t tell what Jones means when he says “singularitarian”, as he doesn’t define it, but I take it he means the latter definition, or perhaps more specifically, people that agree with all the predictions of Ray Kurzweil. Maybe he could clarify. In any case, I am a singularitarian according to the 2000-2005 definition, and completely reject the vague and too-inclusive 2005-2009 definition.

With regard to Jones’ comment, I regard the Core Claim of the “Accelerating Change” school of the Singularity — that rates of technological progress in the past are an unreliable predictor of the future — as plausible, but reject the Strong Claim: “Technological change follows smooth curves, typically exponential. Therefore we can predict with fair precision when new technologies will arrive, and when they will cross key thresholds, like the creation of Artificial Intelligence”. Adhering to the Strong Claim of the Accelerating Change school is what leads Kurzweil to say implausible and overconfident things like that human-level AI will be invented in precisely 2029.

In his article, Jones does not reject molecular nanotechnology outright, but says, “I can’t take seriously the predictions that life-altering molecular nanotechnology will arrive within 15 or 20 years and hasten the arrival of a technological singularity before 2050″, and that “Complete control will remain an unattainable goal for generations to come. But some combination of self-assembly and directed assembly could very well lead to precisely built nanostructures that would manipulate the way light, matter, and electrons interact—an application of nanotechnology that’s already leading to exciting new discoveries”. Instead of totally dismissing radical nanotechnology visions, Jones says, “We shouldn’t abandon all of the more radical goals of nanotechnology, because they may instead be achieved ultimately by routes quite different from (and longer than) those foreseen by the proponents of molecular nanotechnology”. I think that is definitely a possibility, but that diamondoid mechanosynthesis looks feasible enough that it should still be pursued. It is difficult to condemn the DMS approach until it receives adequate funding and continues to fail despite that. The approach has barely even begun to be tested — why dismiss it so soon?

Of course, according to Jones, transhumanists artificially inflate their subjective probability of the feasibility of MNT because MNT would be helpful to achieving transhumanist goals like extreme life extension and nanomedicine. Indeed, there is an entire section in the Transhumanist FAQ devoted to molecular nanotechnology, which says, “A common guess among the cognoscenti is that the first assembler may be built around the year 2018, give or take a decade, but there is large scope for diverging opinion on the upper side of that estimate”. This is a fairly noncommittal and broad estimate — it basically says that the first assembler will be built between now and about 2030, but the upper limit is hazy. Such a vague guess leaves much room for fidgeting and revision over the next couple decades.

It’s hard to tell how much stock transhumanists place in MNT. Many of them make a big deal out of it, like the transhumanists that show up at Foresight Institute conferences (probably making up about 40% of the attendants), while others don’t pay it much mind, like James Hughes at the Institute for Ethics and Emerging Technologies. Myself, I think it’s much more important to create an unbiased probability estimate of the plausibility of MNT than to make a biased estimate for dumb reasons. In his response to Freitas and Merkle, Jones writes, “what is not forbidden by the laws of physics is not necessarily likely, let alone inevitable. When one is talking about such powerful human drives as the desire not to die, and the urge to reanimate deceased loved ones, it’s difficult to avoid the conclusion that rational scepticism may be displaced by deeper, older human drives”. Regarding Kurzweil, I’m starting to think that Jones is right. A recent interview in Rolling Stone with Ray Kurzweil said:

“Using technology, he plans to bring his dead father back to life. Kurzweil reveals this to me near the end of our conversation … In a soft voice, he explains how the resurrection would work. “We can find some of his DNA around his grave site - that’s a lot of information right there,” he says. “The AI will send down some nanobots and get some bone or teeth and extract some DNA and put it all together. Then they’ll get some information from my brain and anyone else who still remembers him.”

This makes no sense to me, and is somewhat creepy. Even if you could create a clone of your father and fill it with memories from faded memories of other people, that wouldn’t be your father in any meaningful sense. He wouldn’t remember most of his life, as people have many experiences alone, and many of the people that observed his father’s childhood would already be dead, making their memories irrecoverable. What kind of puppet would only have memory of himself up to the birth of his children? I respect much of Kurzweil’s thinking, but this new claim is just weird.

As a transhumanist, on a personal level, I am not emotionally attached to molecular nanotechnology. I pay attention to it because I think the technology could plausibly be developed, maybe in the next 20 years, and if it is, the impact it has on the world would be huge, especially in the manufacturing sector. But if it turns out to be impossible, then those ideas should be discarded, and we should move on to something new. As Jones himself says, “We shouldn’t abandon all of the more radical goals of nanotechnology, because they may instead be achieved ultimately by routes quite different from (and longer than) those foreseen by the proponents of molecular nanotechnology”. This is completely correct. So, those transhumanists that have a deep emotional attachment to molecular nanotechnology, nanomedicine, and/or nanobots should drop it. If accelerating the development of MNT is even your goal, you can help the idea’s reputation by acknowledging the probabilistic, non-certain nature of its development. The goal of MNT has already been tarnished enough by starry-eyed futurists attracted to the idea mostly for its promise of science fiction-like outcomes. Such delusion hampers the efforts of serious theorists like Freitas, Merkle, and Drexler.

8 Responses »

I usually don’t just post links, but this was a really interesting news week, so I’m going to go over some of the stuff I saw that is worth caring about.

Nanowerk: NanoInk introduces new desktop nanofabrication system

NanoInk introduces the next generation Dip Pen Nanolithography® system for desktop nanofabrication, the DPN 5000. Having evolved from the popular NSCRIPTOR™ DPN® System, this new instrument brings greater control and performance to the world of desktop nanofabrication. The DPN 5000 offers versatile nanopatterning capabilities coupled with high-performance AFM (atomic force microscopy) imaging for immediate characterization of the deposited patterns. NanoInk has developed a variety of custom MEMS (micro electromechanical systems) based ink delivery devices, allowing a wide range of materials to be deposited under precisely controlled conditions.

Springerlink: Neurogenesis and Exercise: Past and Future Directions

Research in humans and animals has shown that exercise improves mood and cognition. Physical activity also causes a robust increase in neurogenesis in the dentate gyrus of the hippocampus, a brain area important for learning and memory. The positive correlation between running and neurogenesis has raised the hypothesis that the new hippocampal neurons may mediate, in part, improved learning associated with exercise. The present review gives an overview of research pertaining to exercise-induced cell genesis, its possible relevance to memory function and the cellular mechanisms that may be involved in this process.

Nanowerk: It’s all in the wiring: biocomponents at the heart of an artificial photosystem

Plants, algae, and cyanobacteria (blue-green algae) are masters of everything to do with solar energy because they are able to almost completely transform captured sunlight into chemical energy. This is in part because the electrons set free by the photons are transported out of the “light receptor” to be used as the driving force for chemical reactions. Japanese researchers have now developed a new process to capture light energy with nearly equal efficiency.

Eurekalert: Chemists create two-armed nanorobotic device to maneuver world’s tiniest particles

Chemists at New York University and China’s Nanjing University have developed a two-armed nanorobotic device that can manipulate molecules within a device built from DNA. The device is described in the latest issue of the journal Nature Nanotechnology. “The aim of nanotechnology is to put specific atomic and molecular species where we want them and when we want them there,” said NYU Chemistry Professor Nadrian Seeman, one of the co-authors. “This is a programmable unit that allows researchers to capture and maneuver patterns on a scale that is unprecedented.” The device is approximately 150 x 50 x 8 nanometers. A nanometer is one billionth of a meter. Put another way, if a nanometer were the size of a normal apple, measuring approximately 10 centimeters in diameter, a normal apple, enlarged proportionally, would be roughly the size of the earth. The creation enhances Seeman’s earlier work—a single nanorobotic arm, completed in 2006, marking the first time scientists had been able to employ a functional nanotechnology device within a DNA array.

Eurekalert: Scientists isolate genes that made 1918 flu lethal

By mixing and matching a contemporary flu virus with the “Spanish flu” — a virus that killed between 20 and 50 million people 90 years ago in history’s most devastating outbreak of infectious disease — researchers have identified a set of three genes that helped underpin the extraordinary virulence of the 1918 virus. Writing today in the Proceedings of the National Academy of Sciences, a team led by University of Wisconsin-Madison virologists Yoshihiro Kawaoka and Tokiko Watanabe identifies genes that gave the 1918 virus the capacity to reproduce in lung tissue, a hallmark of the pathogen that claimed more lives than all the battles of World War I combined.

ABC: Scientists stop the aging process

Scientists have stopped the aging process in an entire organ for the first time, a study released today says. Published in today’s online edition of Nature Medicine, researchers at the Albert Einstein College of Medicine at Yeshiva University in New York City also say the older organs function as well as they did when the host animal was younger. The researchers, led by Associate Professor Ana Maria Cuervo, blocked the aging process in mice livers by stopping the build-up of harmful proteins inside the organ’s cells.

One Response »

Q. What do you mean by “superintelligence”?

A. Nick Bostrom defined superintelligence in 1998 as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills”. This isn’t incredibly specific, because “much smarter” depends entirely on what you mean by “much”. A stubborn skeptic might exclude any future being from the category simply by refusing to grant that entity X is “much” smarter than humans. The definition of “much” is necessarily subjective, but after a certain point, it seems likely that more than 90% of a survey population would agree on what “much smarter than the best human brains” means.

Allow me to create a more specific definition. I propose that superintelligence be defined as a being at least as smarter than Homo sapiens as we are smarter than Homo heidelbergensis. Why Homo heidelbergensis? According to the current consensus, H. heidelbergensis is the most recent known ancestor of modern day H. sapiens. Our species diverged from H. heidelbergensis about 400-200,000 years ago. This puts the species comparatively closer to us than more famous members of genus Homo including H. erectus (split 1.2 mya) and H. habilis (split at least 1.5 mya).

Of course, this definition is not perfect. The fossils that anthropologists call H. heidelbergensis are distributed over a 350,000-year period — between about 600,000 and 250,000 years ago, with considerable variation across that period. While this is a specific time window relative to most fossil species, it’s still a substantial period of time, and H. heidelbergensis has been accused of being a “wastebasket taxon”. The origin of the category is those problematic fossils with both “erectus-like” and “modern” features, formerly called Archaic Homo Sapiens. Traits of some specimens are listed here. One of the distinguishing features is a larger cranial capacity than earlier Homo species — approximately 1200 cc, relative to the modern human average of 1350 cc.

Still, because H. heidelbergensis is considered by most scientists to be a valid species, for which there is substantial fossil evidence, it provides a concrete reference point in the interspecies intelligence space. We can define “intelligence” using the wiktionary definition, “Capacity of mind, especially to understand principles, truths, facts or meanings, acquire knowledge, and apply it to practice”. H. heidelbergensis had it, as is demonstrated by the stone tools and evidence of organized hunting left behind, and H. sapiens obviously has it in even greater measure. Consider another species, H. novus, with an equal measure of greater intelligence relative to humanity. Intelligence may not be quantifiable on a linear scale, but that does not prevent this definition from being useful in a qualitative sense.

According to my source on Homo heidelbergensis, “The increase in brain size may have also come with an increase in brain complexity, although this is difficult to determine from endocasts, and may have to remain supposition only. However, the increase in absolute size, and the change to larger frontal and parietal lobes indicate that there may have been a reorganization of the functional anatomy of the hominid brain. The increase in size itself indicates changes in behavior that lead to the ability to more easily acquire nutritional resources. This is due to the high nutrition requirements of brain tissue, especially during development. There is increasingly more convincing evidence in the use and control of fire, and in the hunting of animals for food. This time period is important for many reasons, and may be the time period when more modern behavior began to develop.”

There were obviously major changes in the hominid brain around this time, but in the case of Homo heidelbergensis, it apparently wasn’t enough to launch the cognitive revolution ushered in by H. sapiens when it appeared on the scene about 200,000 years ago. This distinct failure should not go unnoticed. If the cognitive solution space were structured differently, Homo heidelbergensis might have had the capacity to develop agriculture and a global civilization, as H. sapiens eventually did. If so, we’d all be members of Homo heidelbergensis, and H. sapiens might never have evolved, the fast clip of hominid cognitive evolution being cut off by its own success. Phenotypic change happens fastest when there is strong selection pressure and beneficial alleles to evolve towards, but the triumph of H. sapiens over our environment has lifted the intense selection pressure experienced by our ancestors. The evolution of new human species via natural means now appears unlikely.

Because Homo heidelbergensis lived only a few hundred thousand years ago, it may be possible to recover enough DNA to revive the species and create living individuals. If so, then the cognitive performance of this species relative to modern humans could be directly compared. Until then, we will have to settle with what we know about the species from fossil and other archaeological evidence. Homo heidelbergensis produced Acheulean stone tools, also known as Mode 2, the second of four divisions of prehistoric stone-working industries. Mode 1 tools were Clactonian or Oldowan tools, which consisted of simple choppers and unretouched biface hand axes. Mode 2 tools consisted of retouched biface hand axes with an average useful cutting edge of 20 cm (8 in), relative to the less useful 5 cm (2 in) average of Mode 1 tools. Mode 2 tools are associated with H. erectus, Homo heidelbergensis, and other pre-human hominins. Mode 3 tools are Mousterian tools, which use the prepared-core technique. Mode 3 tools appeared just 200,000 years ago and are only associated with humans and Neanderthals. Mode 4 tools include the more spectacular lithic blades and are associated with human cultures from the Upper Paleolithic.

For the sake of simplicity, let us assume that Homo heidelbergensis never would have been able to develop Mode 3 tools, even if given millions of years to do so. Indeed, Mode 3 tools were invented practically immediately after the evolution of H. sapiens. A corollary of this is the claim that Homo heidelbergensis would never have been able to develop behavioral modernity, agriculture, or cities. If Homo heidelbergensis were alive today, it would almost certainly be considered too stupid to qualify as legally human, but it would probably occupy a unique intermediary role between humans and other living animals such as chimps. Whether Homo heidelbergensis had language is entirely unknown, but it might have. At the very least, its system of grunting and other non-verbal communication would likely have been substantially more complex than that of chimps or gorillas.

My definition of superintelligence is just an interesting and slightly more specific way of looking at the concept than the standard definition. It makes no specific claims about the form of a possible H. novus, just that it would be broadly smarter than us in the way that we’re smarter than Homo heidelbergensis. I advance the concept as a starting point for debate. Under this definition, there are those that might consider superintelligence impossible, and those who might consider it possible. Sometimes the word “superintelligence” is associated with entities like Jupiter Brains, so this new definition provides a service by offering a less radical and more easily defensible position. Personally, it seems difficult to deny the in-principle possibility of superintelligence under this definition, but many would likely disagree with me.

13 Responses »

Say that you’re a venture capitalist and some researcher-entrepreneur comes up to you, pitching an idea for a new battery that costs the same as current lithium-ion batteries but holds 50% more charge. You’d evaluate that technical proposal on technical grounds — reviewing your own knowledge of the physics of batteries as well as consulting others who are scientifically well-versed in the matter to determine whether the proposal is possible. Near-term feasibility would also enter into the equation — if the battery took a very long time to make, then you might prefer to invest in something else with a near-term payoff.

Now consider the proposal that many transhumanists are putting forth — that we should create Friendly, roughly human-equivalent AI with the capacity for recursive self-improvement. Obviously, this is a more ambitious project that creating a better battery. Still, like the battery, it deserves an evaluation on technical grounds to the extent that this is possible. Unfortunately for advocates of Friendly AI, this proposal is also subject to obstacles of questioning that the battery proposal is not — evaluations based on philosophical baggage. Batteries aren’t loaded down with much philosophical baggage, due to their near-indifferent political relevance and general absence of moral valence.

The philosophical baggage around the possibility of self-improving Friendly AI is abundant. Even the very notion of human-equivalent AI offends human sensibilities, not to mention the idea that morality is something that can just be… programmed. To scientific materialists who believe in functionalism, many of these questions are non-issues, but Friendly AI is still subjected to scrutiny above and beyond a straightforward technical proposal, much of it justified. Ethical questions like, “would it be right to create something smarter than us?”, feasibility questions like, “can a being of intelligence N create a being of intelligence N+1?”, and so on. These numerous questions make arguing that Friendly AI is feasible and a good idea very challenging and multi-faceted. A philosopher might do well to take up the issue as an interesting challenge even if they had no personal attachment to it.

Exploring the philosophical obstacles around acceptance of Friendly AI is a topic worth 1,000 posts at least, so I won’t go into detail here, merely call attention to their existence. I will just mention the second challenge in the battery proposal as applied to Friendly AI — the near-term feasibility issue. As Ray Kurzweil argues in The Singularity is Near, if brain-scanning resolution and computing power continue to improve at the rate they have for decades, then we will have the technology to upload detailed functional algorithms of the human brain by around 2030. Arguing against this upper limit requires either stating that 1) brain scans and computers will halt their exponential improvement at some point in the next 20 years, 2) extremely subtle and complex low-level features must be duplicated to duplicate human intelligence, or 3) human intelligence cannot be simulated in a computer, even in principle.

I feel that arguments #2 and #3 can be dismissed rather readily. I also think that much of the motivation for standing behind #2 comes from a milder sentiment of #3, namely that if human intelligence can in fact be simulated in a computer, we should expect it to take hundreds of years for the task to be accomplished. That is the position of Doug Hofstadter. Those that take this line of argument would be very disappointed if humans were “so simple” that simulating our minds in computers by 2030 would be possible. Rebutting this argument comprehensively would take numerous posts on its own — I hope you realize why I’m bringing up issues without exploring them each in detail. (I would be writing this post for hours and have no time for anything else.) I can just say that the complexity and awe of the human mind would be in no way diminished if it turned out that analyzing features “merely” at the neuron level — of which there are 100 billion — would be a sufficient level of description. There are arguments that can be made from the way that evolution works and cognitive science experiments that make it implausible that exabytes of specific sub-neuron details of brain biochemistry play an absolutely necessary and indisposable role in minds-in-general.

I think that a lot of the ideological fuel behind #2 and #3 comes from the misconception that AI would need to be a straightforward copy of the human brain to work at all. This argument is on par with the idea that someone would need to perfectly copy a bird to make a flying machine, or perfectly copy a mole or earthworm to make a digging machine. The reason that people consider it acceptable to apply to intelligence and not to digging is because of the mystical association of intelligence with a specific Gift From God, a unique cosmic quality only bestowed on Homo sapiens sapiens due to our unique specialness and spiritual importance. For some that have been raised to believe this since they could understand language, changing their minds might be outright impossible. They might also act like the targets in whack-a-mole, avoiding making specific claims about their deep-held view of human exceptionalism, instead extending arguments which sound superficially more plausible, but in fact are only advanced because of deeper reservations about the entire enterprise and its philosophical implications.

Still, say that I granted that intelligence was too complex to instantiate in computers for hundreds of years. Even then, I would argue that the near-term feasibility requirements usually considered so important for research proposals and venture capital-funded projects would be absolutely inapplicable in this case. Due to the earth-shaking implications of creating another intelligent species, especially one which would have the qualities of “instant intelligence, just add computing power”, pursuing a benevolent course for the technology would be justified, even if we didn’t think it would arrive for hundreds of years. Failure would mean the extinction of our species and its replacement by AI (a highly undesirable outcome in the eyes of many), while success would mean a tremendous injection of intelligence and wisdom into our society, wisdom that could be turned to humanitarian ends. Perhaps others are visualizing a smaller impact from “instant intelligence, just add computing power”, than I am, in which case ignoring it if it were centuries out might make sense, but I think the “we might as well ignore this” quality more often comes from evaluating Friendly AI as if it were a business idea with a 3-5 year profitability horizon. Viewing such a momentous and transformational possibility in that light is obviously inappropriate.

6 Responses »

From Physorg:

Eighty computers have been lost, stolen or gone “missing” at a major US nuclear weapons lab, the nonprofit watchdog group Project On Government Oversight (POGO) has said.

It is a major center for research related to national security, outer space, renewable energy, medicine, nanotechnology, and supercomputing.

World leaders have started to get serious about nuclear risk in recent years, but current risks from synthetic biology and all-but-certain near-future (2015+) risks from nanotechnology and AI are pretty much ignored. When the new bio or nuclear 9/11 happens, I’ll be able to look back and say that I was constantly sounding the alarm and proposing countermeasures. Will you?

Recently, in The Global Spiral, an online magazine that barely anyone reads (according to Alexa.org), transhumanists responded to recent criticism of our philosophy. This was a good issue and I liked a lot of the articles. Immediately relevant, however, is Mark Walker’s article, “Ship of Fools: Why Transhumanism is the Best Bet to Prevent the Extinction of Civilization”. Walker is a member of the Scientific Advisory Board of the Lifeboat Foundation, the only organization on the planet devoted to advancing a comprehensive set of safeguards to extinction risks. I am Fundraising Director, United States for the Lifeboat Foundation.

10 Responses »

It looks like I had a point on Friday when I was talking about the dangers of space junk. There’s a press release on PhysOrg titled, “Space crash called ‘catastrophic,’ lots of debris”.

Russian Mission Control chief Vladimir Solovyov: “debris from the collision could stay in orbit for up to 10,000 years and even tiny fragments threaten spacecraft because both travel at such a high orbiting speed.”

James Oberg, aerospace engineer: “At physical contact at orbital speeds, a hypersonic shock wave bursts outwards through the structures. It literally shreds the material into confetti and detonates any fuels.”

David Wright, Union of Concerned Scientists: “The collision had possibly generated tens of thousands of particles larger than 1 centimeter (half an inch), any of which could significantly damage or even destroy a satellite.”

20 Responses »

Note: most of the first half of this post may be made irrelevant by the Orion space laser proposal. Most of the last four paragraphs are valid, though. Still, the architects of the laser acknowledge that it would be useless for large pieces of space debris. I also added the phrase “in the near term” in the sixth paragraph.

—-

Two satellites, Iridium 33 and Kosmos-2251, slammed into each other at 12 km/sec on Tuesday, obliterating one another and creating a major junk cloud. The junk cloud will probably continue orbiting the Earth until we deploy a mote of utility fog hundreds of miles wide to clean it up. This is the first step on the road to the dreaded Kessler Syndrome, a phenomenon whereby more and more fragments of space junk are created in collisions giving rise to more collisions. The eventual result will be that it will be impossible to launch spaceships that aren’t heavily shielded, and you won’t be able to engage in extra-vehicular activity without a thick, unflexible, and well-armored power suit. We sure as hell won’t be able to build a space elevator without major risk, because a cable a couple centimeters thick will get snapped easily by space junk with that much kinetic energy, unless heavily shielded.

The Liftport FAQ gives a 1/625 chance of catastrophic failure of a space elevator given a heavy meteor shower from the Leonids, and those rocks are about 3 inches big at most. Space junk can be much larger. You can move the anchor around to dodge most pieces of junk, but at some point it gets difficult, plus, if tourists are risking their lives every time they go up there, who will want to use it? Bits of space junk going at 12 km/sec travel about 10 times faster than a bullet, which gives them not 10 times as much kinetic energy, but, you guessed it, 100 times as much. That can hurt, even if you did the smart thing and replaced all your bones and muscles with fullerenes before going up there.

Space elevators, space elevators, space elevators. Who says transhumanists are excessively optimistic about technology? Here I am, criticizing space elevators and constantly going on about the risks of nanotech, robotics, synthetic biology, and AI. Meanwhile, yesterday I read an article by some conservative Christian who is criticizing transhumanists by saying we are “embracing any and all forms of the new technologies”, with “almost no qualms about all the controversial technologies of the day”. Working for the Lifeboat Foundation, qualming is practically all I ever do all day. What else? Two weeks ago, when I published the benefits of uploading post, some people were going on about how I was only focusing on the upsides and not the downsides. But notice how the category “risks” on this blog has the second-most posts out of any tag, second only to “transhumanism”. I act positive for one minute, write a post short enough that people might actually read it, thereby necessitating my leaving out the potential downsides, and someone’s there to jump on me.

In fact, there’s been points in the past where all I do is talk about risks and possible roadblocks and people say I’m being too Apocalyptic. Then, I write something about the benefits of some possible future technology, and get people who say I’m being too Utopian. Make up your mind! If I had to choose one, I would definitely take Apocalyptic. Given the stupidity of humans and the power of our technology, I think you’d have to be uneducated not to be Apocalyptic, frankly.

Back to the space situation. The more we colonize space, the more junk we will create and the more heavily shielded every craft will need to be. So you can forget rockets. We’re already just throwing away money right now by even bothering to send people up into space on rockets, which have the terrible tendency of spontaneously exploding (what else would you expect from a bomb with a hole poked in the side?) We should be investing all our money in novel ways to get to space, like developing better manufacturing technologies to actually build spacecraft that are truly strong and light. Meanwhile, all the articles on the satellite collision are saying, “Litter in orbit - caused in part by the break-ups of old satellites - has increased to such an extent that it is now the biggest threat to a space shuttle in flight.” If it’s a threat now, when there’s only been one major satellite-satellite collision, I can’t wait to see what it will be in a few decades, when we see more of these events occurring. Of course, even if we start making spacecraft out of fullerenes, fullerene debris will be generated soon enough.

It’s hard to get around it — space junk is going to be a showstopper when it comes to colonizing orbit in the near term. That’s alright, though, because we have a lot of other colonization to do. How about colonizing the oceans? They’re empty. Or hey, what about colonizing the deserts? Barely anyone lives in them, and they’re brimming over with solar and thermal energy. How about mountains? 25% of the world’s land area is mountainous, including 67% of Asia, but barely anyone lives on the things. How about colonizing Antarctica? Way, way, way easier and cheaper than colonizing space. People don’t think of these wonderful opportunities because they grew up letting television (Star Trek) think for them.

Space has no air, warmth, water, life, pressure, or much matter to speak of. Antarctica has all these things. Before we colonize space, we should be able to colonize Antarctica easily. If we can’t colonize Antarctica, then what are we doing in space? Without molecular manufacturing, it will lead to nothing but tears and broken dreams. Even with it, colonizing Antarctica would be much more exciting. Of course, there’s the Moon, but the Moon is freezing, geologically boring, and there’s nothing there that isn’t already here. Plus, there’s the danger of solar storms, which can kill anyone in the open in mere minutes. That will ruin your day.

The excitement of space will end when people go there and get over the novelty of eating M&Ms in weightlessness and the ability to see the Earth. People will realize the shocking fact that there’s nothing there. Hence the term space, as in empty space. What matter does exist up there will be constantly threatening to punch a hole right through your body, like a rail gun, moving at 12 km/sec. If you think the Earth is crowded now, try living in a space station. Until we gain the ability to create huge (miles wide or larger) air bubbles in space enclosed by rapidly self-healing transparent membranes, it will be cramped and overwhelmingly boring. You’ll spend even more time on the Internet up there than down here, and your connection will be slow.

Why life extensionists would want to go up in space without advanced molecular nanotech (MNT) to protect themselves is beyond me. If you take a look at the history of space exploration, it often consists of people dying in unpleasant and unexpected ways. Death by depressurization. Death during launch. Death during reentry. If the Apollo 13 crew didn’t stir their oxygen tank on the way to the Moon, earlier than they planned, then they would have stirred it while one astronaut was in lunar orbit and the other two were on the surface. The astronaut in lunar orbit would have died due to freezing to death, meanwhile the two on the Moon’s surface would be stuck there until their oxygen ran out. Imagine the impact that would have had on getting the new generation excited about space travel. How many more have to die before we realize that sending people into space without MNT is stupid?

24 Responses »