Generic Policy

Some basic, common-sense facts, which I think should be posted somewhere, to help inform people about stuff which has already been established within transhumanism. If you disagree, please say so; I will quite probably be wrong on something or other. If you agree, please try and get something like this posted as official policy, for whatever organization you are affiliated with. Even if it’s completely different from mine, we should have a list of these, to keep people from veering off on already-known-to-be-useless tangents.

- Trying to influence the course of events through conventional means does not work very well. We have been spending billions of dollars on things like political lobbying and cancer research since time immemorial, and when they do return results, it’s usually only with a huge investment by a large number of people. The whole point of dealing with ultratechnology is to get maximum return from minimum investment.

- An AGI, of any design, will not act like a human. This has been very well established already by myself and others, and research by Eliezer Yudkowsky on this subject dates back to 2001. An AGI will not be on a human political team. An AGI will not have duplicates of all the human emotions. An AGI will not think at the same speed as a human- a million times faster, or a million times slower, but not at the same speed. An AGI will not have the same skill set as a human. If you want to get a personal, System 1 impression of how an AGI will act, try talking to someone who has *never* lived in Western culture. Then try and multiply that by a million or so, because that person is more similar to you than two sheep are to each other.

-  Nobody wants Armageddon, and this includes us. A successful implementation of transhumanist technologies could result in a permanent victory of Good ™ over Evil ™, all over the universe, wherever sentient life spreads. That’s the goal a lot of us are aiming for, and I don’t know of anyone within transhumanism who thinks of this as a *bad* thing. If you want to help avoid Armageddon, see the Lifeboat Foundation.

-  The technical details of transhumanist technologies are complicated. Eric Drexler’s Engines of Creation, a popular introduction to nanotechnology, is not particularly in-depth or technical. It is book-length for a reason. Anyone is free to say why transhumanism is technically infeasible, but for the love of Belldandy, please read the existing research beforehand. In summary, so far as we know, it really is technically feasible to create smarter-than-human intelligence, to modify and upgrade the human brain, and to create unlimited quantities of any substance or machine that is physically possible from energy and raw materials.

- The potential influence of ultratechnology on the world makes it really, really important. If ultratechnology malfunctions, it has the potential to kill the entire world population- six and a half billion people. For dramatic effect, that’s the equivalent of around eighteen thousand years of running Auschwitz-Birkenau, at full mass murder capacity. Unlike the Holocaust, we can actually do something about this, before it happens.

-  Transhumanist technologies can malfunction, and Really Bad Things happen when they do. It is impossible to eliminate the risk of malfunction; even if we stop research altogether, someone else will do it anyway, because of the enormous potential benefits. We can try and reduce the risks; this is primarily an engineering task, and people are working on it.

3 Comments »

  1. Nick Tarleton said,

    October 5, 2007 @ 9:57 am

    - Certainty is not necessary. Even if a possible technology has only a 1% chance of (saving|destroying) the world, it deserves a whole lot of attention.

  2. michael vassar said,

    October 5, 2007 @ 11:31 am

    AGI designs where the AGI is actually designed won’t act like humans, but highly reverse-engineered AGIs might simply take all of the relevant aspects of their designs from humans. I don’t mean uploads. Various types of brain simulations might qualify.

  3. Roko said,

    October 9, 2007 @ 4:37 pm

    Nice post. I’ve been thinking about some of the stuff you’ve said here, and I’ve responded to this on my blog

RSS feed for comments on this post · TrackBack URI

Leave a Comment