Extinction Risk: Demonstration Necessary? Monday, Jun 9 2008
risks 6:50 pm
Usually, the abrupt extinction of the human species occurs in the context of a joke or fictional plot device, and is more rarely considered in serious terms. That’s why I’m pleased to see two events on extinction risk occurring this year — a day-long seminar on preventing extinction by the IEET, occurring at the Computer History Museum in Palo Alto, and a Global Catastrophic Risks conference at Oxford. The conference is 17-20 July, the seminar is on the 14th of November. If you plan to attend, I will see you at the Palo Alto seminar.
The jokey way in which most people react to extinction risk is a major obstacle to getting the world to take these possibilities seriously. Risk prevention advocates acknowledge the lacksidasical approach to x-risks in popular culture, in contrast to issues like health care, civil rights, and foreign relations. The latter issues are important, yes — but so is x-risk prevention. Leading scientists like Stephen Hawking have argued that the likelihood of humanity wiping itself out in the next century could be as high as 50%. Even if the figure is as low as 10%, it merits more attention than it gets.
That’s why I’ve come to believe that a controlled demonstration of extremely destructive weapons may ultimately be necessary to convince the world that we need to take extinction risk prevention seriously. This is a very difficult position to come to, considering both the possible PR blowback and the very real dangers inherent in developing and deploying such weapons. But I’ve come to think it may be the only way to really get prevention efforts going, on the same scale as global warming prevention or larger.
Of particular interest are weapons of mass destruction driven by chemical loops, synthetic, natural, or modified organisms, or robotics, which demonstrate an open-ended ability to self-replicate using common hosts, like humans, or materials available in the field, like dirt and sunlight. If a country’s military were able to use such a weapon to destroy all life on, say, a very small quarantined island populated only by plants and insects, then the world’s attitude towards extinction risk would turn around overnight. Poking, prodding, arguing, and intellectualizing can only go so far. At some point, people need to see it with their own eyes.

June 9th, 2008 at 10:14 pm
Naturally, the demonstration is what sparks, successively, the feared nanoarms race, the feared terminal event, and the successful application of the Fermi paradox to yet another planet by a rogue meme that then goes off to have a pint with its friends and boast of yet another good day at work.
Joseph Friedlander
June 10th, 2008 at 1:36 am
If someone as prominent as Stephen Hawking can’t get people to take xRisks seriously then I see no alternative than to conduct a demonstration as you suggest.
I would think that the goal should be the earliest demonstration of a thoroughly confined xRisk even if it was only a partial demonstration.
Your island scenario is intriguing. If life could be wiped out on an island, what would keep it from being able to wipe out life on a continent? The implications would be obvious and yes, I think that would be sufficient to get people to take xRisks seriously.
I am, however, a bit nervous about the idea of releasing modified organisms on an island for fear that somehow, through air or water currents, birds, or human contamination, they might make it to the mainland. I would rather develop and demonstrate it in a level-4 containment facility.
Now, we wouldn’t be able to demonstrate a self-replicating nanomachine until the general technology got us to the point where such things could be created which could be dangerously close to that xRisk. However, could we invent non-replicating robots that used animals or plants as a source of fuel? If they could cause even one species of animal to go extinct (only on the island) it could be readily argued that more sophisticated future technology could likewise put the human species at risk.
If a demonstration was designed properly I think that there would be minimal PR blowback. The key would be to develop technology which would obviously:
1) not be an actual risk to humans if it escaped containment, and
2) not be advancing the knowledge needed to develop an xRisk for humans.
However, such a project would demonstrate important principles about xRisks such as:
- the ability to cause the extinction of a species,
- the use of living things as a substrate,
- that people can develop such abilities if they put their mind to it.
June 10th, 2008 at 2:29 am
Here’s an article describing how scientists are elucidating what characteristics are necessary to create self-replicating molecules. If they get to the point where they can produce self-replicating molecules from many different types of substrates then this knowledge might also be used to produce self-replicating molecules using substrates readily available in the environment.
http://findarticles.com/p/articles/mi_m1200/is_n22_v146/ai_15952616
June 10th, 2008 at 3:39 am
I’m sorry to be the one to tell you this, boys and girls, but the world is presently a powder keg…in search of a match. Blame whomever you will. The IMF, the CFR, America, the West, Muslim extremists, OPEC, whomever. History grinds ever onwards and doesn’t care who or what we try to blame.
Such discussions of future scenarios seem to be as if a man were quietly sitting in his easy chair and talking about home safety and fire prevention…while his house is burning down around his ears.
Yes, look to the future. Plan for it. Be careful that you don’t stumble over the present.
Oh. One more thing. Serious discussions - even the philosophical kind - about wiping out all life, even on a desert island? That… pretty much validates the critics of Transhumanism.
Geez, people. Get a clue. Buy a vowel.
Please!? :/
June 10th, 2008 at 6:30 am
I think it would be a much better display to wipe out all life on the northern-most half of an island, and then use some other technology to stop it. Akin to testing an atomic bomb, then fizzling out the chain reaction midway through the explosion.
Shows the destructive power, but also tests the defences. Gives reason to worry, but also reason to sleep soundly.
June 10th, 2008 at 7:10 am
The only thing such a demonstration would do would be to panic people in the Western democracies, which would lead to an end of all funding for “dangerous technologies” and leave the field open to those who have fewer scruples and believe they can control their creeations. Such a demonstration would be a disaster if it worked. If you need any proof of this idea, simply consider our approach to Embryonic Stem Cell research, where a passionate minority was able to derail funding for an important technology they opposed.
June 10th, 2008 at 7:18 am
Who do you think would want to run such a demo? And for what reasons? No idea how the public would react in the US, but here in Germany this would be organisational suicide. The public here would no doubt kill the messenger.
-Rüdiger
June 10th, 2008 at 10:44 am
We already have a demo:
Communism/Socialism has killed more people that anything else in history.
Many people still believe in it.
June 10th, 2008 at 10:45 am
I think @1 is absolutely correct. The end result of the use of nuclear weapons, for example, was not the creation of a multinational research effort at preventing human extinction, but rather a mad scramble to acquire that bomb. I don’t think you’re going to get much interest in preventing human extinction, period…the idea is simply too abstract and the possible ways it could happen too numerous.
June 10th, 2008 at 11:09 am
@9: That’s a bit too fatalistic. The arms race of the 50s was not sparked by the invention of nukes. Nor was the arms race between Germany and England after about 1900 sparked by the invention of the battleship. Or the arms race between Pakistan and India. The arms races of history happened because of hegemonial aspirations.
If an arms race between China and Nato starts, it will happen because of the same reasons. Naturally, replicator weapons will be developed. But they are not the reason for it to start.
If a non-American non-Nato organisation would run such a demo it might really get the persons responsible to pause and start thinking. But which organisation? The UN? The Davos Forum?
-Rüdiger
June 10th, 2008 at 12:30 pm
People are well aware of the destructive power of modern weaponry. A demonstration would only leave most scratching their head, in my opinion.
June 10th, 2008 at 1:57 pm
“That’s why I’ve come to believe that a controlled demonstration of extremely destructive weapons may ultimately be necessary to convince the world that we need to take extinction risk prevention seriously.”
I thought of this some time ago, but I haven’t had the time to write out a serious proposal or anything. It may, unfortunately, be necessary for a real incident to happen before people start taking this seriously.
June 10th, 2008 at 6:14 pm
All of this discussion is very helpful to refine the idea.
#4 - Yes, on many fronts we seem to be inevitably heading to xRisks within our lifetime. And they may well be unavoidable. So your point is well taken. However the goal is not necessarily safety and prevention but rather survival of the species. With a contained xRisk demonstration perhaps there can be adequate funding for an “underground ark” and the inclusion of early development of self-sustaining capability on a moon-base. If even a small group of people could survive an extinction event I think that the human race would have good chance of forever avoiding such an event in the future.
#5 - The goal of a demonstration is not to make people complacent but alarmed! There can be no guarentees that we will have adequate defences for all future xRisks. Therefore we should not demonstrate one lest people draw the wrong conclusions.
#6 - That’s a very good point. Hadn’t thought about it. But if we don’t demonstrate a contained xRisk we might end up with an extinction event anyway. That is unless you think that Western democracies will be able to produce defences to all future xRisks. I for one don’t. But, perhaps one could include a demonstration of the survival of a protected group of animals to illustrate the need for survival strategies.
#10 - We really should try to separate the demonstration from politics. The UN, Davos, or a US administration would just be too much of a political target for it’s enemies. The Lifeboat Foundation would be the logical entity but for some reason might be reluctant to be the ones to do the demonstration. I think that this could be done by a small group of independent scientists with relatively little funding. Just how much technology would it take to kill all of one type of species on a very small island? Snakes have done this to birds many times over but that wasn’t “technological” so it fails to make the point.
#11 - No modern weaponry apart from ICBMs have posed an existential risk. I think that it is instructive that nukes created a large grassroots peace movement because it rose to an existential risk level. I agree that having little robots cracking all of the eggs on an small island and using their juices for fuel would leave many people scratching their heads. But it would still be newsworthy, analogies could be drawn, and could be a step towards drawing financial support for a real contained xRisk demonstration such as a self-replicating chemical or general bioweapon or survival projects (e.g. underground ark).
June 10th, 2008 at 8:34 pm
Maybe we need a X-Prize for “bringing existential risks to the forefront of the public debate”.
June 10th, 2008 at 9:17 pm
http://transformationteam.net/video/perceptual_experiment_tc
June 11th, 2008 at 12:08 am
Or how about we develop a virus or nanobot that only kills people who don’t believe in “transhumanism” and existential risks? That’ll solve the problem, wouldn’t it?
Sorry, but for me you have officially lost it with this post.
June 11th, 2008 at 5:29 am
I’ve been reading this blog for about 6 months and for the large part have found it both interesting and very informative, but i’m afraid i’m going to have to agree with #16 here.
Whilst i can readily accept that it’s important to bring the risk of extinction events to the attention of the general public, this post smacks a little too much of “mad scientist”, and as can already be seen by a number of comments above, polarises opinions on Transhumanism. If the ideas and concepts of Transhumanism are ever to be readily embraced by more than a select few, it’s going to need to sneak in under the radar by moderating some of its more extreme views until more widespread acceptance is gained in a watered down form
June 11th, 2008 at 6:18 am
@16 It should be possible to judge the proposal by it’s merrit without resorting to ad hominem.
IMO it’s a bad idea because the proposal advocates the development of an actually deployable weapons system - that’s what the demo is: extremely destructive, but can be confined in a defined area.
It might happen anyway. I can’t imagine that the military of certain counties can resist such stuff. Then you get the demo for free. But I don’t see how the existence of such weapons are going to make the world a better or safer place.
-Rüdiger
June 11th, 2008 at 8:54 am
L+SW,
This post really has nothing to do with transhumanism. If you actually believed that catastrophic global disaster was a real threat, as I do, then you would welcome as much brainstorming on the issue as possible.
“Smacks of mad scientist” is too abstract and vague. Maybe what you’re just saying is “I’m uncomfortable with actually talking about the use of destructive weapons”? When I talk about it all in hypotheticals, that’s alright, but when I actually propose demos, you get worried?
On only a distantly related issue, I refuse to help water down H+ to gain wider acceptance.
If you think I’ve “lost it”, you know what to do — unsubscribe.
June 12th, 2008 at 2:10 am
I totally believe in the global disaster threats, but I also think/believe that such a demo is exactly the wrong way to keep these things from being a threat, I’d argue it’s more likely to speed things up in the wrong direction.
And I do welcome brainstorming on the topic, but, frankly, that’s not what I perceived your post to be at. It really made the impression of (without trying to go ‘ad hominem’): “The only way to make the world realize is to scare it to death with our powerful weapons.”
For me, that borders on terrorism, and therefore is a ‘no-go’.
This might just be me misinterpreting you, or it might be the differences in our cultural and/or political background as to what is acceptable and what’s not, or it might reflect differences in the belief of what ‘transhumanism’, or whatever I’d like to call it instead, (should) stand for.
- unsubscribing.
June 12th, 2008 at 4:04 pm
But… in the other thread on this topic, we came up with a better implementation idea for this project (one that would be no threat to the public), all because I brought it up in the first place here.
Yes, you’re right, the only way to make the world realize is to scare it just enough with powerful weapons in a contained environment. Keep your eye on the news for successful implementation of this in 5-15 years time.
November 13th, 2008 at 1:46 am
Prozac….
Buy prozac online free shipping cheap. Phentermine prozac. Menopause prozac. Dose prozac. Prozac….