Ideas for Mitigating Extinction Risk Tuesday, Sep 23 2008
risks 5:14 pm
As I see it, there are three main categories of risk: bio, nano, and AI/robotics. These man-made risks make up the vast majority of the threat magnitude over the coming century and deserve most of the attention.
Threats of low probability include asteroid strikes, supervolcano eruptions, alien invasions, simulation getting shut down, and many others. Though there is disagreement on whether nuclear war, particle accelerator disasters, or runaway climate change deserve to be counted as substantial-probability extinction threats over the coming century, I would say they are not.
A word on focusing on low probability threats alongside higher probability threats. Mentioning low probability threats just for the sake of comprehensiveness is rhetorically damaging. It distracts from the central thrust by introducing superfluous information. Worse, it can damage credibility of the entire message. Whether fair or unfair, we have seen the doom-worriers of the Large Hadron Collider heavily maligned by both scientists and laypeople in print and online. Even if an x-risk mitigator thought there was some probability of planetary doom due to the LHC, say one in a hundred thousand, the credibility sacrifice of pushing the issue is bound to detract from one’s ability to advocate mitigation of other, much higher-probability threats. So it should be avoided. Of course, if the LHC occupies a dominant portion of the risk pie in one’s personal estimate, it would be rational to devote attention to that, despite the credibility penalty.
Regarding natural vs. artificial threats, there is a credible argument that all natural threats are of substantially low probability. We’re still here. Homonids have been around for at least two million years, despite radically inferior numbers and technology for 99.9% of that time. If our ancestors could survive natural disasters, then we’ll be able to also, with our far superior technology and numbers. Asteroids capable of causing major extinctions only strike the Earth about once every hundred million years or less. In the 600 million or so years that there has been complex multicellular life, there have only been six major extinctions, if you include the present one that humans are causing.
For the central risks that I mentioned, which can also be abbreviated GNR (genetics, nanotechnology, robotics) or GRAIN (genetics, robotics, AI, nanotechnology), I recommend the three S’s: science, standards, and security. Scientific investigation of the risks provides a sound basis for further policy. This takes actual money and work, and won’t occur automatically. Taxpayers should foot the bill. Free market incentives for self-regulation are not enough. Industries have an incentive to downplay the magnitude of risk for short-term gain. I say this as a capitalist and advocate of science and progress. (In our polarized political climate, such disclaimers are unfortunately mandatory.)
After science comes standards. All of an industry, say the nanotechnology industry, or the synthetic biology community, needs to come up with some basic set of safety rules, both for individual workers and for the effects of their industry on the planet and environment as a whole. Examples of industry standards are too numerous to list. How much government involvement should be included in the approach will vary depending on your political philosophy. Too much meddling will cripple an industry and encourage clandestine workarounds, and too little meddling may cause an industry to adopt a “no rules” policy that maximizes profits while ignoring risk. If your libertarian philosophy causes an industry to pursue dangerous practices that increase global risk, then your philosophy has failed to adapt to the dangers of the future. If your interventionist philosophy causes an industry to become frustrated and transfer their operations to another country with no rules, then you’ve failed again. Insofar as it’s possible, discard your context-insensitive political beliefs and adopt context-sensitive, non-partisan approaches to these new challenges. Only then will enough people actually agree with you that the approach is adopted and makes a difference.
After standards comes security. The standards have to actually be enforced, or they are useless. If dangerous genetically engineered microbes are not kept under lock and key, unsavory individuals may get ahold of them and use them to fulfill nefarious ends. Security measures will be bolstered by transparency and increasing surveillance and sousveillance, a natural consequence when you combine human curiosity and cheaper/smaller cameras. Local and global agencies have to cooperate as effectively as possible to ensure that standards are being enforced, both in private and public realms.
Those are my thoughts for today. In summary:
1) if you’re an academic, author, or journalist, write about extinction risks,
2) if you’re otherwise involved in science and technology, help create structures to manage global risk,
3) if you’re someone that makes a decent salary or otherwise has money or resources, consider contributing some of it to efforts to mitigate extinction risk,
4) if you’re anybody else, think carefully about the issue and get informed.
The goal is a world where the annual probability of risk is extremely low, approaching zero. Even if the annual probability is just one in a million, then after a million years we’re likely to destroy ourselves. It would be great if the human race and our descendants and variants lives for a long time, billions of years, colonizing the universe and living happy and fulfilling lives. Our thoughts and actions at this crucial juncture could make the difference.
3 Responses to “Ideas for Mitigating Extinction Risk”
Leave a Reply
You must be logged in to post a comment.
September 28th, 2008 at 9:56 pm
If your libertarian philosophy causes an industry to pursue dangerous practices that increase global risk, then your philosophy has failed to adapt to the dangers of the future.
Forgetting about those crazy anarchy-loving Libertarians (and I don’t think there are too many Libertarians of that variety); there is no true conflict between liberty in the marketplace and safety.
It all comes down to rights. I have no right to endanger my neighbors by raising poisonous snakes in my backyard and letting them wander around. I have no right to hurt or endanger other adults without their consent (within reason, of course.)
Unfortunately, a lot of people talk about rights but few people seem to understand them. I once heard a story about A person who spray-painted on another’s front door and defended it as his right to ‘free speech.’ As I mentioned before in a comment on this site, one of the most important protections in the future (in regard to dangers and risk) will come from a proper understanding of rights.
Quite simply: a company, group, or individual has no right to endanger others (again, within reason, a person can find “dangers” anywhere); and it is the role of government to legitimize and define these boundaries; just as it legitimizes and defines property rights, contracts, and a hundred other things; which allow people to live in peace without coercion.
September 28th, 2008 at 10:11 pm
Oh, and I forgot to mention: the word “Regulation” has become such a politically loaded word that it has lost most of its meaning. Three people can say the word ‘regulation’ and mean completely different things.
It is for this reason that I don’t use it much, it order to avoid confusion. The word has, among some circles, become synonymous with “socialism”; this is because many socialists throughout recent history have used the word to promote there ideals. But I don’t think any of these people complain about the ‘regulation’ of weapons-grade Uranium. And furthermore, I don’t think such ‘regulation’ is socialist.
Unfortunately, ‘regulation’ is one of those words that has been abused and man-handled through most of the twentieth-century (mostly by political types); maybe it’s time to put the poor thing to rest, and adopt a few new words that won’t create such confusion
October 2nd, 2008 at 9:39 am
The problem is that everything can be misused.
You talk about Friendliness. Some of the meanest people in the world are very friendly and are masters of getting peoples’ confidence. Friendliness is just a ruse in this case for the Psychopath to get their way.
We have already has mass dieoff events in human history due to climate change ( warm turns to cold ), war, plague (both human and plant-based), and out of control goverment. What made those possible?
How would AGI fit into human structures and how would AGIs then be misused?
A bank of AGIs who monitor and mind people is the most likely first scenario and the nation most likely to do this is China. If friendliness can be put into an AGI, it can just as easily be stripped out, or twisted to some other purpose.
Lets build surveillance systems which can then me misused. Lets build a regulatory framework which can then be misused. Lets spend taxpayer money to make advances by a mere few while our ethics and thoughts must catch up.
All of these ideas are top-down approaches that seek to converge to a goal which cannot be met without greater and greater control.
If the chance that an all-powerful minder will be misused is 5%, then in a hundred years, that minder will go rogue - which is what we have seen over the years as nations have eaten their own citizens and others.
A better approach is to set the rules and then motivate the actors to use them everyday in order to survive while identifying (for tracking and removal ) those who do not follow the rules.