The Relationship Between AI and MNT
The start of a nano arms race, and the lack of willingness to regulate potential synthetic biology through the modification of existing treaties or the application of existing treaties or the development of new regulations is short sighted.
Nano or synthetic biology weapons will diffuse into hands other than the inventor and first user, and it is easier to reverse engineer nano or synthetic biology military products than to make a nuclear weapon. Once they exist they can be copied, and diffusion of the resulting products will make local and global security nearly impossible. Security would come with a hefty price tag — not just in financial terms, but in changes to societal interactions.
Yes, and we have to be willing to make these sacrifices. Not sure about the nuclear weapon thing. I am almost done with a paper that goes over which nanoproducts I should think be regulated or outright banned. Basically, out of all the products that will become possible almost overnight with molecular nanotechnology (MNT), my recommendation is that only a tiny portion should be permitted. Products below a certain size, above a certain size, above a certain energy consumption level (2 MW/person should be appropriate), etc etc etc would all have to go. This shouldn't be a problem, as even the remaining products would offer orders of magnitude better performance than the products we are used to. However...
The unfortunate fact of the matter is that a long-term, human-only nanotech society is not viable. We are not pacificistic enough, or restrained enough, or wise enough not to do horrible things (mostly by accident, though deliberate misuse is also a problem) with a technology that lets you build products of the quality, quantity, and sheer size that MNT will permit. It doesn't really matter if the technology arrives in 2015 or 2035 (though obviously I think the former is more likely) - it's suicidal within a matter of years either way. A central reason is that a billion times present-day computing capacity is guaranteed to make human-level, autonomous AI much, much easier than it is today. This will lead to MNT-equipped AI, which offers such an entity "optimization power" far in excess of multiple human races. Optimization power will be thrown behind a utility function, or supergoal, or whatever you choose to call it, which will be initially specified in terms of computer code by human programmers. The problem is that most utility functions we can currently imagine lack the stability, complexity, precision, and moral sophistication to permit the existence of information theoretically-unique, astonishingly specific matter-structures called "human beings pursuing their own goals". We are used to inhabiting a world where each agent has roughly the same quantity of optimization power, and nearly the same goals (though selfish goals cause divergent interests, the information content of the utility function itself is the same thing), leading to equilibria that involve the majority surviving and reproducing indefinitely. Not for very much longer. MNT alone introduces power differentials of heretofore unseen proportions, far greater than those introduced by mere nuclear-tipped ICBMs. AI would produce power differentials literally larger than we can imagine.
Back to nanotech, a committee of the NATO Parliamentary Assembly says:
Additional lengthy and scrupulous studies of military NT are urgently needed. The prospects of molecular NT should be assessed with particular attention, as this is the most controversial aspect of NT and would present extremely grave consequences if its feasibility is confirmed.
Grave indeed. Several independent groups have confirmed the feasibility of molecular manufacturing. Although MNT is a favorite speculative technology of many techies, few understand the details, and thus lack the means to determine the difficulty or developmental timeframes except based on intuitive guesses based on projected consequences in popular articles, "inspiration" from video games and novels, or hearsay. For example, if a techie hears that MNT is likely to lead to immortality, they will put it hundreds of years in the future. If they hear that MNT will do little more than make clean water available to 3rd world countries, and allow immersive VR goggles, then they might put its development in 2030. This "forecasting based on consequences" is often concealed using subtle language, but easily destroyed by knowledgable persons asking prodding questions, like "are you familar with concept of a Stewart platform, prior work on mechanosynthetic tooltips, the evolution of the scientific community's opinion of MNT over the past ten years?", etc.
In the last few years, I've realized that a necessary prerequisite of grasping the consequences of AI is understanding MNT in a non-trivial way. This is likely because those unfamiliar with MNT find it impossible to imagine how an AI would get "out of the box", even with human help. Because true MNT understanding is still extremely hard to come by, the foundation for AI understanding is simply not there. However, I am hopeful for improvements in MNT understanding, because the technology, when discussed in conservative and straightforward terms, is not radically difficult to grasp, however, there are such deep biases in human modeling of nonhuman minds that explaining advanced AI issues can be near-impossible.
Regards to Chris Peterson for the links.
February 27th, 2007 - 13:26
Indeed, we need more precautions on this stuff than most people would think.
““Are you familar with…”
No, I’m not- where can I find out about this stuff?
February 27th, 2007 - 13:35
I suggest searching for answers that don’t involve bans.
Drugs and other banned items are in plentiful supply, so it follows logically that one shouldn’t depend on a ban. In fact bans may artifically increase the supply; banned items tend to be more expensive than they would otherwise be, thus encouraging more people to try and supply the market.
February 27th, 2007 - 14:44
Here are some of the main sources:
Freitas and Merkle have some of the best work with tooltips, stewart platforms
http://www.molecularassembler.com/
http://www.molecularassembler.com/KSRM.htm
http://www.molecularassembler.com/Nanofactory/
http://www.merkle.com/merkleDir/papers.html
http://www.crnano.org
http://ideasfactory.wordpress.com/
http://www.zyvex.com
http://www.e-drexler.com
http://www.foresight.org/cms/resources/51
(has the crnano wiki)
http://www.foresight.org/Conferences/Conferences.html
my own site
http://advancednano.blogspot.com
February 27th, 2007 - 16:34
Precisely (part of, at least) the argument against most victimless *malum prohibita* (as distinguished, of course, from *malum in se*, inherently wrong/immoral acts or activities). Prohibition, whether of drugs or whatnot, merely instantiates a black-market with higher prices, more risk (since there is, ex hypothesi, no legal recourse or due process), and, usually, more violence…
And, yeah, Michael (A.): While I try to be a well-versed on both AI and nanotech as I can, I agree with Tom: Can you provide some citations and links for the “Are you familiar with…” stuff…?
February 27th, 2007 - 19:29
Anissimov:
Please excuse me if I meander off-topic.
As August explained in his response, banning something artificially limits supply and increases price, and virtually guarantees a black market will arise. Even if strict government controls are enforced, it is almost certain that someone will circumvent those safeguards for profit or other motivations.
How much money would it take to get the average government worker to “ignore” someone walking out of a lab with, say, some military-grade replicators? Or to get an inspector to “overlook” illegal tooltips? Or to get a government contractor from Zyvex to “accidentally” e-mail the specs for a functional MEMS or NEMS? I’m certain every government, para-military group and corporation would gladly offer several billion dollars to get their hands on such technology.
Secondly, let’s talk about the actual structure of a ban on certain nanoproducts. What if, for example, the United States bans certain nanoproducts, but a country such as North Korea doesn’t? Does the US try to prevent them from obtaining MNT equipment? Does the US launch a military strike? Much like the nuclear genie, the nano genie can’t be put back in the bottle, and attempting to do so can only mean perpetual war or reverting to the Stone Age. Even then, the meme will live.
I am incredibly wary of trusting anyone other than a select few with regulating MNT or AI (remember, vote Michael Haislip for Federal Director of Transhuman Technologies). Hell, most people and politicians consider global warming to be the main threat against humanity. Global warming, for Christ’s sake! Global warming is like the third cousin of existential threats. As you said in the post, “AI would produce power differentials literally larger than we can imagine.” You are absolutely correct. Try to imagine an ant crawling along the ground. Then imagine a giant boot hovering over the poor ant. With AI, you are the ant.
February 27th, 2007 - 21:21
Michael,
Couldn’t nanotech, by virtue of allowing extremely precise and accurate engineering, allow weapons that become much more deadly and dangerous because of their efficiency, without even requiring more power than current weapons?
February 27th, 2007 - 22:40
August and Michael, numerous bans are enforced on products, chemicals, and weapons (a topic for a future post), and many of them work great. The reason why you guys aren’t even familiar with them is that the bans work so well! Illegal drugs are very popular because they push certain buttons in the brain. Most of the products I am talking about are merely weapons, and there wouldn’t be the same type of urge to obtain them that we see with drugs. For example, one might think a radiological bomb would be heavily desired by terrorist groups, but despite the fact that they are downright EASY to create, no group, anywhere, has yet detonated one, as far as we know!
Regulating nanotech will be slightly easier than certain other technologies – the prerequisite tech is incredibly complicated, and would be extremely difficult to reverse-engineer in the long run. Notice how nuclear weapons have been around for 60 years but only a handful of government scientists have actually succeeded in making them.
Regulations would need to be global. Yes, if NK didn’t comply with a ban, they’d need to have the carpet yanked out from under them. Today’s doctrine is already setting the stage for this.
Chris, yes, millibots made using less than a gram of diamond and programmed to inject botulism toxin into the vein of a target would be an example of a product I would like to see banned. Perhaps 2 megawatts is way too large of an upper energy limit. However, as a reference, “supercars” drain about 1/2 MW, and yachts substantially more. Even with MNT, to build a yacht that can carry you and all your crap would require a significant power drain. There should be different energy limits for different products, and a personal energy limit for the sum total of all products you have deployed at any given time.
And Michael, about trust… would you trust me? There’s a really funny thing that happens in discussions about regulation – you talk about regulation early, and you influence everyone and frame the discussion for years to come. For example, The Bulletin’s primary announcement recently included a misunderstanding in the MNT section that was brought on by misreading CRN’s site. So, my hope is that by talking about MNT regulation pretty much non-stop until the technology emerges, I might actually, maybe inspire someone with “real power” in the government to do the right thing.
February 28th, 2007 - 01:26
I’m looking forward to seeing that list.
I think it’s important in constructing any ban such that the punishments fit the crime. Small violations should have small deterrents.
This is critical when it comes to the deployment of any Trusted Manufacturing/DRM systems (which I still don’t recommend at all, btw – better to limit the base design of released products than to attempt to limit their capacities after the fact). If you’re going to design systems that protect low-consequence things like patent rights, along with high-consequence things, like explosives, it’s a good idea to use //separate// designs, so that hackers who publish research that cracks low-consequence systems don’t get over-labeled as TERRORISTS (from a civil liberties perspective), but also, more critically, so that their cracks research is not applicable to the systems that protect against explosives production.
February 28th, 2007 - 02:18
For links to further information, see Brian’s comments. The key names to follow are Drexler, Merkle, Phoenix, and Freitas.
February 28th, 2007 - 16:57
“For example, one might think a radiological bomb would be heavily desired by terrorist groups, but despite the fact that they are downright EASY to create, no group, anywhere, has yet detonated one, as far as we know!”
I think this is a different effect than the creation of a black market under a government ban. I COULD, if I wanted to, synthesize up a couple kilos of high explosive and blow myself up. The necessary knowledge and chemicals are not banned or even regulated. And yet when was the last time a bomb went off at the local store?
“and many of them work great.”
Some examples, please? Even something as seemingly innocent and effective as the ban on Freon has caused a black market to develop (http://outside.away.com/outside/magazine/0897/9708freon.html)
“substantially more.”
A standard Learjet uses a good deal more than 2 MW. In fact, the Learjet requires more raw power than a CO2 laser that can blast the Learjet (and an F-16) out of the sky.
“Most of the products I am talking about are merely weapons, and there wouldn’t be the same type of urge to obtain them that we see with drugs.”
Most world militaries would be willing to pay large sums of money for these weapons.
February 28th, 2007 - 18:29
QUOTE I am almost done with a paper that goes over which nanoproducts I should think be regulated or outright banned.ENDQUOTE
I’m thinking of Josh Wolf (use the google). He’s sitting in a prison cell right now because he realizes that he’s the bottleneck. If he talks, the genie comes out of the lamp.
Michael, where are the articles about how the responsibility for this technology depends on the people who develop it?
If, by some incredibly unlikely series of events, I manage to develop an AI, I’m going to be sure that I do not then race to hand it over to a government or corporation. Why? I might not trust myself, but I trust a government or corporation even less.
The community needs to be strongly developing the ethical mindset. These tools aren’t for wiping out “Americuh’s” enemies or forcing anyone to do anything. These tools aren’t for wiping out civilization; they’re for making everything easier for everyone. Why does it always have to be weapons and death and mayhem?
March 5th, 2007 - 20:14
I think some people are a bit lacking imagination here.
Regulation won’t work. To smuggle a nanofactory out of a ‘secure area’, you only need to instruct the factory to produce a tiny ‘seed’ which can later reproduce itself until you have a complete new nanofactory. Such a seed can be made nearly arbitrarily small and is thus impossible to detect. And I don’t think that there is a way to prevent proliferation once only a single nanofactory leaked out. The possible duplication rate is simply to big.
But OTOH with MNT people can use this to protect themselfs: Killer-nanobots can be countered with a nano-immune-system. Nano-surveilence or Small-smartbombs with a utility-dust based personal defense system. And even if you get hit by a weapon your body is enhanced with lots of MNT-based subsystems which could limit the damage. Also uploading will become possible quite quickly and to kill a person you then also have to destroy all backups. So in the end despite all the improvements in weaponry personal defense and human durability will improve a lot, too. So the arms race will simply continue.
And while I think that AI becomes quite probable with MNT, if MNT is developed relatively early then humans can start to upgrade themselfs before the first AI reaches human level. This would level the playing field a bit and would make the construction of friendly AIs more probable.
March 5th, 2007 - 20:46
I’m thinking of Josh Wolf (use the google). He’s sitting in a prison cell right now because he realizes that he’s the bottleneck. If he talks, the genie comes out of the lamp.
Hey, thanks for comparing me to a nutcase. I appreciate it, really.
Michael, where are the articles about how the responsibility for this technology depends on the people who develop it?
Use “the Google”.
These tools aren’t for wiping out civilization; they’re for making everything easier for everyone. Why does it always have to be weapons and death and mayhem?
Even if 1/1000 people want to make mischief, it makes total sense to devote regulations to making sure that tiny percentage can’t cause catastrophic trouble. It doesn’t matter if you would prefer people not to use MNT for weapons. They will ignore you and go right ahead anyway. I pay attention to the risks because not enough people do, and because enjoying the benefits is contingent upon us eliminating the largest risks (which are numerous). Your “full speed ahead and everything should turn out okay” attitude is endemic to transhumanist community, and I’m tired of it.