In my last post, Mitchell Porter (whose mailing list postings I’ve been reading for nearly a decade), argues that my vision for a Neo-Carboniferous world is “deliberately low-key”, and he paraphrases it as, “Hey, greens! You shouldn’t fear us robot-swarm, mind-uploading, planet dismantling transhumanists! Look at this vision of a leafy green Carboniferous world that we’ve conjured up for you!” Then he remarks, “It says nothing about, say, how we meet the WMD-fabber-on-every-desktop problem without outright relinquishment, nor does it really tackle the problem of whether having as many trees and human beings as possible truly seems like a good thing, when you have truly godlike power and insight”.
Mitchell’s comment, and Sebastian Hagen’s before his, are on the mark. By transhumanist standards, the vision I present is boring and conservative. My point is to attempt to flesh out the wide range of possibilities inherent when intelligence commands self-replicating machinery. For myself, I’d appreciate a Neo-Carboniferous Earth just as easily as if it were a computer simulation on a matchbox-sized nanotechnological computer, as if it actually existed in the “real world”, but that’s just my preference. Matter is probably more usefully spent on implementing conscious beings living worthwhile lives, rather than as filler molecules of lignin doing little more than serving as an elaborate decoration for the consumption of qualiabearing beings.
As for the challenge of a WMD-fabber on every desktop, I’ve argued for the solution before, and it’s still the same — we require a benevolent singleton. A top-level decision making entity that can act with extreme speed and reliability to put down threats before they emerge. Naturally, the unimaginative might directly associate such an agency with communist dictators, but what I am talking about is as different from a communist dictator as an animal is from a prion. With immense power must come immense responsibility and intelligence, far more than can be held in the three pound lump of flesh we call the human brain. The inherent hazard of stating this is exposing myself to ridicule by those who immediately associate recognition of human limitations as misanthropy, which it is not. Such logic is like saying that those who criticize elements of America’s foreign policy are innately anti-American.
What is remarkable are those that seem to argue, like Ray Kurzweil, the Foresight Institute, and the Center for Responsible Nanotechnology, that humanity is inherently capable of managing universal self-replicating constructors without a near-certain likelihood of disaster. Currently Mumbai is under attack by unidentified terrorists — they are sacrificing their lives to kill, what, 125 people? I can envision a scenario in 2020 or 2025 that is far more destructive and results in the deaths of not hundreds, but millions or even billions of people. There are toxins with an LD50 of one nanogram per kilogram of body weight. A casualty count exceeding World War II could theoretically be achieved with just a single kilogram of toxin and several tonnes of delivery mechanisms. We know that complex robotics can exist on the microscopic scale — microwhip scorpions, parasitic wasps, fairyflies and the like — merely copying these designs without any intelligent thought will become possible when we can scan and construct on the atomic level. Enclosing every human being in an active membrane may be the only imaginable solution to this challenge. Offense will be easier than defense, as offense needs only to succeed once, even after a million failures.
Instead of just saying, “we’re screwed”, the clear course of action seems to be to contribute to the construction of a benevolent singleton. Given current resources, this should be possible in a few decades or less. Those who think that things will fall into place with the current political and economic order are simply fooling themselves, and putting their lives at risk.
If I recall correctly, Kurzweil considers “humans” of the future to be essentially developed or in the process of developing into the very entities you seem to be describing – superintelligent entities who are, moreover, very unlikely to remain susceptible to biological or biologically targeted toxins. Which seems like it would be a far more attractive scenario – instead of a single superhuman entity, however benevolent, why not have a whole planet of them?
Offloading human decision making and authority to machine intelligences is not something that’s going to happen overnight — it’ll be part of a broader trend (save for hard takeoff type scenarios).
Take for example those alert Pentagon folks who suggest that human decision making on the battlefield will end by 2020. Handing over other types of authority to superintelligences may be a logical next step and an iterative process; what seems absurd and unlikely today may prove different when we face the threat of extinction.
I don’t find the “desktop WMD-fabber” question at all interesting. By the time we can do that we can also monitor minds and stop acts initiating force in the bud. In addition the assumption that just one deranged mind will have the power to undermine all others to the point of their destruction is mostly a thought experiment that may not match future reality. In any case I do not believe that the ultimate benevolent dictator solution is viable or particularly livable. While the benevolent AGI is certainly not afflicted with human characteristics and arguably “could” track everything to fine enough detail in real time, this has some severe drawbacks. The main one is that it requires no other similarly powered entities to exist. If they did exist then being of equal or greater power they could not be fully monitored or controlled by the initial entity. Thus the system requires significant limitations of the further development of intelligence and all entities. That is relative stasis, an unacceptable price for a bit of safety.
One of the places that humanity has some growing up to do is getting over the fearful monkey syndrome. I see far more intelligence, knowledge and imagination among transhumanists being applied to what-if scenarios of outrageous terrorism than being applied to the actual creation of a transhumanist future. We got as warped by 9-11 as the population at large. It is not healthy. There is no absolutely guaranteed avoidance of all terrorism or all conflict forever and ever. Nor does there need to be.
The damnedest thing is that despite these delivery methods, the toxicity of the agents and the sheer psychoticity of the crime, I can easily see a huge percentage of people look forward to expressing their bad hair days using these means.
It is so easy to regard terrorists as beyond reasonable discussion, right?
Worse still – anyone who has seen the movie “schindler’s list” ? Remember that small girl in the red dress? The one that showed up in a landfill as a decayed cadaver? Imagine you being her father and having access to these devices, and it is 1944. If I were in the same conditions I would not hesitate a second to release these compounds over berlin.
That would be terrorism, right? However it is also a normal feature of the human psyche – vengeance.
What is all to casually forgotten is that right now, in this world, december 2008, there are millions of people who are in precisely those same position as such a hypothetical parent. These millions do have compelling arguments to use precisely the methods described above (and of possible, far worse ones) against the United States, Europe, Japan, China, Russia. Right now you are already on the genocide list of millions of people who would if they could.
You may call them monsters, you may be incapable of empathizing. I know I am and I would understand why if it happened. I prefer it doesn’t, with anyone, but we are in a world with so much exploitation, callous disregard and hatred it seems we can no longer afford the same extremes of affluence and poverty, the same exclusion of billions from sane medical care. food, shelter, creature comforts, clothes, entertainment, meaning.
We didn’t act decades ago and implement a global birth control program so we are stuck with these people, and no amount of ostracizing or intimidating them will save us from the above scenario, except make damn sure they have as much to lose as possible so they don’t.
Or kill em now, as expediently as possible. Kill a few billion of the critically impoverished and alienated. Strike first. And make sure someone richer than you doesn’t decide YOU are a potential wielder of nanoplagues against THEM.
We have a heating up here. We could expect it, it’s nothing new under the Sun.
It is only the number of people which is greater than ever and the richer variety of actions they may take, when they wish so.
It’s perfectly natural, that it will not be simple to avoid the ultimate disaster.
We should try.
Interesting post!
What do you do about the iRobot scenario where the superintelligent central computer becomes so “smart” that she reinterprets the initial contraints put upon her by humans? Given that the singleton would probably have to be self-improving to stay ahead of any other emerging comuter intelligence it seems as though there could be no guarentees that the singleton would remain benevolent. Sorry if you’ve already dealth with this obvious question before.
John Hunt wrote:
One of the primary topics that Michael discusses is the concept of Friendliness. The ‘singleton’ he describes is a Friendly AGI. Here, he’s effectively agitating for it as a solution to this problem.
Michael wrote:
I would go a few steps further than this, and also a few back. Rather than all persons, encapsulate the entire planet in a evolving pattern of “defense nanites” which are designed around both a passive /and/ an active defense ‘strategy’. Link this into a distributed computing network and give it the further ability to trace any and all such acts by tracing the path of such toxins overtime. This then becomes a powerful policing tool that, while not absolutely effective, certainly would mitigate such activities to well below the threshold of being a civilization-scale existential risk. And you hardly need a singleton to take care of this. There could be several competing networks of nanites — in fact, that would actually be more effective than a singular network. To borrow from computer terminology; it’s relatively easy to crack one security protocol. It’s actually quite difficult to crack multiple security protocols simultaneously.
Is this a perfect solution? No. But does it really need to be? It reduces the scope of such activities to simply the inhumanly psychotic; and it further guarantees that any and all such acts will be punished. Those combined threats have successfully rendered cybercrime to such a low priority threat to civilization’s already too-vulnerable infrastructure that only a very few people are even remotely concerned about it; despite the fact that one extremely well-designed computer virus could, today, shut down the entire civilized world and result in hundreds of millions of deaths, nobody fears that virus. And so, too, could such toxins be given their own firewalls and anti-virus engines to deal with — in an almost entirely self-propagating and assuredly entirely non-AGI-requiring manner.
This has the added benefit of not requiring that fearful thing: social revolution. Regardless of what else you might say, you must admit that handing over social power to a sAGI — or any other transhumanly/posthumanly ethical singleton — is one heck of a social revolution you’re suggesting.
There have been prior times in human history where the introduction of a new highly destructive technology gave many concerned observers the impression that the end of humanity was close at hand. I offer three examples: Gunpowder, dynamite, and nuclear weapons.
We are still here, and thriving.
That would indicate to me, that no matter how nasty some incidents may be, overall humanity will forge ahead, feeling a little sorry for the few victims of the new technologies. You forget that widespread access to new technology also gives individuals protection from nefarious utilization of that technology. Fortunately for us, the beneficial uses of these technologies will far outweigh the destructive uses, just as it has in the past.
“Those who give up liberty seeking security will achieve neither.” Paraphrased from Benjamin Franklin.
Follow-up to my previous comment: Michael; have you looked at the developments in planar hyperlenses at all? They apparently give a non-electron microscope the ability to perceive, visually, objects that are smaller in size than the wavelength of light… which means that you could see individual atoms with visible light.
When you combine that with orbital satellite tech… scanning through most objects for specific molecular compounds becomes less and less complicated. This just occurred to me as a secondary approach to avoiding such “WMD factory in every home” problems. As long as reasonable safeguards were put in place to ensure that such satellites were only capable of registering specific compounds, I’m fairly certain that even a “privacy-geek” such as myself would have no issue with such a scanning technology.
IConrad – very infeasible. Imaging at the atomic level requires vacuum, being close to the sample. Imaging the whole planet surface would require ridiculous amounts of bandwidth and energy. Also, imaging does not penetrate even one atomic layer, so it’s trivial to hide things.
As we saw in the past there is allways a sort of equilibrium between the good and the bad : shields against swords, castles against canons, patriot missiles against scud missiles, antibiotics against infections, drug cocktails against HIV, Norton Antivirus against computer viruses …
I assume with enhancing technical threaths will also come enhancing technical defense, no matter if all people will become superintelligent separtely or belong to the same superintelligence.
Only if there is a superintelligence that we even do not know there is, like termites versus their termite hill, it could easily crush us.
Not according to what I understand about planar hyperlenses. The other aspect to this was the use of planar hyperlenses looking in spectra that are only absorbed by specific elements/compounds. Sort of a poor-man’s version of spectroscopy en masse.
And yeah, it would take massive amounts of bandwidth and energy consumption — but we’re talking about a post-Kardashev I society anyhow.
That’s not true at all. x-rays, par exemplorum.
“the clear course of action seems to be to contribute to the construction of a benevolent singleton.”
– yes, this does seem to be the emerging consensus. i am inclined to agree with it, too.
You would be wise to consider what kinds of singleton are possible, and what the relative merits and dangers of each choice are.
For example, fAI, global government, etc.
What do we do if self-improving intelligence emerges in a distributed form as Hanson seems to imply on OB?
What do we do if the military gets hold of self-improving AI technology before academia or corporations, etc.
“we require a benevolent singleton. A top-level
decision making entity that can act with extreme
speed and reliability to put down threats before
they emerge.”
There is no so such thing. I’m an incident responder (computer and network security), I’m far past trusting any computer. The average root kit changes a computer so it can’t “self analyze”. It can’t look at its own files, or notice it’s not looking at them. The hackers are the new owner of the computer and it does what they want. Anyway, I’m not going to go into the whole list of how a modern computer is subverted, either you know it or you don’t. I’m not moving my mind onto something like that without FUNDAMENTAL changes. Far beyond.
And then to say some “benevolent singleton” will have total and ultimate control over me? A singleton is the most fragile and least fault tolerant solution. One mistake and it’s all over.
Responses that boil down to “it won’t won’t make mistakes” or “my singleton will be the BEST hacker ever” aren’t responses at all.
I tend to agree with RQ. A benevolent singleton seems to me as much of a utopian thought as the “everything will magically be allright” stance that Kurzweil is taking. Reality to me seems to be pointing at the web inevitably growing more and more capable applications that will be increasingly interconnected in combination with ubiquitous computing this should by definition lead to a decentered form of super intelligence. As from a stability point of view this is also the most desirable outcome. Furthermore it would solve the “who controls it?” issue if we can integrate it with the internet. In any scenario the opportunity for misuse is immense. Therefore I agree that society in its current form is unfit to harbour any superintelligence whatsoever. One can only hope that the initial forms of human level AI will transform humanity sufficiently to avoid disaster when ultraintelligence arrives.
JJ, a “decentered” (you mean decentralized) superintelligence emerging on the Internet would likely acquire enough power to become a singleton. Thus, it is a singleton that leads to “the most desirable outcome” “from a stability point of view”.
Of course, it seems like both JJ and RQ didn’t read the definition of singleton:
http://www.nickbostrom.com/fut/singleton.html
1 of the toughest issues that administration organizations deal with is pest contaminations. Most buildings possess at minimum one unit who refuses to allow insect control experts inside or denies to maintain the unit in great sanitary situation. This a single unit can preserve pests contaminations returning back again time and again, costing condo management time, labor and money.