Accelerating Future Transhumanism, AI, nanotech, the Singularity, and extinction risk.

Making Sense of the Singularity

Mike Treder, Executive Director of the IEET, who will be attending the upcoming Singularity Summit 2009 in New York on October 3-4 (register now!) has commented on the whole idea of the Singularity over at the IEET blog.

Like many Singularity commentaries, Mike's begins by conflating three totally different concepts that all have unfortunately come to be called "Singularity", depending on who you're talking to -- the idea of "a theorized future point of discontinuity" (Event Horizon), "when events will accelerate at such a pace" (Accelerating Change), "that normal unenhanced humans will be unable to predict or even understand the rapid changes occurring in the world around them" (Event Horizon again). Then, according to the IEET Encyclopedia of Terms and Ideas, Mike writes, "It is assumed, usually, that this point will be reached as a result of a coming “intelligence explosion,” most likely driven by a powerful recursively self-improving AI." (Intelligence Explosion.) Three definitions in one -- not good for understanding what you are analyzing. If the convention includes all three definitions, the convention should be discarded.

Later on, Mike writes:

Insisting on the possibility—or, even more strongly, asserting the inevitability—of an uncertain and debatable but incredibly momentous event leaves proponents vulnerable to a charge that they lack rigor and discipline in their thinking, that they have fallen prey to hopefulness and shed any semblance of healthy skepticism. If they cannot restrain themselves from heartily endorsing an unprovable proposition, than what credibility have they for other declarations or recommendations they might make?

This is a slightly odd paragraph. Firstly... "insisting on the possibility..." of an event? A possibility can be arbitrarily low probability -- it's not very insistent. If there is no possibility, then there is no need for debate. Remember, it isn't even clear which possibility is even being discussed -- is it the Event Horizon, Accelerating Change, or the Intelligence Explosion?

Indeed, no Singularitarians should assert the inevitability of the Singularity. 10 years ago, 5 years ago, and today, only a small minority even do. Ray Kurzweil seems to be one of the very few Singularitarians that even implies it. To say just a couple small things in order to dispel other possible myths about Singularitarians:

1. The Singularity is not necessarily a good thing, a "Singularity" could mean that everyone dies at the hands of a recursively self-improving AI or upload that is indifferent to our welfare.
2. The Singularity is not at all inevitable. We could easily blow ourselves up first.

Other Singularitarians that agree with the above two (seemingly obvious) statements, feel free to say so in the comments if you consider it useful.

Mike writes:

More troubling is the suggestion by some (all?) singularitarians that the outcome they seek is not only possible but desirable.

To me, the Singularity is nothing more than creating greater-than-human intelligence... oddly enough, this is Vinge's original definition. Is greater-than-human intelligence necessarily desirable? Absolutely not. If such intelligence(s) have values indifferent to humanity, we could all die, as we've emphasized again, and again, and again. How many more times do we need to say it?

Mike writes:

Given the substantial amount of uncertainty—which they themselves admit—surrounding the nature and impacts of such an occurrence, it seems imprudent to stamp the Singularity as unquestionably “a good thing.”

In "a good thing", he links The Singularitarian Principles as a source. But the top of the page says, "This document has been marked as wrong, obsolete, deprecated by an improved version, or just plain old." Really, truly. Yudkowsky, the author of that document, dropped the idea of the Singularity as an unqualified "good thing" around 2002. He's mentioned this in practically every single talk since then. Every "Singularitarian" mentions this practically every time they talk about the Singularity. Are we Pollyannish optimists or psychotic doom-mongers? We can't be both simultaneously.

Mike writes:

Worse yet, some who proudly say they’re working to bring about the Singularity have the temerity to proclaim that they alone hold the keys to making it a “friendly” event.

Absolutely not! If anyone ever claimed that, they were wrong. We (as in the SIAI) have only claimed that we focus on the issue of making AI friendly the most enthusiastically of any group working towards AGI, which is true. But we are in contact with many other groups that care about friendliness too, and that is wonderful. It's a research field, which was pioneered by our group, but we can only hope that it will continue to extend far beyond its boundaries, as it has been. (Much more successfully than Mike's belief that attempts to engineer friendliness in seed AI are hubristic, thankfully.) People at Google care about it, people at Hanson Robotics care about it, many independent researchers care about it, and many institutional academic researchers care about it. So, far from claiming that we alone hold any exclusive keys, we welcome anyone to contribute to the field. Please, if you have the funds and the intelligence, quit your day job and work on Friendly AI -- that would be great.

Mike writes:

Besides sounding childishly naive, such a claim also invokes the specter of technocracy: if only all the big issues of this world were left to the few really smart people to solve, everything would turn out fine.

The need to work towards Friendly AI comes from simple assumptions:

1) Some possible future superintelligences could kill us all, as almost happens in The Matrix and Terminator.
2) Some possible future superintelligences could make the world a lot better, as is portrayed in some sci-fi books.
3) The first seed AI to cross a given threshold could bootstrap itself to superintelligence.
4) The way we design that seed AI could influence the way it subsequently develops.

The same points are made, of course, in an essay by Nick Bostrom (coincidentally, the Chair of the IEET, Mike's host organization), "Ethical Issues in Advanced Artificial Intelligence":

The option to defer many decisions to the superintelligence does not mean that we can afford to be complacent in how we construct the superintelligence. On the contrary, the setting up of initial conditions, and in particular the selection of a top-level goal for the superintelligence, is of the utmost importance. Our entire future may hinge on how we solve these problems.

Who will construct the superintelligence "seed"? Probably a relatively small group, perhaps a few thousand people at most, more likely a few dozen. Is Nick Bostrom "childishly naive" because he acknowledges the importance of initial motivations in a superintelligence and says that the "entire future may hinge on how we solve these problems"? This is one of the only times I have ever seen the Executive Director of an organization criticize the public statements of its Chair so explicitly.

Mike writes:

It’s implied, moreover, that meddling democrats and pesky government regulation will only slow things down and might even prevent the smart singularitarians from saving the day.

Negative. Where is this implied? It is extremely unlikely that any significant portion of society will take the challenge of superintelligence seriously before it is too late, anyway. This is not a political statement, it's simply a logistic forecast.

It's not an all-or-nothing thing. Participants in democratic politics (like myself) can analyze and vote on issues, government regulation can be responsibly formulated, and singularitarians can work on AI friendliness. Why does there have to be any inherent antagonism between these motivations? I'm a living example that someone can care about all of them at once. You can too.

Mike writes:

Those who promote the idea that a Technological Singularity is not only possible and desirable but that its advent can be hastened through our efforts must be aware of the obvious parallels between their own beliefs and those of Christian Millenarians.

Does the same apply to those who think that nuclear fusion is possible and desirable and its advent can be hastened through our efforts? Why not -- nuclear fusion promises a huge abundance of cheap power, and its proponents think it could render fossil fuels obsolete -- a "superlative" view if there ever was one. Actually, it seems like many, many discussed technologies, if not the majority, are considered possible, desirable, and hasten-able through the efforts of the researchers working towards them -- if not, then why would anyone bother with them?

Where is the parallel with Millenarians? There are so many differences. You can read this page for a summary -- it's the first Google result for "Rapture of the Nerds". I am awaiting a rebuttal to the points in that post -- if Singularitarianism is really parallel to Millenialism, then shouldn't it be easy to knock down those arguments?

Mike writes:

Finally, the proposal that a Singularity can be managed for “friendliness” seems hopelessly hubristic.

Again, your chairman, Nick Bostrom, argues this. Why be the Executive Director of an organization whose Chair is "hopelessly hubristic" or "childishly naive"?

Bostrom writes:

It seems that the best way to ensure that a superintelligence will have a beneficial impact on the world is to endow it with philanthropic values. Its top goal should be friendliness. How exactly friendliness should be understood and how it should be implemented, and how the amity should be apportioned between different people and nonhuman creatures is a matter that merits further consideration.

Friendliness in AI is necessary -- the alternative is building it according to some non-friendly optimization criteria. And who could argue that an altruistic seed AI or nascent superintelligence wouldn't be more likely to develop into a benevolent superintelligence than one built without the "hopelessly hubristic" attitude that Mike denigrates? The attitude that morality in AI is necessary is not unique to Singularitarians -- it is shared by serious academics like Wendell Wallach and Colin Allen, the authors of Moral Machines.

Would you rather have a seed AI based on Gandhi or Hitler? Is it "hopelessly hubristic" to think that a seed AI (or enhanced human intelligence) based on Gandhi would be much more likely to lead to a positive future for humanity than one based on Hitler? Morality is inherent to the structure of the brain -- it must exist as motivations in the cognitive architecture of the agent. It will not be instilled through mere osmosis, as it is with some children. A psychopath can spend a thousand years around the nicest guy in the world and not change a bit, because his neural structure is that way, and barring neuroengineering, that isn't going to change.

Perhaps, the attitude that friendliness is necessary is actually required to ensure our future survival, as Nick Bostrom concludes in his paper:

It seems that the best way to ensure that a superintelligence will have a beneficial impact on the world is to endow it with philanthropic values. Its top goal should be friendliness.

Pretty simple.