Not So Much a Prediction as a Notice Tuesday, Jun 3 2008
friendly ai and singularity 1:01 am
There are two types of “Singularitarians”. One type of Singularitarian, mostly imaginary because no one I’ve ever met actually self-identifies this way, is the type defined in Ray Kurzweil’s 2005 book The Singularity is Near. Since this book has a permanent place on my desk, I need to reach only about 10 inches to retrieve it and open to the page that defines it. Here it is:
“A Singularitarian is someone who understands the Singularity and has reflected on its meaning for his or her own life.”
Alright. Unfortunately, I’m motivated to entirely ignore this definition, as it was introduced five years after another, different, better definition. The better definition was introduced by Eliezer Yudkowsky on January 1st, 2000, in the Singularitarian Principles, which is admittedly slightly out of date:
Definitional Principle #1: Singularity
A Singularitarian believes that the Singularity is possible, that the Singularity is a good thing, and that we should help make it happen.
Definitional Principle #2: Activism
Singularitarians are the partisans of the Singularity.
A Singularitarian is someone who believes that technologically creating a greater-than-human intelligence is desirable, and who works to that end.
A Singularitarian is advocate, agent, defender, and friend of the future known as the Singularity.
Definitional Principle #3: Ultratechnology
The “Singularity” is a natural, non-mystical, technologically triggered event. We, the Singularitarians, are allied in the purpose of bringing about a natural event through natural means, not sitting in a circle chanting over a computer. There are thousands, perhaps millions, of stories and prophecies and rituals that allegedly involve something that could theoretically be described as “greater-than-human intelligence”. What distinguishes the Singularitarians is that we want to bring about a natural event, working through ultratechnologies such as AI or nanotech, without relying on mystical means or morally valent effects.
Definitional Principle #4: Globalism
Similarly, although the Singularity is simply the creation of greater-than-human intelligence, the “Singularity” in “Singularitarian” is the Singularity as seen from the perspective of the vast majority of humanity. It’s the event seen from a global perspective, just like the “liberty” in “libertarian” is global. If you don’t advocate global liberty, you aren’t a libertarian. If you don’t advocate global Singularity, if you just advocate a personal, private Singularity, then you’re not a Singularitarian.
My main qualm with the above is the notion that the Singularity is necessarily a good thing, when it isn’t. Superintelligences could easily wipe us out to make way for structures that provide them greater subjective utility. I would suggest that Mr. Yudkowsky take the few minutes necessary to update the page to reflect this.
Anyway, the biggest difference between Kurzweil’s Singularitarianism and Yudkowsky’s Singularitarianism is that practically anyone can qualify for the former, while the latter is reserved specifically for rationalist activists who want a Singularity to benefit all humanity. Anyone who has watched any of the Terminator movies and thought that such a scenario might be possible in several decades has thought about the “Singularity” and its meaning in his or her own life, so that may be millions of people, but this isn’t a significant delineation. It’s conspicuously inclusive, in a boring way.
Yudkowsky’s Singularitarianism is an activist singularitarianism — we’re going to work towards smarter-than-human intelligence, join us if you wish.
In this sense, this Singularitarianism is more of a notice than a prediction.
This group, however small it may be, is focused on constructing a self-improving artificial general intelligence (AGI) that acts beneficently towards all humanity, however long that takes.
Join us if you wish.

June 3rd, 2008 at 7:04 am
Already with you, man.
June 3rd, 2008 at 10:05 am
I agree, Michael, that the Singularity is, by definition, neither good nor bad. I’ll even go so far as to say that it is not even something we should be working for, but rather something we (particularly as Transhumanists) should be working against. The Singularity is an event horizon on our ability to predict and affect technological progress, otherwise the word really doesn’t fit what we’re trying to describe. We should be attempting to push back that event horizon rather than pull it closer to us. That doesn’t mean we should be trying to slow down our technological progress generally, but it may mean that we should be more cautious in some areas and seek better to understand ramifications before diving carelessly into the unknown without careful calculation and mitigation of risks.
June 3rd, 2008 at 2:44 pm
As Warren commented regarding a post a week or so ago, and as I elaborated, humankind is now at the very outtermost boundary of the penumbra of the Event Horizon of…the Singularity, *qua* hyper-exponential techno-economic (and therefore social) change/progress. Part of this Singularity will consist, of course, of that *other* meaning of Singularity, i.e., creation and instantiation of greater-than-human intelligence.
Our job is to make other folks aware of the benefits and risks: Facilitating the accelerated development/deployment of the *former*, eliminating or dampening-as-much-as-possible the latter.
Welcome to the future.
“We’re **living** science fiction.” — Robert Anton Wilson.
June 3rd, 2008 at 9:13 pm
Are the controversies of today the doldrums of tomorrow?
Most people don’t remember that organ transplants used to be highly controversial on even conceptual level; yet only a few years after they where made possible and widespread, the controversy died. There are still issues with organ transplant, but rarely does somebody say that transplanting another person’s organs is fundamentally bad.
Maybe AI will follow this same path. Or maybe not.
Our caution serves an evolutionary purpose, while it sometimes leads to bad things, its generally been helpful. I’d rather consider possible problems before they become problems then run head first into one; even if these actions are viewed as “paranoid” in the future.
June 3rd, 2008 at 11:24 pm
True enough…
June 4th, 2008 at 4:27 am
I thought the reason it’s called the Singularity is our inability to see what happens beyond it. If so, it follows that we can’t know whether it will be good or bad for the human race. Therefore believing that it is a good thing must be largely an act of faith.
June 4th, 2008 at 11:53 am
My problem with Ray’s definition is the
”
A Singularitarian is someone who understands the Singularity
”
part…
Who the hell understands the Singularity??
Almost by definition, we cannot…
– Ben G
June 4th, 2008 at 3:57 pm
Well-said, Ben. We can try to anticipate things, and guide or “nudge” them as best we can (as Hans Moravec says in *Robot*…), but the Singularity, both qua Event Horizon and qua Greater-than-Human Intelligence obviously can’t be fully understood. (Gee, didn’t Popper point this out in an even more generalized way back in *Poverty of Historicism*?!—sheesh!)
June 4th, 2008 at 8:55 pm
Charles,
That’s why I said it should be changed. Most people who self-identify as Singularitarians would argue that the Singularity is not necessarily a good thing.
June 5th, 2008 at 3:54 pm
I give Yudkowsky the high five on the principles mentioned until landing on that of point 4, Globalism. Terms like “liberty” and “libertarian” are not self-evident. There’s clear inductive reasoning in the prose, yet he might want to define these often broadly defined terms. Also, I don’t see the harm in having either a private or global (public) life during what may be considered the Singularity Age.
Ray-Ray gets the high five on ‘all qualify’. If Eliezer speaks of liberty, then its important that his version be defined, particularly when the text seems to suggest some sort of qualification–something that conflicts with many notions of liberty.
Mike, I’ve yet to come across an argument that’s convinced me that AI, when “conscious” or perhaps “self aware,” might want to do away with us human folk.
I’m more concerned with what humans do with tech than what tech does with itself, given the foundational stages of AGI operate within steady state hardware. Dynamic molecular hardware, based on simulations of the human brain is something to have more concern with.
Perhaps, in itself, it can liken to a gun or the holy grail. For some a holy grail is the gun while for others it is of the reverse. Preferences, such as how one views liberty, are important for notions of a ‘positive’ Singularity. However, if something that operates like or better than the human brain, functions better than the human body, even if the ‘new kind’ does not use force, there will not be much to stop others, particularly those who already favor radical augmentation, to, as a once classifiable species, become and therefore terminate one’s former self by reconstructing into someone (something) else, thus killing the previous self based on ever evolving spiral of preferences.
To me, perfection is a noble pursuit, yet if attained, this, I’ve concluded, is only achievable through death. There’s something to ponder on. ;P
This may be something far more complex than the human mind could fathom, particularly since these technotoys briefly mentioned are not yet available. Building thought experiments or scenario building based on foreseeable technology in the ever fleeting present can at least help foster methods of action for a most preferable future for us all.
June 6th, 2008 at 4:10 pm
The thing that I love about the debate is that there is any fear about the possibility of the singularity happening.
I mean, when you break it down;
Option A: Singularity doesn’t happen, everyone keeps aging and dying until the sun burns out, if we live that long as a species, and we fade into history.
Option B: Singularity does happen, but we are exterminated by robots, and we fade into history, but we’ve given birth to something technically better than us, at least in the metrics of genocide.
Option C: Singularity happens, we become immortal, and as a species we spend an eternity exploring the multiverse and our own consciousness.
I think that you’ll find that fear of Option C is the same fear that makes 90% of speculative fictional future some kind of dystopia.
Fear of the unknown. Even if it might be really good. Death and disease are bad, but they are known, and understood, or at least familiar.