Response to Cory Doctorow on the Singularity

Cory Doctorow is an editor of what, for a long time, was the most popular blog on the Internet, Boing Boing. (It’s now #2 after Engadget.) He is also a science fiction author who is known for copyright activism on behalf of the Electronic Frontier Foundation. In the Spring 2003 issue of Whole Earth Magazine, an article of his was published, “The Rapture of the Geeks”, that ripped into advocates of the Singularity and intelligence enhancement, such as myself. I will respond to the central accusations.

First, a couple definitions. The Singularity is the technological creation of smarter-than-human intelligence. We can further specify good Singularities, where this smarter intelligence is on humanity’s side, and bad Singularities, where it isn’t. So-called Singularitarians are individuals who advocate intelligence enhancement for global benefit. Rather than tackling the really hard problems – poverty, war, hatred, poor infrastructure, mental and physical illness – at our present level of intelligence, Singularitarians advise pursuing intelligence enhancement and then applying qualitatively smarter intelligence to these age-old problems. We also foresee a recursive self-improvement process resulting from smarter-than-human intelligence, where the first superintelligence is much better than humans at coming up with new intelligence enhancement techniques, and applies them iteratively, further magnifying the initial gains. To a Singularitarian, intelligence enhancement that improves benevolence as well as brainpower is the best possible investment in humanity’s future.

Cory Doctorow: “The Vingean Singularity is at the center of a classic mystical belief system: to believe in The Singularity is to believe in the transcendence of human flesh and the ascension to a higher state—a belief that, in turn, depends on several highly dubious articles of faith.”

Here, Doctorow associates the Vingean Singularity with transcending the flesh. While it is true that many advocates of the Singularity believe in the possibility of mind uploading, cyborgization, etc., none of these things are necessary to make intelligence enhancement a highly desirable prospect. Even if life and intelligence were somehow permanently affixed to proteinaceous water envelopes forever, it would still would be prudent to pursue intelligence enhancement for its own sake.

Humans share 98% of our genetic material with chimps, but the step from chimps to humans produced creatures that could walk on the Moon, exploit the power of the atom, and build skyscrapers. If a similar jump up in intelligence could produce the same discontinuous results, then wouldn’t it be fascinating to take that step? And if that step is theoretically possible and will happen one day anyway, wouldn’t it to responsible of us to help guide it so that the first superintelligences at least are given human-friendly initial conditions rather than human-unfriendly initial conditions? If the Singularity were sparked by human intelligence enhancement, would you rather the first augmentee be more like Fred Rogers or Vladimir Putin?

There are multiple reasons why the Singularity is not a mystical belief system, but the most obvious is that it is experimentally testable. If we cannot build smarter-than-human AIs despite our best efforts and human-equivalent computing capacity, if no brain-computer interface or genetic enhancement project gives rise to improved intelligence, then it will be proven that smarter-than-human intelligence is forbidden by the laws of the universe.

But if a reliable intelligence enhancement procedure is developed and can be applied to anyone for a low cost, would that not be an “ascension to a higher state” of just the type that Doctorow is belittling? It would be an ascension to a “higher state” in the real world, based on making deliberate neural modifications to let people think faster, more creatively, more empathically, with expanded working memory and capacity for complexity. This can be done by taking control of our own brains at the physical level, rather than the more superficial route (but so far the best we’ve had) of traditional learning.

Doctorow: “First off, Singularians ask you to believe that a model of a brain in a computer, properly executed, will become conscious—will, in fact, have a consciousness continuous with that of the person whose brain was scanned. While it’s true that consciousness depends on the brain— judicious experimentation with a bone saw and scalpel can readily demonstrate this—it’s an enormous leap to conclude that consciousness’s seat is in the brain.”

Where else could it be? Doctorow is contradicting decades of work in brain science by suggesting that consciousness may reside somewhere external to the brain. Does consciousness reside in the stomach? The heart perhaps? Or is it floating by us at all times, on the supernatural plane? I’m not sure what he is suggesting, but it sounds pseudoscientific.

Regardless, Singularitarians are not asking anyone to believe that a model of the brain in a computer is continuous with its real-life counterpart, or that uploading is possible. It just so happens that many do believe it, but this seems like more of a common belief than a central component of Singularity advocacy, namely intelligence enhancement.

But if the subject is brought up, why not respond: if consciousness disappears when a brain is implemented on a computer, then it should be possible to observe consciousness disappearing in partly-computerized brains. For instance, people with hippocampal implants would be less conscious than ordinary human beings. Somehow I doubt this. Even if computers as we know them turn out not to be able to simulate conscious beings, who says we are limited to traditional computers based on the serial von Neumann architecture implemented in silicon? We could try parallel computers, biological computers, ultrafast neuron-equivalents, carbon computers… whatever works. The point is not to have a philosophical shouting match, but to dismiss carbon chauvinism – the idea that all life or intelligence must depend on traditional biological building blocks. For a humorous angle on this, see the short story “They’re Made Out of Meat” by Terry Bisson.

Doctorow: “Then there’s the further presumption that consciousness exists at an atomic or even molecular level: that an atom-by-atom copy of the brain, properly modeled in a Turing Machine, will have all the data necessary to awaken into consciousness. As more and more subatomic particles are catalogued, particles whose properties range from counterintuitive to goddamned spooky, it seems equally probable that nano-disassemblers’ pincers will be far too clumsy to ever extract the important information contained in a brain. It’s cargo-cultism: the airstrips bring the airplanes, so if we lay down airstrips, the planes will come back. Put all a brain’s atoms into a brain-shaped pile, a mind will come back.”

How come embryogenesis keeps putting the brain’s atoms in a brain-shaped pile a third-million times every day, and the mind keeps coming back? If it is necessary to duplicate pregnancy rituals in order to create a conscious being in a non-carbon substrate, then it will be an inconvenience we’ll have to bear. Again though, the issue of whether or not a human brain can be uploaded is irrelevant to the primary issue of whether or not intelligence enhancement (the Singularity) is worth pursuing.

Doctorow: “The Singularity depends on hypothetical technological events — nanotech, brain scanning, consciousness in the brain, sufficient granularity in the scan—but none is more wishful than the belief that the correct model will be lucked into.”

It’s interesting that Doctorow set out to write an article on the Singularity and ended up writing an article on mind uploading. This, along with his use of the term “Singularian” when he means “Singularitarian”, shows that he didn’t really research this article very well, but probably wrote it as a reaction to something he read that conflated the Singularity and mind uploading. A Google search for “Singularian” only yields a few results – all instances where people made up the word erroneously. “Singularian” is sort of like “irregardless” – a made up, etymologically incorrect word that gets spread about on a limited scale through repetition.

Doctorow: “After The Singularity, we’ll be immortal. All goods will be nonscarce. Entropy will be tamed. We will have complete mastery over our selves and our environment — we’ll be ascended masters. The best part is, we’ll get there using Moore’s Law. Write code, get smart, advance the cause and soon, you, too, will be immortal.”

This seems like an attack on Ray Kurzweil’s particular views, but it is a straw man because by definition, we cannot know precisely how things will go after the Singularity. The standard view is simple: if we aggressively enhance our own intelligence, the benefits could be large. Because most intelligence enhancement advocates are also transhumanists, the ideas of molecular manufacturing and radical life extension co-occur with discussions of the Singularity, but they are not the same thing. It’s hard to tell if Doctorow picked this up while onstage at the Singularity Summit at Stanford, but I sure hope so.

Doctorow: “Your mystical belief: that everything will just transform on its own, for the infinitely better, because, well, because that’d rock.”

From the beginning of an organized movement in support of the Singularity (around 2000), there has been the idea of personal responsibility and direct activism. So, as far as I know, there are zero thinkers on the Singularity that think it would be totally inevitable. Support of routes to smarter intelligence is a pro-active thing, that primarily manifests itself in Artificial Intelligence projects.

Comments

  1. Joshua Fox

    In his recent interview in the G’day Podcast, Eliezer Yudkowsky criticized some Singularity fiction as postmodern analyses of our contemporary difficulties in dealing with advances in technology. I think that he was referring to some of Cory’s work, among others.

    Cory Doctorow, too, has repeated the well-known truth that all science fiction is about the present day. Thus. perhaps we should re-interpret even this non-fiction piece as “future as symbol for the present” — a comment on society in 2003, rather than on what may emerge in the future.

    That’s rather difficult, given that it is not the plain meaning of Cory’s article.

  2. Just wondering: why are you responding to this article now? it’s 4 years old!

  3. Joshua Fox

    Roko, we’re living in an accelerated world, but lets not forget that in the world of scholarship, for example, it is not unusual for people to comment on texts tens, hundreds, or thousands of years old — indeed, articles are 3-4 years old the moment they appear in a hardcopy journal! So it’s OK to comment on something from a few years back.

    I was actually looking around for an intelligent response to this article by Doctorow. I am a big fan of his and often quote him, but his talk at the Singularity Summit was barely on the Singularity and this article shows that he does not grok the concept.

    When a up-to-date, future-looking thinker doesn’t get an idea like the Singularity, one should ask why.

  4. I wonder if Cory has changed his mind since then.

    In any case, good post Michael. His piece deserved the deconstruction; it was pretty badly researched and had a very annoying condescending tone.

  5. Roko, because I feel like it. I can respond to The Epic of Gilgamesh if I want. Why are you hassling me?

  6. Definitely agree that “Singularity = religion” is a misguided notion at best, and your response made a good work of singling out the worst parts of this particular text. Still, there were a couple of points where I thought it could’ve used some work:

    There are multiple reasons why the Singularity is not a mystical belief system, but the most obvious is that it is experimentally testable. If we cannot build smarter-than-human AIs despite our best efforts and human-equivalent computing capacity…

    I wouldn’t really call this an experimentally testable prediction as such – after all, one can always respond “well, while we haven’t figured out how to build smarter-than-human AIs quite yet, but if you give us another 20 years…” It’s a bit like saying that traditional religions are experimentally testable – if Armageddon never occurs and Jesus never arrives to raise the dead, then it will proven that it isn’t allowed by the laws of the universe.

    But if the subject is brought up, why not respond: if consciousness disappears when a brain is implemented on a computer, then it should be possible to observe consciousness disappearing in partly-computerized brains.

    But this assumes that we have an objective method of knowing when another person is conscious and when they aren’t. Maybe we’ll figure it out, or maybe a superintelligence will figure it out once we’ve failed, but that line of argument probably won’t help much against a crowd that’s already sceptical about the Singularity…

    But other than that, good post. :)

  7. Tom McCabe

    “I wouldn’t really call this an experimentally testable prediction as such – after all, one can always respond “well, while we haven’t figured out how to build smarter-than-human AIs quite yet, but if you give us another 20 years…”

    You could always come up with a mathematical proof that smarter-than-human intelligence is disallowed.

    “It’s a bit like saying that traditional religions are experimentally testable – if Armageddon never occurs and Jesus never arrives to raise the dead, then it will proven that it isn’t allowed by the laws of the universe.”

    Most religions have statements which turn out to be both false and testable (all the Abrahamic ones have blatant myths in them like the Flood and such).

  8. uuum I was just wondering. It seems a decent and necessary thing to do.

  9. that is, it seems decent and necessary to respond to Cory Doctorow’s article. No hassle intended.

  10. Kaj,

    The ultimate way of disproving the viability of the Singularity would be creating a high-resolution simulation of the human brain and seeing that it turns out to not work, or lack intelligence.

    Not sure how to measure consciousness but I think that claiming you’re conscious is a good enough measure as most… heavily cybernetic people would still claim they’re conscious.

    Thanks for the criticism though…

  11. Zaph

    Lets say an ordinary human was interfacing with some type of intelligence enhancing implant and they were aware of the major processes that occured within that implant (ie. they felt that the implant was a part of their “mind”). Wouldn’t that give substantial credit to the idea that neural tissue isn’t necessary for conscious thought?

    If they were not conscious of its inner processes then it would be reasonable to assume they would describe it as a simple input-ouput relationship, and not something more complex, no?

  12. Anissimov wrote:

    The ultimate way of disproving the viability of the Singularity would be creating a high-resolution simulation of the human brain and seeing that it turns out to not work, or lack intelligence.

    Now, that’s a much better line to use. :)

    Asking if somebody is conscious or not might work as very weak evidence, but that hardly stronger than that (otherwise, I could claim that my one-line piece of BASIC code, 10 PRINT “I am a living, thinking being that was born in the sea of information, just as conscious as you are” was conscious). Very doubtful that it’ll convince Singularity sceptics.

    Myself, I’d probably use (for that matter, I have used) a line of argument not too dissimilar from the hippocampal implant one – point out that we lose brain cells every day, and if losing them doesn’t make us lose consciousness, why would replacing them with artificial ones do it? (Of course, even this isn’t a perfect argument, since you do lose consciousness after having lost enough brain cells… not to mention that, strictly speaking, we don’t know whether or not people actually do become consciousness-less zombies once they’re old enough ;))

    Zaph wrote:

    Lets say an ordinary human was interfacing with some type of intelligence enhancing implant and they were aware of the major processes that occured within that implant (ie. they felt that the implant was a part of their “mind”).

    I’m not aware of the major processes going on in my lower sensory areas, my memory, or for that matter really any part of my brain other than my immeadite consciousness.

    I simply get input from my eyes and senses and thoughts pop up in my mind as a response. I can direct attention to particular tasks or instruct my body to move in a certain fashion, and I get sensory data back: a simple input-output process, not anything more complex.

  13. I do not think consciousness is a requirement in terms of the Singularity.

    If an AI or program had self contained self initiated goal setting and integrated sensors and responses that iteratively improved and learned fast enough. Then all the trappings and behaviors of intelligence could be achieved then the consciousness factor is not necessarily relevant.

    We have highly complicated programs now that can win at chess that are not intelligent. We have systems for solving the rubiks cube optimally. We have expert systems on topics. We have new theory generators. If enough of those kinds of system could be integrated into something very fast, robust (error resistant), etc… then you could still get an intelligence explosion. If such a learning and self motivated system did not have limitations (some local maxima) which caused it to plateau in its ability to improve.

    I suspect that thing would at some point gain something that we would consider real intelligence. But I could imagine something both useful, powerful and dangerous if it did not have controls but did not have consciousness. Some kind of complex chain reaction.

  14. Warren Bonesteel

    Sometimes the words “consciousness” and “intelligence” seem a bit slippery, don’t they? From what little I’ve read about the AI community over the years, the goalposts get moved around on a regular basis.

    “We’ll achieve AI when “This” happens!”

    When that goalpost is reached, the detractors cry,

    “No, that wasn’t the solution! When we do “That,” only then will true AI be achieved!”

    When “That” goalpost is reached…

    As for myself, the first time I used a search engine (years ago) – and it told me that I had mispelled a multi-syllabic word – I thought, “How did you know what I was trying to spell?” (…and having a simple checkers beta-program defeat you is…humbling.) The first time I dealt with a voice recognition program was also a bit …spooky.

    The first time I was involved in a Turing test,
    it wasn’t at all easy to figure out which was the computer and which was the “real” human being. I did figure it out, but I was completely stumped for almost five minutes.

    I’ve also played a few computer games where I “knew” that it wasn’t the programmers but the game itself ‘who’ was trying to defeat me. I would swear in a court of law that it was anticipating my moves and actively and purposely engaged in the use of strategy and tactics.

    My cat is more intelligent than many a human being I’ve ever known. A friend’s parrot is more conscious of its surroundings than most people I’ve ever met. The parrot can hold his own in a decent, if simple, conversation. Days or even weeks later, he will often try to engage you in further discussion on the topic.

    Sometimes, it’s all a matter of perspective… and of egocentric pride.

    All of that said, in the next decade or two, I think that we will experience both an AI and a technological Singularity Event. After that occurs, things are going to be very interesting.

    I personally think that the line between AI and consciousness is becoming increasingly blurred. If it were not, we would not be having such conversations.

    As for postings in the comments section of a blog, the only thing I know for *certain* is that I am the only one of us who is self-aware. For the time being, I do operate under the assumption that most of you are also
    conscious, intelligent, self-aware human beings…but…I am only certain of my *own*
    conscious, self awareness.

    From that POV the discussion just as
    well concern whether any – or all – of us is actually a conscious, self-aware, intelligence.
    (Simulation Argument)

  15. Brian,

    I agree with you that consciousness should not be necessary, although it may emerge in any sufficiently introspective system.

    Warren,

    We’ll achieve AI when the AI obviously has general intelligence and is publishing scientific papers and achieving goals in the real world… this is how the majority has always defined “successfully achieving AI”. Only a small minority has ever said stupid things like, “when we have something that beats humans at chess, we have AI”, because a chess machine obviously isn’t a general AI.

    Some philosophers like to overcomplicate matters so that they can write doctoral theses on them. Intelligence and consciousness should not be that hard to define. Intelligence is the ability to solve complex problems in complex environments with limited information, and consciousness is something that has subjective experience.

    Your cat isn’t smarter than humans, heh. Seriously. Only a brain-dead human could be dumber than a cat. “Acting in ways you don’t like”, and “unintelligent” are two different things. By being all part of the same species, we all have the same basic cognitive hardware, the same way we have the same basic physiological hardware. This hardware is superior to a cat’s, no question. If you think your cat is actually human-intelligent then you are just anthropomorphizing it.

    It’s all fun to be philosophical and spacey and say we only know about our own conscious awareness and everybody else could be zombies, but honestly, to anyone who isn’t playing some elaborate mental game, it’s pretty damn obvious that everyone else is conscious.

  16. Warren Bonesteel

    There’s lots of brain dead humans running around out there. ;) Cognitive biases galore. The elaborate mental games are not my own. If you think that’s the case, you didn’t understand a thing that I shared. Although it can stand alone, what I shared cannot be removed from context with the previous three posts. (Think: “irony.”)

  17. Kevin Osborne

    I’ve got to say I got more out of the original article than the rebuttal.

    Doctorow is pretty much right on the money about the “Rapture of the Geeks”; though it’s clearly Transhumanism and not Singulatarianism which is an “Afterlife for Atheists”.

    I think what Cory is talking about with the seat of consciousness and runway metaphor talk is a (poorly phrased) pointer towards the developmental and environmental context that is so important to the human brain; the same kind of arguments that successfully poke holes in genetic determinism and evolutionary psychology. It’s not enough to replicate the strata, you may well need to replicate the input and affects and signalling and you may need to simulate doing so from conception to maturation in order to culture a sufficiently capable vehicle for conciousness-as-we-know-it. This -is- very much an attack om the Kurzweil(/Moravec?) approach however and says nothing about custom-grown conciousnesses, but it does have serious implications for uploading… which very much -does- border on the mystic.

    As a born-and-raised Atheist alot of the rhetoric on the Singularity does echo very closely to the same “Second Coming” excitement and apoplexy shown in religion. Its a bit pat but irresistable to look at oh-so religious Americans lapsing their childhood-indoctrinated faith and latching onto these promises for the future in order to satisfy the nicotine-like neural pathways that their religious upbringing has left behind. There is even a Mighty Sa-Tan character a.k.a Unfriendly A.I to act as a boogeyman unless we all go to Sunday school.

    Whether or not any of these comparisions are valid – certainly they’re incomplete – the fact is they can be made and have a certain resonance.

    Thats not Cory Doctorow’s fault; I think it’s wrong to shoot the messenger here, and maybe we should reflect on the message instead; “Truth Hurts” is the cliche that seems most apt.

  18. *shrug.* I think you guys are wrong. Singularity advocacy is no more a religion than preventing nuclear proliferation is. You also make the same mistake of mixing together the uploading issue with the Singularity issue. If you can’t even draw this simple distinction, why should we take your accusations seriously?

    It seems to me that you and Doctorow are “masochistic atheists”. It’s not cool to believe that life could get a lot better, or a lot worse. Because if that were possible, then certain actions would become very important at the exclusion of others. To masochistic atheists, anyone saying that something is very important is equivalent to a religion. You want to be able to do whatever you want and not worry about the planet being at risk. But the planet is at risk… welcome to the 21st century.

  19. Kevin Osborne wrote:

    Thats not Cory Doctorow’s fault; I think it’s wrong to shoot the messenger here, and maybe we should reflect on the message instead; “Truth Hurts” is the cliche that seems most apt.

    I’d say that it’s exactly *because* of singularitarianism’s superficial resemblance to religion that we should be so careful about making sure people understand the differences. No matter what it might look like on the surface, there’s a world of difference between a religion (where you believe something simply because somebody told you so) and Singularity activism (where you believe something because it becomes almost blindingly obvious when you actually look at science close enough).

    I do agree, though, that exactly because of the elements that make religions so attractive, singularitarianism is likely to also attract many people who will treat it like a religion, because it seems like a cool thing to believe in rather than because they’ve thought it through. (To be clear – I’m not implying that anybody commenting on this discussion is one.) While the core of singularitarianism is most definitely not religion, it could very easily develop to such on the fringes. In fact, it probably has already done so, though I haven’t encountered very many people treating it like a religion. Still, there are some… let’s not forget this discussion: http://www.sl4.org/archive/0602/14037.html

    I’d say Kevin is right, in a sense. Because of the unfortunate human tendency to pick up irrational beliefs, or to believe in rational things but without thinking them through rationally, we should be very explict about what we are. Singularitarians should uphold the highest standards of rationality, so that outsiders coming across the movement for the first time would see the core of Singularity activism first, not the ones thinking it’s a religion.

    But then, you’d all be maintaining the highest standards of rationality anyway, wouldn’t you? ;)

  20. You write “Humans share 98% of our genetic material with chimps, but the step from chimps to humans produced creatures that could walk on the Moon, exploit the power of the atom, and build skyscrapers.”

    We probably shared more than 99% of our genetic material with Neanderthals. They existed for (I think) around 250,000 years but made almost no technological progress. So, it’s the very slight differences between humans and Neanderthals that have allowed us to build our civilization.

  21. Kevin Osborne

    let’s make this clear – i don’t think belief in a Vingean Singularity is anywhere near equivalent to dogmatic belief in a religious text or belief (faith) system.

    Singlarity advocacy is a good thing. The Singularity may well be the best thing to have ever happened to humanity.

    But, as always, there’s a rub:

    from http://en.wikipedia.org/wiki/Dogma:
    “Dogma is the established belief or doctrine held by a religion, ideology or any kind of organization, thought to be authoritative and not to be disputed or doubted.”

    Where Doctorow is needling successfully isn’t the grounds of Singularity idea, it’s the aura, the spectre, the cultural and personal aspect and how it shapes our very human attitudes torwards it. The Singularity has become symbolic, and with that symbolism also came a certain amount of our baggage.

    Here’s some aspects that ‘Singularity Advocacy’ and ‘Born-Again Christianity’ could be said to share:

    - Evangelism: once you accept, you are bound to spread the word (hence the ‘advocacy’, no? otherwise why do we even care what people think?)
    - Promise of Eternal Life: radically enchanced longevity is equivalent, as is thoughtspeed overclocking
    - Everyday Miracles: performed by practitioners that herald the justness/righteousness of the cause; seems like both camps are keen on healing the blind, immobile etc
    - Commandments: highly idealised morals and ethics that have been agonisingly thrashed out by thought leaders (a.k.a monks) and which we all should agree upon, less we are seen to be unenlightened or barbaric
    - Impending Doom: the rest of the world is blind to our impending destruction, and only we are aware of the pitfalls ahead
    - The Rapture: as per Doctorow pretty much, the parallels here are glaring.

    There’s probably more comparisons that can be made, but just thinking about it is making me rather sick.

    One final thing: I don’t think anyone should stop being an advocate. I think the message is incredibly compelling, and that the effort expended and work being done is valuable and muchly needed. If we want to get the message across however, we should have the presence of mind to be aware of how we’re coming across and who we’re going to be compared to if we sound like.

  22. Evangelism – no one is “bound” to do anything. “Eternal life” – Aubrey de Grey promises life extension and no one calls him religious. Everyday miracles – no one is performing them or claiming anyone is. Commandments – none to speak of. Impending doom – Bertland Russell took great pains to avert catastrophe during the Cold War, yet he’s one of history’s most prominent atheists. Rapture – intelligence enhancement for all who want it is completely different than living in New Jerusalem as angels.

    If thinking about the comparisons makes you sick, then maybe thinking about any radical improvement in human life or any substantial risk would make you think the same thing. If radical opportunities exist, and radical risks exist too, then much of what you call “parallels to born-again Christianity” automatically fall out of it, apparently. Radical opportunities and radical risk are built-in aspects of technological development in the 21st century – aspects we can’t change. We can choose to ignore them, or pay attention to them. Comparing policy discussions around the Singularity to religions founded by Bronze Age tribesmen is only done by people who have a relatively superficial view of both. Sorry to be so blunt, but even “technology in general” matches the criteria for religion in this view you’re putting forth.

    In my experience in the past, those who compare Singularity advocacy with religion don’t actually believe that smarter-than-human intelligence is really possible. If they do see it as possible, then they can also see past superficial similarities to the underlying distinctions, and the words and actions seem like a perfectly rational response to the both the opportunity and the risk. The technological creation of smarter-than-human intelligence would represent the arrival of a new intelligent species – no less significant (in fact, quite more so) than meeting ETs. If we think that those aware of this are somehow behaving religiously, then we don’t actually believe such a technological advance is in fact plausible.

    It’s not a “damned if you do, damned if you don’t” situation. Either superintelligence is possible and Singularity advocates are being rational, or superintelligence is impossible and Singularity advocates are wrong.

    Note that Singularity advocates aren’t saying that the Singularity *will* definitely give radical life extension, or abundance, or solve social problems – we just say that a superintelligence would, by definition, come up with faster routes to these goals than we would.

  23. MCP2012

    Good discussion (as typical). Haven’t thrown in my 2 cents in a while (other pressing matters…), but thought I’d toss in a penny or two here.

    Michael you were right to respond to this snipe by Doctorow, and did so rather well (let’s hope Cory reads it & heeds it).

    Brian (& Michael in his concurence) is also right that an entity can have **intelligence**–perhaps even superintelligence–w/o anything all that especially like (as in the Nagelian “What’s It Like…”) **our** consciousness. From both an info-theoretic and control-theoretic POV, there would presumably be some sort of fairly robust (yet perhaps in a way, rudimentary) *awareness* (in the since of obtaining info from and apprending the environment). And there would quite possible be some sort of rudimentary “self” due to a self-monitoring, self-reflecting, self-adjusting (and, indeed, quasi-organismic) “loop”. But that self and its awareness might very well be rather alien as compared to our garden-variety consciousness (as indeed I suspect it would…)

    And when it comes to our *knowing* (or even intuitively/empathically guessing as to) “What It’s Like…” to **be** (from its own internal POV) that synthetic intelligence/rudimentary-consciousness, then, of course, we’re right back to the Other Minds (and I suppose also Zombie) question(s). We still will only be able to take a Rylean/Wittgensteinian/Turing approach and say, “Well, it sure does *behave* (or *act*, if you prefer) intelligent, and *seems* to have some sort of operational awareness of its environment…” etc., etc.

    Now if we brain-link with the thing, then is that (“merely”) IA, OR is it a real “Vulcan Mind Meld” (as it were…)? Well, that question is a bit tough, but as the entity itself grows (self-augements/enhances), I should expect that we ourselves will tend more & more to think in terms of the latter (Mind Meld) rather than in terms of (“mere”) IA, even “robust” or “powerful” IA.

    And Singularitarian pursuit of Friendly Artifical General Intelligence (which has the slightly unfortunate 6th-grade-boys’ humor/tease/snicker-susceptible acromym, FAGI) is certainly NOT “religious” or at all similar to LaHaye-esque Rapture-crap. As Michael, and Mitch Howe’s superb contributions, have made clear: It is a strategy to cope with super-exponentially-accelerating tech-progress, to cope with our naked-ape meat brains, which, while ***so far*** have indeed been the Shakespearean “paragon” of the cosmos, are still—let’s be candid—potentially murderous & psychopathic. That Doctorow—who’s obviously [from his fiction works] a reasonably bright fellow—fails to even fully grasp (much less fully **appreciate**) all this is a combination of sad & astounding—which is to say, APPALLING! (Cory, please take note! You’re a good kid…*get with the program* [wink])

    I’m outta here…Ciao for now…

  24. If you want Cory to read this, you have to do this.

  25. that is…link to his site.

  26. Rather than tackling the really hard problems – poverty, war, hatred, poor infrastructure, mental and physical illness – at our present level of intelligence, Singularitarians advise pursuing intelligence enhancement and then applying qualitatively smarter intelligence to these age-old problems.

    Do you really think that poverty, in particular (not to ignore the others here) is a hard problem that is unsolved because we’re not smart enough?

    I wouldn’t discount the possibility that some application of greater insight might make the costs of alleviating poverty easier to persuade the world to pay. But the implicit temptation of singularitarianism is to depend upon greater intelligence to solve problems that are not going unsolved for the lack of intellect, but for the lack of WILL. We already KNOW how to solve most poverty; the problem is not in being smart enough, but in being motivated enough. The fact is that the people who can don’t care enough. The problem isn’t poverty – it’s apathy.

    So, ok. Now that I think about it some more, is the point of getting a superhuman intelligence to “solve” these problems really more the task of motivating us to solve them ourselves? Is it just inventing something smart enough to devise and launch the ultimate philanthropic marketing campaign?

    I think more of the point Doctorow was making was not that it is dogmatic to suspect that these things might happen, but to expect that they will. The problem with optimism is that it calms people to the point complacency. If good things come of emerging technologies, it won’t be because people let it happen; it will be because they MADE it happen. That means having less faith, whether in the form of optimism – “it’ll work out fine” – OR pessimism – “We’re all screwed”.

    There is a parallel between advocacy and dogma, I think, because religious dogma is an expression of desire in the form of wanting to believe some assertion, unconditionally. Advocacy is similar, in that it’s the expression of desire as well, although it may be in the form wanting to persuade people to act, rather than simply believe (which would be to convince, not to persuade).

    A rational approach to singularitarianism predicates it’s advocacy on the likelihood that its predictions are likely, not on the sheer desire to believe positive outcomes are unavoidable. As we discover evidence that the outcomes are less likely or impossible, are advocacy or enthusiasm for them should flag in proportion. Otherwise, it becomes a religious zeal.

    If that is the approach you take toward singularitarianism, then Doctorow’s not talking about you, and I would think you’d have no problem agreeing with him as he criticizes the actual dogmatic singularitarian straw man.

  27. yeesh. Please excuse my typos.

  28. If we were smart enough, poverty would be solved despite any political barriers.

    I don’t think poverty hasn’t been solved due to lack of will but lack of money. Solving poverty with current technology would be incredibly expensive.

    We can be motivated all we want but practical barriers make it incredibly difficult to succeed at this and maintain our own economy. Millions of people would need to give up their day jobs to go to remote areas of the world to teach people to use technologies to stop poverty.

    Superintelligence can solve the problem with no help from us. The point of superintelligence is not just to encourage us to do it.

    I do expect superintelligence will happen if we don’t blow ourselves up. And I also expect superintelligence would be incredibly powerful, almost magical, like human technology is magical to apes.

    What we will make happen is either human-friendly superintelligence or human-indifferent superintelligence. From then on, we may have some control over how things go, which will depend on how much the human-friendly superintelligence listens to us or depends on our approval to take actions.

    One should never believe any assertion unconditionally… I agree with your last three paragraphs. I do think that Doctorow, and basically all critics of singularitarianism, are attacking straw men, and don’t actually know any singularitarians themselves, or really even know precisely what a singularitarian is supposed to be. (Except for maybe a minority.)

  29. If we were smart enough, poverty would be solved despite any political barriers.

    That’s rather unfalsifiable, don’t you think? If no amount of intelligence could help us with this, how would we know?

    How are we supposed to distinguish the claim that superintelligence will solve poverty by some yet-unknown-by-definition means from the claim that someday, Jesus shall return! After all, I can never be proven wrong.

    I hate to say it, but I think I’m starting to see some of what Cory was talking about.

    I don’t think poverty hasn’t been solved due to lack of will but lack of money. Solving poverty with current technology would be incredibly expensive.

    Before I knew you were the one posting this reply, I was laughing my ass off. Thank you for the dose of obvious!

    Poverty *IS* a lack of money! And saying that redistributing money is “expensive” is only true 1) if you’re the one paying for it, and 2) if it represents a significant fraction of wealth that would otherwise remain concentrated.

    We HAVE the money to end poverty. Here and now. The problem is that those who have most of it want to keep it. This is not news.

    We can be motivated all we want but practical barriers make it incredibly difficult to succeed at this and maintain our own economy.

    No arguments. Of course we need to change our economy to solve poverty. “Our” economy (and I presume you’re talking about the US) is and has been utterly dependent on exploiting the inequality of the rest of the world.

    You have a very technology-obsessed sense of what it would take to do this. If they had the money, they could attract the educational resources necessary to learn to use the technology. Having people “go to remote areas of the world to teach people to use technologies to stop poverty” would BE your day job.

  30. bud

    On the religion front, the idea of Freindly AI seems very similar to the communist paradise (in which we solve all the world’s problems by emancipating ourselves from our own wants and desires and work for the ‘common good’). The only differance is that Marx puts forward the idea of a community of individuals working for each other, you argue for a single machine to work for us to create a paradise where ‘everyone gives according to their skill and takes according to their needs’

    The issue comes when this noble idea is corrupted (??) (the great terror for example).

    michael, you do come accross to (an impartial observer) as a ‘high priest’ for the idea of AI and the singularity, and all I can hope and pray for is that this comment does not have me executed for a thought crime in the future..

  31. JohnSmart

    Hi Michael,
    Just stumbled on this post today. You state:
    “The only Singularity advocate I can think of that believes the Singularity would come about on its own, as a developmentally predetermined inevitability, is John Smart of Acceleration Watch. Otherwise, support of routes to smarter intelligence is a pro-active thing,”

    Just a small clarification for you. Developmental processes in biological systems, and perhaps also in the universe, if the analogy holds, are only statistically inevitable given the appropriate environment, and individual developmental failures occur all the time even in great environments, though it is true that they seem much less frequent in more developed than less developed organisms, for example.

    So though I believe universal constraints seem likely to be pushing all evolutionarily unique intelligent civilizations toward the singularity as a developmental event, I don’t think the singularity is at all assured in any individual case. It seems likely to be be high probability transition for any sample of civilizations, perhaps precisely because so many intelligent organisms are likely to be impelled to ask these questions and to be activists.

    Among the questions we might ask in relation to development would be whether the recognition of universal developmental processes would finally move us beyond the social relativism that may be a consequence of our low level of understanding of information processing in the universe, to an understanding of universal morality in complex systems, and a quantitative and theoretical understanding of what actually constitutes universal, social, organizationa, and individual progress.

    At this point it should be clear that the the “otherwise” statement you make isn’t the way I view things. You might want to change that second sentence as a result, as it implies I don’t see the need for activism in such a transition. I think we need serious activism in both the understanding and management of accelerating change if we are to maximize our chances of a successful tech singularity, and beyond. Without a good understanding of what is going on, and activism behind the most useful and resilient strategies, we may have a seriously lower chance of making the transition as we are just one evolutionarily unique case of what seems likely to me to be very many civilizations in the universe.

    Warm Regards,

    JS

  32. We can all say that people from various ages have experienced the problems of acne. This skin problem can affect any age group with various degrees. Usually …

  33. I just like the valuable information you provide in your articles. I will bookmark your own weblog along with check yet again here frequently. I’m very sure I will learn lots of new stuff in this article! Good luck for that next!

  34. English is not my first language.I really wanted to make a word to be able to express gratitude to you for all of the remarkable facts you are posting on this website. My extensive internet search has finally been rewarded with wonderful insight to go over with my pals. I ‘d declare that we site visitors actually are very fortunate to be in a magnificent community with so many outstanding professionals with good tips. I feel pretty grateful to have discovered the web pages and look forward to many more thrilling minutes reading here. Thanks once more for all the details.

  35. English not my native language.I truly appreciate this post. I’ve been looking all over for this! Thank goodness I found it on Google. You’ve made my day! Thanks again

  36. hi!,I really like your writing very much! proportion we keep in touch extra about your post on AOL? I require a specialist on this area to resolve my problem. Maybe that’s you! Taking a look forward to see you.

  37. Looking forward to reading through much more. Great post. Fantastic.

  38. These inspiring ideas likewise acted like a fantastic way to know that the rest have the same dreams really like my personal own to see a whole lot more concerning this problem. I’m sure there are thousands of more enjoyable times in the future for many who check out your blog.

  39. Hey There. I found your blog using msn. This is an extremely well written article. Iíll make sure to bookmark it and return to read more of your useful info. Thanks for the post. Iíll certainly comeback.

  40. This is something I truly need to try and do a great deal of analysis into, thanks for the post

  41. Very efficiently written post. It will be useful to anyone who utilizes it, including myself. Keep up the good work – looking forward to more posts.

  42. Thank you for helping out, great info. “Those who restrain desire, do so because theirs is weak enough to be restrained.” by William Blake.

  43. uk bingo bets

    Good post. I discover something far more challenging on distinct blogs everyday. Most commonly it truly is stimulating to see content material off their writers and exercise a little at their store. I’d would prefer to use some while using content in this small weblog whether you do not mind. Natually I’ll offer you a link for your internet weblog. Many thanks sharing.

  44. Well said. You’re aware of, It’s my job to like coming here and reading your stuff.

  45. I can not seriously assistance but admire your blog, your weblog is so adorable and good ,

  46. This is the right web site for anyone who would like to find out about this topic. You know so much its almost tough to argue with you (not that I really will need to…HaHa). You certainly put a new spin on a topic that has been discussed for a long time. Great stuff, just wonderful!

  47. I couldn’t currently have asked for an even better weblog. You might be there to provide exceptional assistance, going on for the point for quick understanding of the readership. You are actually a terrific professional in this matter. Thanks lots for always being there visitors like me.

  48. Do you have got a spam situation on this blog; I also am a blogger, and I was wanting to know your circumstance; many of us have developed some good methods and we are wanting to exchange strategies with other folks, why not shoot me an e-mail if interested.

  49. Thanks for the fantastic post in your blog, it seriously gives me an insight on this subject.~”:~:

  50. I couldn’t have definitely asked for a far more rewarding blog. You are there to give fantastic suggestions, going directly towards the point for basic understanding of the target audience. You’re certainly a terrific expert within this topic. Thank you for remaining there human beings like me.

  51. I’ve been exploring for a little for any high-quality articles or blog posts on this kind of area . Exploring in Yahoo I eventually stumbled upon this site. Studying this info So i’m satisfied to convey that I’ve a very good uncanny feeling I found out just what I needed. I so much unquestionably will make sure to do not disregard this web site and provides it a glance on a relentless basis.

  52. This is a great tip especially to those new to the blogosphere. Simple but very precise information… Thanks for sharing this one. A must read post!

  53. I gotta bookmark this internet site it seems very useful very helpful

  54. I’d must examine with you here. Which is not something I normally do! I get pleasure from reading a put up that can make people think. Additionally, thanks for allowing me to comment!

  55. Very interesting info !Perfect just what I was searching for! “The only gift is a portion of thyself.” by Ralph Waldo Emerson.

  56. I like what you guys are up too. Such smart work and reporting! Keep up the superb works guys I have incorporated you guys to my blogroll. I think it will improve the value of my web site :).

  57. I have been surfing online greater than three hours nowadays, but I by no means discovered any interesting article like yours. It’s beautiful worth sufficient for me. In my opinion, if all website owners and bloggers made excellent content material as you did, the internet can be a lot more helpful than ever before.

  58. Dead pent content material, thanks for information. “Life is God’s novel. Let him write it.” by Isaac Bashevis Singer.

  59. After examine a number of of the blog posts on your web site now, and I actually like your means of blogging. I bookmarked it to my bookmark web site listing and will probably be checking back soon. Pls take a look at my web site as nicely and let me know what you think.

  60. Hi! Someone in my Myspace group shared this site with us so I came to take a look. I’m definitely enjoying the information. I’m bookmarking and will be tweeting this to my followers! Outstanding blog and brilliant style and design.

  61. I have been exploring for a little for any high quality articles or blog posts on this sort of area . Exploring in Yahoo I at last stumbled upon this website. Reading this info So i am happy to convey that I have a very good uncanny feeling I discovered just what I needed. I most certainly will make certain to do not forget this web site and give it a glance on a constant basis.

  62. Thanks for this glorious article. One other thing is that almost all digital cameras arrive equipped with a zoom lens that permits more or less of a scene being included by way of ‘zooming’ in and out. These kind of changes in the aim length tend to be reflected inside the viewfinder and on significant display screen right at the back of this camera.

  63. With havin so much content and articles do you ever run into any issues of plagorism or copyright infringement? My website has a lot of exclusive content I’ve either written myself or outsourced but it appears a lot of it is popping it up all over the web without my permission. Do you know any methods to help reduce content from being stolen? I’d genuinely appreciate it.

  64. Can I just say what a reduction to find someone who truly is aware of what theyre speaking about on the internet. You definitely know tips on how to convey a difficulty to mild and make it important. Extra individuals need to read this and understand this facet of the story. I cant consider youre no more common because you undoubtedly have the gift.more tips i found on!

  65. I just could not depart your site prior to suggesting that I really enjoyed the standard information a person provide for your visitors? Is going to be back often to check up on new posts

  66. Thanks for your write-up. What I want to comment on is that when you are evaluating a good online electronics go shopping, look for a web-site with full information on critical factors such as the level of privacy statement, protection details, any payment methods, and also other terms plus policies. Generally take time to look into the help and also FAQ pieces to get a far better idea of what sort of shop will work, what they are able to do for you, and how you can make best use of the features.

  67. It is truly a great and helpful piece of information. I’m glad that you shared this helpful info with us. Please keep us informed like this. Thanks for sharing.

  68. Excellent post. I was checking continuously this blog and I’m inspired! Extremely helpful information particularly the closing part :) I care for such info much. I was seeking this particular information for a very long time. Thanks and good luck.

  69. This web site is mostly a walk-by way of for all the information you wanted about this and didn’t know who to ask. Glimpse right here, and also you’ll definitely discover it.

  70. Howdy! I simply would like to give an enormous thumbs up for the nice information you have here on this post. I might be coming again to your blog for more soon.Useful info!

  71. Thanks for the interesting things you have discovered in your blog post. One thing I’d like to comment on is that FSBO associations are built as time passes. By introducing yourself to owners the first end of the week their FSBO is announced, prior to the masses start out calling on Monday, you generate a good network. By sending them methods, educational supplies, free reports, and forms, you become an ally. By taking a personal affinity for them plus their situation, you make a solid link that, oftentimes, pays off in the event the owners opt with a broker they know and trust – preferably you.

  72. What i do not realize is actually how you’re not really much more well-liked than you might be now. You are so intelligent. You realize therefore considerably relating to this subject, produced me personally consider it from a lot of varied angles. Its like women and men aren’t fascinated unless it is one thing to accomplish with Lady gaga! Your own stuffs outstanding. Always maintain it up!

  73. I’ve learn some just right stuff here. Certainly value bookmarking for revisiting. I surprise how a lot attempt you set to make this sort of magnificent informative site.

  74. Thanks for another magnificent post. Where else could anyone get that kind of information in such an ideal way of writing? I have a presentation next week, and I’m on the look for such information.

  75. I am typically to blogging and i actually respect your content. The article has really peaks my interest. I’m going to bookmark your website and maintain checking for brand new information.some tips here!

  76. Appreciate you for sharing all these wonderful discussions. In addition, an excellent travel as well as medical insurance plan can often ease those concerns that come with visiting abroad. Some sort of medical crisis can rapidly become expensive and that’s sure to quickly set a financial weight on the family’s finances. Having in place the perfect travel insurance package prior to setting off is worth the time and effort. Thank you

  77. My wife and i felt really delighted that John could finish up his web research because of the precious recommendations he gained through the weblog. It is now and again perplexing to simply find yourself giving freely techniques which usually other people might have been selling. And we also discover we have the writer to be grateful to for this. The main explanations you’ve made, the simple website navigation, the relationships you assist to create – it’s got everything astounding, and it’s really helping our son in addition to us reason why that concept is brilliant, which is certainly really essential. Thank you for all!

  78. Thanks for giving your ideas. The one thing is that individuals have a choice between national student loan and also a private student loan where it is easier to go for student loan consolidation than through the federal education loan.

  79. I am curious to find out what blog platform you happen to be working with? I’m experiencing some minor security problems with my latest website and I would like to find something more secure. Do you have any suggestions?

  80. You made some first rate points there. I seemed on the web for the issue and found most individuals will go along with with your website.more tips i found on!

  81. When I originally commented I clicked the -Notify me when new feedback are added- checkbox and now every time a remark is added I get four emails with the same comment. Is there any manner you may take away me from that service? Thanks!

  82. Hiya! I simply want to give an enormous thumbs up for the nice information you have got right here on this post. I will likely be coming back to your blog for more soon.Useful info!

  83. Perfectly indited subject material , thankyou for entropy.

  84. I’m just commenting to make you understand of the beneficial encounter my friend’s daughter had checking your blog. She figured out a lot of details, with the inclusion of what it’s like to possess a very effective helping nature to get others with no trouble fully understand a variety of multifaceted matters. You actually exceeded my expected results. Thank you for giving the essential, safe, informative and even easy tips about that topic to Ethel.

  85. Nice post. I was checking continuously this blog and
    I am impressed! Very useful information specifically the last part
    :) I care for such information much. I was looking for this
    certain info for a very long time. Thank you and good luck.

  86. Thank you a lot for sharing this with all folks you really know what you’re talking about! Bookmarked. Kindly additionally discuss with my site =). We may have a link trade contract between us!

  87. I have been exploring for a bit for any high quality articles or blog posts on this sort of area . Exploring in Yahoo I at last stumbled upon this web site. Reading this information So i am glad to convey that I have an incredibly excellent uncanny feeling I found out just what I needed. I such a lot for sure will make certain to do not put out of your mind this web site and provides it a look on a constant basis.

  88. Thanks for any other informative web site. Where else may I am getting that kind of info written in such a perfect method? I’ve a venture that I am simply now operating on, and I’ve been at the look out for such information.

  89. I want to read some of the comments here more deeply later and give my responses, but for now I will just say:
    1) Let us get some human and/or Bayesian spam filter (one of those everyday miracles ;D ) moderation going. Spam makes kittens cry.
    2) Poverty is less a lack of money, and more a lack of access to money, or rather, resources. Whether or not you agree with Marx’ recommendations, what he definitely did accomplish is providing terms and concepts in a clear manner for debate, and one of them is the “means of production” (not that I am saying all the terms associated with Marx were his originally, but he popularised many). Whether or not you have money, if you do not have a job to get money, or a way to make the things you need, you will always be dependent on others to provide food, shelter, and so on in exchange for the little money you have at any one time (which might only be manifested as your man-hours of unskilled labour). Basically, like technology, wealth is an influence multiplier, so that those in poverty have reduced influence relatively speaking. The flip side is that, given some control over the means of production of things to sustain them, people can give significant assistance to a given effort, and if that effort is reducing poverty then we essentially have a method of bootstrapping prosperity (the term bootstrapping comes from a term referring to poverty, “pull oneself up by one’s bootstraps” which is a ridiculous recommendation for separate reasons). Thus it would not be as resource-intensive to eliminate the negative influences of poverty (in part; crime, hunger, war) as one might think, but as has been said above those who are wealthy prefer to keep others in a state not so threatening to their continued luxury. This is not something that will be improved by more intelligence, at least in the way it is commonly defined (which is itself problematic, but that is another issue).

  90. I intended to write you this very small note to finally say thanks a lot the moment again for all the beautiful techniques you have featured on this page. It has been really strangely generous of people like you to convey unreservedly precisely what most of us would have marketed for an e book to earn some profit on their own, chiefly seeing that you could have done it if you considered necessary. The things additionally worked to provide a good way to be sure that many people have a similar eagerness just as mine to understand a whole lot more with regard to this matter. Certainly there are a lot more enjoyable periods up front for those who read carefully your blog.

Trackbacks for this post

  1. Accelerating Future » Introducing the Singularity: Three Major Schools of Thought
  2. Accelerating Future » John Smart on the Relevance of Activism to the Singularity

Leave a Comment

Awesome! You've decided to leave a comment. Please keep in mind that comments are moderated.

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>