Accelerating Future Transhumanism, AI, nanotech, the Singularity, and extinction risk.

More Singularity Curmudgeonry from John Horgan

John Horgan goes on the offensive against the Singularity concept on his relatively new blog at SciAm.

My own skepticism is based on simple comparisons of Kurzweil's claims with what is actually happening in science. For example, Kurzweil contends that reverse-engineering the brain isn't that big a deal. "The brain is at least 100 million times simpler than it appears because the design is in the genome," he wrote on the blog Posthumans. "The compressed genome is only about 50 million bytes," which is "a level of complexity we can handle."

I agree with John that this estimate of the difficulty of AI is an oversimplification. It carries the assumption that AI will be a copy of the human brain, which isn't necessarily true. It also ignores the complexity of the process of neurogenesis and continued development. The real brain is much, much more complex than the portion of the genome that codes for it, and it probably won't be until after the Singularity until we understand the details of how the brain is created from the genetic code.

Is it really so far-fetched to believe that we will eventually uncover the principles that make intelligence work and implement them in a machine, just like we have reverse-engineered our own versions of the particularly useful features of natural objects, like horses and spinnerets? News flash: the human brain is a natural object.

I think Kurzweil is wrong and overconfident on a lot of specific points, but I appreciate his overall vision.

The World the Singularity Creates Could Destroy All Value

From the letters to the editor section of The New York Times...

Sizing Up the Singularity

To the Editor:

Re “Merely Human? So Yesterday” (June 13), which described the Singularity movement and how it envisions a world of mind merging with machine, to conquer disease and even old age:

But if the movement succeeds beyond its wildest dreams, the world it would create could destroy all that we are that is of true value: our suffering and thus our joy, against which we determine the value of everything else.

Achilles, as depicted in the movie “Troy,” put it well: “I’ll tell you a secret. Something they don’t teach you in your temple. The Gods envy us. They envy us because we’re mortal, because any moment might be our last. Everything is more beautiful because we’re doomed. You will never be lovelier than you are now. We will never be here again.”

Let us never stop striving for us, but let us do it as us so that we still have reason for striving.

Ryan Andrews

Columbia, Mo., June 13

In one sense, I agree with this commenter. The Singularity could lead to a world where mankind is snuffed out by machines, or drives itself into poverty due to falling wages triggered by rapid upload copying, or our complex values are sidelined by mindless replicators, or we wirehead ourselves into oblivion. The Kurzweilian facet that argues "everything will be fine because we will merge seamlessly with our creations" is mistaken. Everything may not be fine. Whether or not the future turns out alright will depend on the actions taken between today and the Singularity. After the Singularity, it will become impossible to rewind, so we had better get it right the first time.

Of course, I disagree with the notion that life is only beautiful because we're doomed, as the commenter quotes. This view is the result of a memetic lineage that has taught us to cope with suffering by embracing it in a Stockholm Syndrome fashion.

The Singularity movement is no place for uncritical, facile technophilia. Instead of inevitable ascension to cyber heaven, the Singularity can be more accurately viewed as the arrival of a swarm of new superintelligent alien species, albeit aliens that we will create with our own hands, at least initially. Without extremely careful engineering (not "business as usual" -- profit motive), these aliens might not care for us much. Our efforts to augment ourselves to be more like them could prove hopeless at first, because of the great complexity and expense inherent in any prototype experimentation. These transhumans, whether human-derived or AI, could have a massive impact on society (perhaps taking over the planet) before the rest of mankind catches up with them, if they even let us.

That's the reason the Singularity Institute exists -- because we reject blind embrace of increasing technological power. We must moderate that power with careful choices and the willingness to self-limit to an extent.

Salim Ismali/Singularity University Response to Times Article

You can read it here. The gist is that Singularity University is more broadly focused than the article implies. (Some people in the comments were wondering what fast-fabrication of houses has to do with the Singularity, and the answer is that Kurzweil's definition of the Singularity is so broad that it can encompass practically anything.) Since that is the case, perhaps it would have been more appropriate if the NYT gave more attention to the Singularity Institute, which is more explicitly focused on the Singularity.