Dudley Lynch on the Singularity Saturday, Nov 28 2009
futurism and singularity 2:01 pm
Dudley Lynch, a self-described “non-scientific observer of what’s being said and written about The Singularity at the moment”, has written up an article on the Singularity. Conclusion: “I suspect it’s still going to be awhile before anyone has an idea about The Singularity worth keeping.”
I get a cameo in his write-up:
Michael Anissimov of the Singularity Institute for Artificial Intelligence and one of the movement’s most articulate voices, continues to warn that “a singleton, a Maximillian, an unrivaled superintelligence, a transcending upload”—you name it—could arrive very quickly and covertly.
Let me add a qualification to that. I do not think that such an entity could arrive quickly and covertly starting from today as a reference point, unless there are extremely well-funded secret projects that have already been working with brilliant researchers and theoreticians for maybe a decade or more (not likely at all). The point I keep making is just that an entity could go quickly from slightly human-surpassing intelligence to superintelligence, a concept known as a “hard takeoff”. To get from here to slightly human-surpassing intelligence could take a while, probably more than 10 years but less than 40 (but who knows), and a project with an annual budget in the millions (maybe tens of millions but probably not hundreds of millions, is my guess). The brain is not magic and we are learning a tremendous amount about it all the time.
I especially stress this point with respect to AI. Even “merely” human-equivalent AI would have a tremendous number of advantages over human thinkers — the ability to copy itself, absorb information more readily, customize and overclock its cognitive modules, design new cognitive modules from scratch, accelerate its thinking speed, avoid the empirically demonstrated biases in reasoning that afflict all humans, explore the entire state space of cognitive features that evolution didn’t think of, blend together deliberative and autonomous cognitive processes, create multiple spheres of attention, and much more. Many of these features are listed in part 3 of “Levels of Organization in General Intelligence”, a Singularity Institute paper.
When us Singularitarians say that an intelligence could potentially bootstrap itself very rapidly from just-barely-smarter-than-human to much-much-smarter-than-human relatively quickly, our reasons aren’t “magic” or “it sounds cool”. We have scientific and rational reasons, it’s just that they don’t fit into soundbites, and there are few people articulate enough to present the arguments in an accessible way.
I don’t personally buy into Kurzweil’s 2029 date — it’s very speculative. The key point is that intelligence operates based on principles and rules that will eventually be reverse-engineered, and once we understand those principles, we’ll have the ability to “teach a rock to think”, to paraphrase Michael Vassar. The ability to teach a rock to think would be no small thing — it could transform the world practically overnight.
Mr. Lynch, here are two ideas about the Singularity worth keeping — one, that artificial intelligences will not behave anthropomorphically, and two, advanced artificial intelligence will be a risk even if we do not program them malignly.
Michael,
I appreciate your deliberate and carefully approach to this issue. I did so in materials you’ve written that I read earlier, and do so with regards to the above.
This caution isn’t always shared by many who comment on “The Singularity.”
That explains the couple of points I sought to make with my blog item: 1) This whole subject is very entertaining, especially to the kinds of minds (often young, male, especially computer scientifically inclined and their non-geeky futuristically inclined, sometimes New Age-y-like hangers-on) who enjoy the titillation of esoteric and and the thrill of legitimately horrific and 2) This whole subject is so currently undocumented, unproven and unpredictable that one can say almost anything that might sound plausible in a good science fiction novel and be taken seriously by many of the persons cited in No. 1.
Good on you for being one of those willing to call a Kurzweilian-type speculation a Kurzweilian-type speculation.
Dudley Lynch
AIs might well behave like people if they come from reverse-engineered human brains. As described by Kurzweil, an electronic copy of a person would initially be identical to the original except for the ability to think a million times faster. While Michael and company don’t care for this model, the possibility shouldn’t be dismissed. If anything, it’s more speculative to believe we can create human-level or superhuman intelligence on an different foundation. To me, both alien and familiar AGIs seem plausible. I wouldn’t be surprised if both types emerge and exist side by side.
When was the last time a transhumanist said something new? A decade ago? Two? Seriously, you could copy-paste your blog from the archives of some transhumanist forum/blog and no one would notice.
I think it would be a good idea to review what has been said, write an exhaustive FAQ and be done with it. Keep the internets clean, you know. Just link to it. Saves you and everyone else time.
Just keep on researching the sooner-than-you-think ultra technologies until you have something genuinely new to say, and then say it, once, and add it to the FAQ.
It’s not news just because it’s about the future.
Hi Cranky,
I have to repeat myself because the field is gaining a huge amount of new attention and 95% of the readers aren’t even familiar with the arguments I consider most important, or even what everyone would agree are the basics. People don’t read FAQs as much as blogs, by a long shot. Nearly everyone new to the field thinks that Kurzweil’s arguments completely represent the beliefs of all transhumanists, when most disagree with him vigorously. Very few people are bothering to rectify this, but the continued survival of humanity may depend on it.
A significant amount of this blog contains new content, but it’s not supposed to be a news blog anyway.
If you think transhumanists never say anything new, then maybe you aren’t being exposed to new material, like the WBE roadmap, the talks at Singularity Summit, or a lot of other things.
If you were copying/pasting old stuff, it would have to be my stuff, because lots of the material I articulate is only discussed by a few other people. Some of it is completely unique, in fact. If more other people demonstrated understanding of the scientific underpinnings of my arguments, I would repeat myself less. I frequently link long papers that I really think get at the essence of the arguments, but practically no one reads them. This serves as “gateway material” sometimes.
People require new material to pay attention to. The front page of this site and a dozen other small pages get about 20 times more views than all the rest put together. Accelerating Future probably gets 50 times more views than the Transhumanist FAQ. Blogs and books are the two primary games in town, I’m afraid.
Did you read my answers to PopSci’s 10 Singularity questions? Many of these questions were very basic, and the way that they need to be asked shows that most people are not familiar with the arguments out there.
Have you read the Overcoming Bias/Less Wrong sequences by Eliezer Yudkowsky? That’s all novel material, but only a minor community reads it.
Most tech blogs all echo the same tech news. I’m not interested in being part of an echo chamber. I especially use the same arguments when I know I am talking to someone who is being exposed to the ideas for the first or almost the first time, like Mr. Lynch here. It’s important to me to reach out to them.
By the way, I am writing a FAQ. If just anyone could write one, there would be numerous other FAQs besides just the Transhumanist FAQ.
Looking just on the front page, my criticism of Gardner’s multiple intelligences theory is new. Listing the AI advantages explicitly is not new but it’s fertile ground for investigation that few people appreciate. The arguments for recursive self-improvement are also fertile ground that few people (including critics) understand. (You have to understand something on some level of detail to criticize it. For instance, I’ve bothered to understand Christianity, Scientology, Mormonism, American bioconservatism, the Web 2.0 craze crowd, and several other groups and doctrines well enough to zoom in much deeper than most critics. The number of people who understand the recursive self-improvement arguments well enough to criticize them can be counted on my fingers and toes.) SENS rallying is not new but it is important. Discussing whether transhumanists are resentful is relatively new and unique. Linking articulate transhumanist critics is relatively new. Pointing out that large-scale point neuron simulations are nowhere near brain emulations is new or at least rare. Addressing causal functionalism with other leading bloggers is not new but it is relevant. Atheism is not new but I also consider it important.
This blog infrequently takes up more than 45 minutes of my daily time anyway. I do many other things as well, like organizing 800-person conferences on the Singularity, helping SIAI staff with research projects, and helping the President of SIAI with administrative tasks.
[quote]an intelligence could potentially bootstrap itself very rapidly from just-barely-smarter-than-human to much-much-smarter-than-human relatively quickly[/quote]
An intelligence much bellow the human level, but of a certain kind could also bootstrap itself to much-much-smarter-than-humanity relatively quickly, too!
And that is going to be, once upon the coming time.
Dudley, what kind of documentation and proof are you looking for? We’re talking about the future. Nobody can document 2029 or prove what will happen that year. However, futurists can and have documented the historical trend of accelerating technological advancement and the transformative power of developments such the emergence of human intelligence, the industrial revolution, and the transistor. This data forms the basis for projections and predictions.
“2) This whole subject is so currently undocumented, unproven and unpredictable that one can say almost anything that might sound plausible in a good science fiction novel and be taken seriously by many of the persons cited in No. 1.”
If you stop and think about the nature of mans’ modern life for a few minutes, it rapidly becomes apparent how much of societies is manufactured based on the musings of past futurists. Airlines, national road networks, electrical infrastructure, water and sewage piping to every household, cable tv, computers and the internet, ….
The flip side is the complete disconnect. Our lives today are so removed from what we were selected for in evolutionary terms that we flounder ( except in several very important instances like nuclear annihilation).
The statement made about undocumented and unpredictable assumes some sort of basis in reality of the current state, but this is another fallacy, a construct of past thinkers and present adherents. Futurist postulation is just as valid, and at least has a hope of rectifying issues within current frameworks.
The ‘rapture’ like conceptualization that many Singularitarians succumb to is a danger however, with an imagined super-abundance just around the corner; hedge your bets, because the event space is far larger than the singularity outcome space.
Michael, I find your blog more informative than Less Wrong, Overcoming Bias, and the AGI mailing list combined. Please keep it coming!
In order for this not to get too long, I’ll jump to the two last and most important points:
one, that artificial intelligences will not behave anthropomorphically
I suppose how that thing behaves will critically depend upon its architecture about which (surprise!) we have no idea yet. So in the end we can conclude that, as long as we don’t know how to design an AI we will not even remotely know how it will act, apart from the fact that it will act intelligently. Which is the whole point of the exercise, but still doesn’t give us a clue, since intelligence itself is a little ill defined in the sense of hazy.
artificial intelligence will be a risk even if we do not program them malignly.
Yes they will be. They might come at us, like a force of nature, like a thunderstorm, a flood or an earthquake.
Which is a nice analogy: We don’t know whether the kind of danger an AI will present will be a flood, an earthquake or an asteroid, since, as mentioned in point one, we don’t know what the hell it might do. So we have no idea whether to think about designing flood shelters, stable buildings or something to nuke asteroids with.
Which makes thinking about the whole topic kind of moot…
MyReality, nicely put! It is my current belief that the singularity and destruction are the two long-term attractors in the outcome space, and come to dominate the landscape given a long enough time.
Ricky, that’s very complementary! Thank you. I don’t read the AGI mailing list anymore but I enjoy it and LW/OB a lot.
Wolfgang, I think there is an element of predictability. For instance, AIs would be motivated to preserve their utility function. Remember that the people who create AI will get to create it as they desire, because they’re the ones who will be inputting all the code, so they’ll certainly be able to predict things. By looking at certain concepts favored in the field as a whole (expected utility, causal chaining, machine learning) we can also make vague guesses about what an AI might look like if it were built by programmer A or programmer B, though certainly there is a lot of uncertainty at the present time.