Denying Superintelligence Friday, May 25 2007
intelligence 1:03 am
There are quite a few individuals that react to the idea of qualitatively smarter-than-human intelligence, AI or otherwise, with extreme skepticism and derision. My guess is that there are four possible reasons for this, which different people display in different combinations and intensity levels.
The first is the folk theory that intelligence is a light bulb - either it’s on, or it’s off. No in between. If you have it, it only varies to a matter of degree, not qualitatively. Humans have intelligence and animals don’t, which is why it’s okay to raise animals for food, for instance. Intelligence and subjective consciousness go hand in hand.
The second is the argument from divine privilege. Man, being made in God’s image, has been given the gift of reason. We cannot magnify this gift on our own any more than we can engineer a machine that turns us into angels. This “gift of reason” argument is what I was taught by my parents as a child.
The third is technological skepticism. For example, my grandfather, who is an atheist, believes it will be centuries before we understand the brain in enough detail to manipulate it significantly. This skepticism derives partially from a linear intuitive view of technological progress, and partly from a pseudo-spiritual worship of brain complexity.
The fourth is outright denial based on fear. Some people associate superintelligence with heartlessness, boring rationality, ruining all the fun, threatening to replace us, and so on. This is primarily based on fictional portrayals. There are dozens of films and books in which superintelligences are the bad guys. Astonishingly, the dumber good guys always seem to triumph in the end.
Can you think of any others?
May 25th, 2007 at 4:13 am
A dualistic approach to the mind / body problem?
May 25th, 2007 at 8:35 am
Lack of intelligence? Some people may simply not be able to fathom the concept of levels of intelligence because they cannot: for whatever reason, their neurologics aren’t powerful enough to support the kind of abstract reasoning that leads to theorizing about superintelligence. I do not mean that they are suffering from any form of disability, genetic, congenital, or otherwise–they simply don’t have the capacity for advanced abstraction.
May 25th, 2007 at 8:37 am
Not quite another, but relevant in its humorousness:
Wile E. Coyote: Genius.
http://www.pioneernet.net/curtis/wile_e/
That right there is part of the problem.
May 25th, 2007 at 10:10 am
The first is the folk theory that intelligence is a light bulb - either it’s on, or it’s off. No in between. If you have it, it only varies to a matter of degree, not qualitatively. Humans have intelligence and animals don’t, which is why it’s okay to raise animals for food, for instance. Intelligence and subjective consciousness go hand in hand.
It’s *not* OK to raise animals for food is exactly what worries me when I consider AI. After all why should AI treat us any better than we treat our animal or even our poor and powerless? The incredibly fast growing gap between the poor and the super rich is in my opinion the thing we must handle with the utmost speed in order to at least give our kind at least a semblance of respectability to visiting aliens or for that matter to AI’s we create ourselves.
May 25th, 2007 at 11:25 am
Good post, Michael. I can’t think of anything to add to it right now.
The fourth point is particularly interesting. I wonder if it’s part of a larger anti-intellectual movement in places like the US. It’s weird that many people, when they think of the super-smart, think more “evil genius” than “benevolent genius” despite the fact that it seems like history is full of the latter.
May 25th, 2007 at 12:47 pm
“The fourth is outright denial based on fear. Some people associate superintelligence with heartlessness, boring rationality, ruining all the fun”
These people should be forced to read some of Iain M Banks’ Culture Novels! The minds have an excellent sense of fun and they usually have a sharp whit.
Seriously though, the underlying problem is that most people respond to life by copying what other people do/say. No-one on the TV is talking about AI, and it’s not a topic of conversation down at the bar either. This means that when people get asked about AI, they have to think for themselves, which, for them, is a frightening and unfamiliar process. They’ll just get angry with you for asking, feel confused and slightly upset, and say the first thing that comes to mind to end the conversation. Don’t pay much attention to what they actually say, they haven’t really thought about it.
Have you actually been asking some people around San Fran?
May 25th, 2007 at 1:09 pm
Anton writes:
“[…] growing gap between the poor and the super rich […]”
I hate to burst your bubble, but the CBO disagrees with you.
May 25th, 2007 at 2:04 pm
“After all why should AI treat us any better than we treat our animal or even our poor and powerless?”
Because we were very careful to design it to. If we designed AI by natural selection, you’re right- it would treat us like animals, or more likely like random blocks of carbon atoms.
May 25th, 2007 at 3:11 pm
As a matter of opinion, I think the greatest reason for the denial is caused by a lifetime of entrainment from our political, educational social and cultural institutions. In the face of daily uncertainty and strife, many people have struggled all of their lives in order to frame a workable - and survivable - worldview. A worldview that may not be particularly accurate. Indeed, it cannot be. The truth - and often the facts - must be abandoned for political, social and career advancements. The justifications for this behavior are what people are loath to face or to abandon. In view of this alone, such topics as the Singularity are threatening. Denial is an automatic and emotive response. (fight or flight)
Whether our beliefs are primarily based upon an evolutionary pov or theological/spiritual beliefs, upon opinions, assumptions and presumptions, or political and social ideologies and agendas, or a ‘mere’ matter of brain entrainment and culture…everything that we think is true - or wish to believe is true - will be changed by the Singularity Event. As a matter of speculation, I think that many people intuitively know this and are threatened by it.
May 25th, 2007 at 3:38 pm
Some atheist skeptics seem, to me, to be what I call “masochistic” atheists. From what you have written Michael, I don’t think your grandfather falls into that category. Rather, what I mean by “masochistic atheists” is that some people take some sort of perverse pleasure in there not being any sort God or heaven. The way these people view the world is that people are basically screwed. Not only do they not believe in God, or heaven, or reincarnation, but they have been taken by a rather trendy view that even man-made progress is a myth. To them, belief in progress is like believing in Zeus, or Apollo, or Thor, or Yahweh.
From what I can tell, this view started on the left-wing of the political spectrum. After the failure of communism, many on the left no longer had a personal stake in the notion of progress. Instead the idea of progress, to them, was only an excuse for imperial endeavors; i.e., the “white man’s burden.” If one accepts that progress is real, then it follows that some groups are more progressive than others. Ask most white liberals in western countries if they view the behavior and lives of their medieval ancestors fondly, most will say, “no.” Ask them if there were hidden patches in Europe where such things like witch trials still happen, what they would do? They would respond, “intervene.” But ask them of their opinion of populations on this Earth who engage in the same sort of practices today, in Africa, or the Middle East, and they will spout-off some cultural relativism mumbo jumbo. The same thing has happened to the extreme environmentalists who see no difference, in terms of progress, between our current world and one with only a tenth of the population and without any modern technology. Sorry to go off on this tangent, I don’t mean to make political waves here, I just wanted to lay out why I think the very idea of progress has become laughable to so many otherwise intelligent people.
Regardless of WHY so many believe progress to be classified in the same category as unicorns and fairies, the damage has already been done. Many non-believers think skepticism and atheism demand that you always take the pessimistic view of any proposal, regardless of evidence. Also, I think there is some kind of psychological protection going on here. If a group of open minded skeptics fully entertain the idea of SI, many will likely not agree with the timing predicted by the likes of Kurzweil. In terms of the grand sweep of human history, 2025 vs. 2525 makes little difference, but would make all the difference to those of us alive today. It would SUCK BIG TIME if you ALMOST made it, better to think you weren’t THAT unlucky to be so close, but yet so far. So, 2525 it is. The same 2025 vs. 2525 thinking holds true if you think SI is bad; it is better to think you, your grandchildren, and their grandchildren will not have to deal with any of this. Check out this Douglas Hofstadter speech between the 2:35 and 6:20 mark for the best example of this:
http://video.google.com/videoplay?docid=8832143373632003914&q=hofstadter
I respect Douglas Hofstadter and Daniel Dennett immensely; I fully subscribe to their reductionist/materialist/functionalist view of philosophy. What they do not understand however, is that all their mental games and thought experiments involving robots who feel pain, and human beings being uploaded one brain cell at a time, will not, if their philosophy is actually correct, forever remain intellectual curiosities used to clobber creationists, vitalists, and assorted mystics in verbal arguments. Rather, they will become real, and whoever is alive at that time will have to actually deal with the consequences of that new reality.
May 25th, 2007 at 4:06 pm
GB2 CFAI. I know that everyone is really busy nowadays and claims not to have the time to read a short book-length, free online document, but you should really do it so you can contribute to the discussion.
Roko, I think you’ve got it, by jove! Philosophers sometimes over-intellectualize the reasons people don’t get their arguments and thereby miss the point. Consider this post as applying to the people smart enough to have at least thought about it personally a little bit.
Some! But as you say, most people get nervous when forced to think about new things for themselves. I’ve had some success with a few people though.
Maybe… more likely is that they DON’T believe it’s possible at all, or that its intensity would be similar to introducing one new Einstein into the world. Thanks for commenting btw.
May 25th, 2007 at 4:11 pm
Gully, great comment! I total agree with you - the notion of progress and the Enlightenment is verboten in some quarters. It’s trendy to not care!
Douglas Hofstadter really disappointed me at Singularity Summit. He didn’t even belong there. I haven’t read his most recent book but I suspect that his time in the spotlight may be through.
May 25th, 2007 at 4:36 pm
Attempts to control the memes and narratives re: the Singularity Event are not going to have any effect - short of absolute global tyranny or a KT Event.
The kitty’s out of the bag. From my own “academic” research, it isn’t a matter of when the Singularity occurs. The question is, “How soon?”
My own research into ‘bleeding edge-tech’ over the last few years has led to the same conclusions I’ve seen in most of the literature about The Singularity. In ten…twenty years? Everything we know or thought we understood is going to change. Any denial of that fact is based either in ignorance or fear …or both.
Transhumanism has already well begun. Post-humanism? Who can say what that will be?
May 26th, 2007 at 2:58 am
Michael wrote:
> GB2 CFAI
Please go back to sane discussion techniques. If you think this document has some specific point relating to my post it is up to *you* to extract the specific quote. What you are trying to pull here comes across as something entirely different from helpfully providing a link for further reading, no, instead it reads more like “hey, until you have read all of this you are unqualified to contribute” which is more than a little arrogant.
By the way thanks for the link, it seems interesting regardless of your attitude problems.
Now to add something to the discussion myself apart from quibbling over discussion techniques I would say that it would make a lot of difference if the world the progammers who create AI live in is considered as just by them or if it is not, for example because of the growing gap between the poor and the rich.
Although someone gave a link I couldn’t open (by the way Michael your link needs some manual editing too) I suppose the argument could be “even the poor are a lot better off now than before”.
This doesn’t help us when the *difference* in income is actually spiraling out of control so that in the end the *whole* world will be the property of a single person or collective.
Don’t you think that it would make a huge difference in the kind of programs that people write if they perceive the world as a “winner takes all” environment versus a more friendly place?
That question should also be something for Tom to think about when he asserts that we will be building AI very carefully.
May 26th, 2007 at 3:34 am
It’s unlikely I’ll ever change.
The Singularity is the most important issue in the world, more important than billions of human lives. It determines humanity’s entire future. We owe it to ourselves to at least read Creating Friendly AI. Sometimes when I respond to comments on this blog it’s the end of a long day, and I’m a little bitchy. I apologize.
However, people with high familiarity with the subject matter have every right to be arrogant. Read the literature, and you can be arrogant too.
This will be invalidated in less than 20 years thanks to molecular manufacturing. “Poor” is irrelevant. “Rich” is irrelevant. We’re all poor. None of us can create 100-story sapphire castles. If we survive the Singularity, all of us will be able to. The only thing money is good for today is to be put towards the Singularity or MNT.
If you don’t know what molecular nanotechnology is, read up on it.
All AGI programmers I know view the world as a positive-sum place, because they aren’t stupid.
May 26th, 2007 at 7:57 am
That statement makes sense to me. Currently, the only long-term winning strategies are “Nice with Retaliation” and “Tit for Tat with Forgiveness.” IMO, after The Singularity? The only strategy that will work will be “Nice.”
After The Singularity, we will be so far beyond any current meme and paradigm of understanding that none of our old narratives will apply. All of our needs will be met and we will know and understand all that there is to know. Our current memes, narratives, agendas, ideologies and assumptions? They’ll be more useless - and have less meaning - than tits on a boar.
Design bias may have an impact, but an SAI will learn…very quickly…that losing/self-destructive strategies don’t work.
Just my opinion.
In the meantime…keep the bills paid, practice good hygiene and go to work every day. ;D
May 26th, 2007 at 10:30 am
Whoa, slow down. All of this stuff will still matter, because we’ll make it matter. By the way, the proper quoting tag is “blockquote” between these brackets: <>.
What do you mean?
I worry when people write this. Do you stand behind what you’re saying or not? If so, then say the reasons why.
Not really relevant to this discussion…
May 26th, 2007 at 12:23 pm
Anton writes:
Anton — the CBO link I provided was a link to a study by the Congressional Budget Office; think of them as a “Department of the Financial Census.” According to that research, over the last ten years the rate of growth for the lower and middle classes has been outstripping the rate of growth for the upper class, by a margin of 50%, IIRC. That is to say; the gap of incomes is closing. (Lou Dobbs is a pompous ass, by the way.)
As to the “one person owning the world” sentiment; that is traditional closed-sum economic logic which is the memetic bane that has trapped what in America is called “liberal” and everywhere else in the world, “socialist” thought.
May 26th, 2007 at 12:26 pm
Greg Egan’s apparent reasons for rejecting the Singularity are interesting. It’s part #1 (intelligence is binary), and you can see Eliezer debunk that further down the thread, but there’s also a relative of the “masochistic atheism” that Gully Foyle mentioned*: since the Singularity vaguely resembles religious ideas, it must come from the same logical errors. This position is shockingly common among transhumanists: I have seen many, many dismissals (even from James Hughes!) of the Singularity as “too Christian”, “cultish”, or whatever. My guess is that when people are scared of the Singularity, they let their old anti-religion heuristics kick in, and the Singularity is one of the few cases where those heuristics don’t work. It’s depressing to watch.
*A semi-relevant idea that I’ve had for some time: I suspect many atheists maintain depressing attitudes like nihilism and the inevitability of death and human extinction, and irrationally reject counterarguments, as a method of signaling/bragging about their mental fitness - “look at me, I’m tough enough to endure the pain of existence, unlike you weak people in denial.” I suspect this is also partly responsible for Michael’s attitude #3: many turn healthy rejection of anthropocentric beliefs into a false humility and denigrate human ability as a result.
May 26th, 2007 at 2:43 pm
Remember when we used an abacus or a slide rule? There were no calculators. Remember a time, when you greeted someone, you used their title or called them “Mr.” or “Mrs.”? Remember when dinosaurs went about on their bellies? Remember when the local Reverend was considered to be a valid character reference on a resume? Remember when television was a bit of a novelty? Remember when most people who went to a hospital for any procedure ‘got their affairs in order’ first? Remember when the word “cancer” was spoken in a whisper, when it was even spoken aloud? Remember when the height of luxury was a Cadillac or a Buick? (Today, the cheapest car on the road offers more luxuries, features, and often, even more power.) Remember when your nation called and you didn’t ask questions? It was
called “Patriotism” back then.
In the last fifty years all of these things – and more - have changed in my own lifetime. Theology has changed. Evolutionary Theory has changed. Cosmology has changed. Society has changed. Cultures have changed. The world today is almost completely alien to what it was just a few decades past. The standard memes and narratives I learned and grew up with have changed. Dramatically. Shockingly so. This is not the same nation I grew up in. The world, itself, is different. It is almost unrecognizable. From conclusions reached from my own multi-disciplinary research, the next decade will see such dramatic change as has never been imagined. All of our memes and narratives will change. They will just change much quicker than they ever have before. i.e. if people like myself think the last fifty years have been a helluva ride, the next ten to twenty are going to be breath-taking.
Any AI which can recursively improve itself will learn very quickly that certain strategies have merit and that others are less than desirable. With known advances in quantum computing, it will be able to make those changes in a, shall we say, quantum heartbeat. The “losing/self-destructive strategies won’t work” was a tongue-in-cheek tautology.
Although I am not as versed in the subtleties of AI and The Singularity as you or many of your readers, I have studied the impact of other technologies upon our societies and cultures. I have also discussed this impact with several politicians, philosophers, physicists, neuroscientists and even biologists and artists. I’ve even gotten the opinions of the “Average Joe.” As I said above, I have also lived through and experienced the impact of modern technologies upon our “normal” memes and narratives. My opinion was, and is, informed. It is, finally, my own opinion. Even without SAI, we are soon coming upon what could legitimately be called a “Singularity,” in any case. When that happens, everything that we know will be changed - almost overnight.
And the final comment earlier was relevant. As much as we seek to plan and dream and prepare for the future, we must remain well aware of the “now.” Mow the lawn. Wash the dishes. Kiss and cuddle with the wife. Watch a movie. Stop and smell the flowers once in awhile.
May 27th, 2007 at 5:36 am
Michael Anissimov Says: “Roko, I think you’ve got it, by jove!”
Glad to be of service! It’s always a pleasure to help out by pointing out the deficiencies of the average joe
May 27th, 2007 at 6:31 am
IanC writes:
That link does absolutely not contain research that would support such a conclusion. It is just about poor people with children. Poor people without children are doing worse. It doesn’t even contain *any* data about the super rich so that it is completely impossible to even begin comparing growth of income between these groups. I read on other some page that data about yearly income above 1 million in US is not even retained so that all these rich guys completely disappear out of the picture of any possible US government research. Do you have some other link supporting your completely counter factual and contra intuitive claims? To think I almost believed you, it was giving me some home for mankind, but instead you turn out to be completely out of whack with reality.
As to Michaels idea about all programmers believing in more-than-zero-sum games: That goes only for the well paid kind of programmers one can find at google. I think most other programmers (and I *am* a programmer) soon realize that such strategies are almost absent at the workplace. Instead is it all about looking good to the boss without actually doing much. In fact the worst thing one can do (financially) as a programmer is to write software that works, because there will be no need for you any more. I am starting to feel like I am among people with their heads in the clouds here, as much as I would like some things to be true. Give me some evidence.
May 27th, 2007 at 9:53 am
Anton, he means AGI programmers, who by and large do not work in zero-sum corporate environments.
May 27th, 2007 at 12:58 pm
Nick Tarleton:
> Anton, he means AGI programmers, who > by and large do not work in zero-sum
> corporate environments.
The way I picture it is that large economic differences create a big gradient along which we will find a lot of different kinds of programmers. I don’t think AGI programmers have a very much better chance of being the ones stumbling on AI earlier than any other programmers along the long tail. If we have large economic differences it will lead to quick and dirty kinds of solutions while a more egalitarian system will cause slower progress, but more coordinated.
May 27th, 2007 at 1:21 pm
Anton, AGI programmers do not have a zero-sum mentality. They are transhumanists working on AGI so they can eliminate nonconsensual death and suffering. AGI programmers certainly are more likely to create AGI - they’re specifically working on it.
Dude, you’re focusing way too much on the economics of this. The way AGI is approached will have the most to do with the specific backgrounds of the people working on it, not the socioeconomic picture of the nation as a whole.
May 27th, 2007 at 9:29 pm
And programmers working on Microsoft Windows don’t have a better chance of producing an OS than programmers working on other things?
May 28th, 2007 at 3:38 am
Nick Tarleton Says: “And programmers working on Microsoft Windows don’t have a better chance of producing an OS than programmers working on other things?”
Sometimes it can seem like that…
May 28th, 2007 at 2:24 pm
The Singularity – a sci-fi future for humanity
Commentary by Futuretalk
Some describe the Singularity as “a point in time when technological progress becomes so rapid that it radically transforms humankind at a faster rate than anyone alive today can comprehend.”
Technological progress expands exponentially. Moore’s law started at doubling computer chip values every 24 months, and then later increased to every 18 months. There are many other areas benefiting from exponential expansion of technology advancements, including projected breakthroughs in biotech, nanotech, infotech and cognitive sciences.
And even the rates of exponential advancements change. For example, a technology that now doubles in speed/value every 12 months might eventually double in 9 months; then 6 months; then 3; 1; every week; every day; every hour, etc.
With possibly thousands or even millions of science and technology areas expanding exponentially, today’s human minds will soon be overwhelmed. It will require a merged human-machine being, circa 2040, to understand a future that expands so rapidly.
As science and technologies continue to advance, I wonder which areas of development will be deemed important for humanity to direct its expanded intelligence.
Focus may switch to wormholes that can break light speed barriers enabling interstellar travel, and time travel systems that conceivably could insure safer advances into the future.
Or an incredibly smart global Internet could evolve that would combine the powers of 10 or 11 billion humans into a single ultra-powerful intelligence bent on searching the galaxies for like intelligences to form a “federation” of universal civilizations.
Truly, a sci-fi future awaits humanity when it achieves Singularity status.
Comments welcome.
May 30th, 2007 at 3:42 pm
Anton; the poor “with children” is the sole category of the poor which can be truly called a class. Almost all those whom are poor, and do not have children, have some other qualifier about them which excludes them from being a part of the “lower class.” By way of example: Berkeley, CA has one of the highest poverty rates in the nation. Yet you won’t find any starving children there. Why? Simple; the jobless college students are counted amongst the poor for national and local statistics.
In Arizona, a large, large portion of the homeless here are the result not of poverty, but of psychological conditions precluding their ability to be productive members of society; many even refuse to partake in the public-welfare programs available to them.
This is all easily and well-available.
If you’re looking for more, however, take this into consideration; the poorest 10% of Americans remain wealthier than 2/3rds of the world, per PBS — and you really just don’t get any “bluer” than PBS without being NPR or open communists.
The point, however, is painfully simple; if disposable income is growing as a percentage of the income of those who need it the most — that being, the “working poor” (most especially those with children) — then you really cannot claim that because the “super-rich” are “getting richer” that this is “on the backs of the poor.”
It just isn’t so, and that’s just the way it is when you discuss a positive-sum game. I’ll use a rather common example of this: I’m willing to be, Anton, that you believe that the trade deficit with China is one of the sources of the loss of American manufacturing jobs, yes? This is a common meme amongst those who believe that line about the rich getting richer on the backs of the poor, while the poor get poorer.
The unfortunate truth which most mercantiliist-protectionists (those who concern themselves with the trade deficit qualify as such) fail to recognize is that as the deficit has increased, so too has the manufacturing output… moreover, as the deficit has gone down, so too has the manufacturing output of our nation.
The majority of the inbound trade in our nation is reflected in two areas, by “rank”: investment capital for State-side ventures (which creates jobs hand-over-fist), and raw goods for manufacture.
This is all easily documentable with a tiny bit of digging, and this really isn’t quite the correct forum for this kind of a discussion… lol.
So here’s the basic thrust: So the poor are “getting poorer in comparison to the wealthy” — I’ll stipulate that. Why is this a bad thing?
June 1st, 2007 at 6:43 pm
“and you really just don’t get any “bluer” than PBS without being NPR or open communists.”
If you are truly interested in politics, please learn more about the space of political ideas. The old line “Anyone who opposes plan XYZ is a DIRTY COMMIE!” is really becoming an empty cliche (well, it always was an empty cliche, but people realize it now).
June 5th, 2007 at 2:22 pm
Tom: the Communist Party is recognized to be a far-left extremist organization. PBS is in a well-documented fashion known to be part of the political left. My statement, while “colorful”, was factual, and had nothing to do with comments on agendas or the like.
This is all simple enough. [chest-puffing]And the idea of you attempting to school someone who is a sustaining member of the LP and a previously-avowed (in this forum) libertarian on the topic of “the space of political ideas” is, frankly, facetious.[/chest-puffing]
September 18th, 2007 at 5:01 am
Buy ephedra online drugstores yellow swarms….
Ephedra….
November 3rd, 2007 at 2:46 pm
Cia;is online shop….
Cia;is online shop….
July 15th, 2008 at 5:35 am
Recall tadalafil….
Recall tadalafil….
July 15th, 2008 at 10:37 pm
Vicoprofen side effects when pregnant….
Free vicoprofen. Where to buy vicoprofen. How to get vicoprofen. Vicoprofen….