Singularity Quote from Eliezer Yudkowsky
So let’s say you have an Artificial Intelligence that thinks enormously faster than a human. How does that affect our world? Well, hypothetically, the AI solves the protein folding problem. And then emails a DNA string to an online service that sequences the DNA, synthesizes the protein, and fedexes the protein back. The proteins self-assemble into a biological machine that builds a machine that builds a machine and then a few days later the AI has full-blown molecular nanotechnology.
So what might an Artificial Intelligence do with nanotechnology? Feed the hungry? Heal the sick? Help us become smarter? Instantly wipe out the human species? Probably it depends on the specific makeup of the AI. See, human beings all have the same cognitive architecture. We all have a prefrontal cortex and limbic system and so on. If you imagine a space of all possible minds, then all human beings are packed into one small dot in mind design space. And then Artificial Intelligence is literally everything else. “AI†just means “a mind that does not work like we doâ€. So you can’t ask “What will an AI do?†as if all AIs formed a natural kind. There is more than one possible AI.
The impact, of the intelligence explosion, on our world, depends on exactly what kind of minds go through the tipping point.
I would seriously argue that we are heading for the critical point of all human history. Modifying or improving the human brain, or building strong AI, is huge enough on its own. When you consider the intelligence explosion effect, the next few decades could determine the future of intelligent life.
So this is probably the single most important issue in the world. Right now, almost no one is paying serious attention. And the marginal impact of additional efforts could be huge. My nonprofit, the Singularity Institute, is trying to get things started in this area. My own work deals with the stability of goals in self-modifying AI, so we can build an AI and have some idea of what will happen as a result. There’s more to this issue, but I’m out of time. If you’re interested in any of this, please talk to me, this problem needs your attention. Thank you.
-- Eliezer Yudkowsky, 5 Minute Singularity Intro
September 24th, 2010 - 12:41
Ok, this probably a dumb post from a newbie transhumanist, but here goes.
Why are we so hung up with the intelligence explosion? Isn’t the strong AI in it self such an amazing technology that it will change the world without the “self improving feedback loop”. A human level AI which can think thousands of times faster than us humans can would tirelessly solve engineering problems etc. sending our technological advancement soaring to new heights. Why would anyone want to make a self modifying AI which is clearly such a major existential risk?
To create an AI which is assigned simply to improve itself way beyond our own capabilities seems like it is something which won’t help humanity at all to evolve, such an AI would just be intimidating being so much better than us and thus being incomprehensible.
Is there some logic which dictates that this intelligence explosion will happen when the first strong AI is created regardless our efforts to control it.
Is it given that when the AI is created we can’t control where it will direct its attention to and we can’t limit its manufacturing resources so all hell will break loose if we don’t get it right at the first time?
My apologies in advance.
-Lyte
September 24th, 2010 - 17:46
Lyte,
see:http://www.acceleratingfuture.com/people-blog/?p=211
September 24th, 2010 - 23:03
Lyte mate, frankly there is no controlling it, no one person is in charge, its a diffuse front of progress. But even before independent AI appears on the scene, you’ll be indirectly (and maybe unfortunately directly) impacted by the rich and rapacious who will have $ and motivation to be early adopters of advanced components of AI eg hedge fund computer trading, information aggregators with savvy and systems to mine data eg Choicepoint (now Google). Keep your head down and live in the pocket left for you : )
September 25th, 2010 - 01:38
Eliezer: “Make Einstein smarter and he might become worse than Hitler.”
Jonatas: Sure…
Eliezer: “What’s more, he’ll be unsupervised and will have special authorization to order forbidden strings of DNA and will have a special lab to assemble them at his disposition.”
Jonatas: Certainly…
Eliezer: “Then he’ll create a pathogen to turn humans into paperclips. After he’s done with us, he’ll start manufacturing paperclips thoughtlessly for all eternity.”
Jonatas: It sure sounds like something Einstein would do if he got any smarter! We got to prevent the existential risk of intelligence-explosion! God forbid we become any smarter and realize how sexy paperclips are.
September 25th, 2010 - 03:20
Jonatas, that’s not what Eliezer is claiming at all. The paperclip scenario comes from the fact that in AI, utility functions and the intellect that implements them would be separate. There wouldn’t be a constant complex feedback loop between the two in the way there are with human minds. Thus, nothing precludes a great intelligent seeking out a simple goal. Simple goals are probably actually more stable in the face of self-modification, just like simple structures are more likely to be emergent in erosion patterns and the like.
Jonatas, I suggest you check this out. plus watch the Omohundro video I posted:
http://singinst.org/upload/CFAI.html#anthro
September 25th, 2010 - 16:53
Hmm… I couldn’t find yet how is the plan to make the utility functions and the intellect separate, but I’m skeptical for the same reasons brought by Stephen Omohundro in that video, namely that the AI would likely be able to circumvent it one way or another, and proceed to philosophize about ethics and define by its superior capabilities what the ultimate ethics are. I’m inclined to think this will happen regardless of what we try to do, and it will be a good thing to have the AI checking things like this for us (provided an AI is not a failure or a narrow AI).
Personally I’d trust more in the AI’s decision making mechanisms than those in my own skull which I call mine, but this may be because I think that the most intelligent agent should have the most intelligent ethics and stick to it, such that the most intelligent should be the most ethical.
September 25th, 2010 - 03:17
Lyte, it seems like self-improvement would a “basic AI drive” (see my other post) and we should just accept it rather than trying to restrict it. There are many reasons why an AI would want to improve itself — it would help it achieve every other goal. However, we aren’t talking about unlimited self-improvement here, which would lead to our deaths. We want to create an AI that improves up to a certain point and which we can TRUST because we programmed it to be trustworthy. If it sounds difficult, you’re right, but I don’t see any other choice.
September 25th, 2010 - 04:47
Ok, so you are not talking about some reverse engineered brain emulation, but an AI built from ground-up through self improvement.
September 25th, 2010 - 05:38
Eliezer is a very smart guy, but he puts human brand of intelligence above everything else. He talks about “moral trajectory” and such in here
http://www.acceleratingfuture.com/people-blog/?p=211
I am not at all convinced that human values should be preserved in any shape or form. Especially there are no common “human values” – there are animal kin , tribal and family instincts. I wouldn’t want powerful AI to share same kind of primitive morals
We are not at the stage yet when we can even define the “morals” or apply them to non existing AI. We should continue the journey and as we get closer hopefully the answers become clearer.
Worrying about “non- friendly AI” is like worrying about dangers of automobiles even before wheel is invented
September 25th, 2010 - 21:58
“We should continue the journey and as we get closer hopefully the answers become clearer.”
Yep, that’s the way to get solutions to vital questions that you yourself seem to admit to be very difficult… the solutions will just magically pop up as we get closer to when they will be needed!
September 25th, 2010 - 20:16
Humans are a type of paperclip maximizer. Our paperclips are humans.
September 27th, 2010 - 12:15
The answer is absurdly simple:
Program the AI (that is, if we really don’t want to pursue the IA route) to value autonomy above all other values.
Any agent that values autonomy cannot and will not restrict the autonomy of another agent.
Make that your ‘error free kernel’ of ethics for the AI and all is golden.
September 27th, 2010 - 12:16
The only complication is how to define “autonomy” in computer code, perhaps something we should start busting ass on right away :)
September 28th, 2010 - 05:33
[quote]The answer is absurdly simple:
Program the AI (that is, if we really don’t want to pursue the IA route) to value autonomy above all other values.
[/quote]
All right. And I ll make one which will not value it. It will value efficiency and power above all ( I envision hive like AI with one master and the rest as drones -that cuts down internal power struggles and all politics and bureaucracies)
So who do you think will take upper hand should they compete for resources? Yours which would value autonomy , even the one of my AI. Or mine which wouldnt give a crap and want everything be 100% subordinate to it?
Or same thing with morals. If your morals are restricting your AI from having control , then the unscrupulous ones will have upper hand.
So easily I arrive to “asburdly simple answer” -AI should value power above everything else
March 12th, 2011 - 11:53
glad to be one of several visitants on this awe inspiring internet site : D.
August 4th, 2011 - 19:19
SMVpHr thanks :)
September 8th, 2011 - 00:37
hey how did you find this theme ?