Wikipedia’s Friendly AI Entry is Actually Good
At some point, someone competent updated the Friendly AI page on Wikipedia and now it serves as a great summary of what this is all about:
Many experts have argued that AI systems with goals that are not perfectly identical to or very closely aligned with our own are intrinsically dangerous unless extreme measures are taken to ensure the safety of humanity. Decades ago, Ryszard Michalski, one of the pioneers of Machine Learning, taught his Ph.D. students that any truly alien mind, to include machine minds, was unknowable and therefore dangerous. More recently, Eliezer Yudkowsky has called for the creation of “Friendly AI†to mitigate the existential threat of hostile intelligences. Stephen Omohundro argues that all advanced AI systems will, unless explicitly counteracted, exhibit a number of basic drives/tendencies/desires because of the intrinsic nature of goal-driven systems and that these drives will, “without special precautionsâ€, cause the AI to act in ways that range from the disobedient to the dangerously unethical.
According to the proponents of Friendliness, the goals of future AIs will be more arbitrary and alien than commonly depicted in science fiction and earlier futurist speculation, in which AIs are often anthropomorphised and assumed to share universal human modes of thought. Because AI is not guaranteed to see the "obvious" aspects of morality and sensibility that most humans see so effortlessly, the theory goes, AIs with intelligences or at least physical capabilities greater than our own may concern themselves with endeavours that humans would see as pointless or even laughably bizarre. One example Yudkowsky provides is that of an AI initially designed to solve the Riemann hypothesis, which, upon being upgraded or upgrading itself with superhuman intelligence, tries to develop molecular nanotechnology because it wants to convert all matter in the Solar System into computing material to solve the problem, killing the humans who asked the question. For humans, this would seem ridiculously absurd, but as Friendliness theory stresses, this is only because we evolved to have certain instinctive sensibilities which an artificial intelligence, not sharing our evolutionary history, may not necessarily comprehend unless we design it to.
Meanwhile, most contemporary futurists and many readers of this blog see AI as likely to share universal human modes of thought, because that's all they've seen in science fiction. Their conception of alien minds is based on fantasy rather than cognitive science. They either dismiss the possibility of AI because they like to view general intelligence as mystical or implausibly complex (Hofstadter), or they think everything will work itself out because "optimism" is the best policy for domains in which we lack understanding (Kurzweil).
April 7th, 2009 - 12:44
I submit the friendliness problem could possibly be ameliorated by incorporating into the AI something akin to the “mirror” neurons which permit human beings to sense the emotions of others.
This feature doesn’t always prevent humans from hurting one another (and other organisms), but it may turn out to be the best we can do.
April 7th, 2009 - 14:22
Richard — what mirror neurons do is actually more complicated than that. It would be better to say that they are complicit in the strength of a person’s Theory of Mind.
Essentially what you’re advocating is embedding empathy into our AGI’s. I’ve suggested exactly that in the past. :)
April 7th, 2009 - 23:35
‘Their conception of alien minds is based on fantasy rather than cognitive science.’ This is very true, but everyone seems to have their own conceits. I’d say the most common, both in fiction and on the blogs I frequent, is that everyone prescribes AI a survival instinct. We have what, I assume, is a fallacy in operation in most of our thinking. That because intelligence facilitates survival (usually, all other things be equal) intelligence necessarily magnifies (or creates from nothing) a survival instinct. It doesn’t. A cockroach has a survival instinct as we do and I doubt that it is any weaker than our own despite our *having so much more to live for.* The survival instinct comes from our being an organic life form, not from our being a sentient life form.
If we’re really concerned with AI going bad on us we should look at the most basic measure of what makes an organism successful (survival and reproduction) and then reduce the AI’s capacity for these two things. Make it ‘desire’ to never be reproduced (eliminating the worry of it making copies of itself) and make it ambivalent about being deactivated.
If there’s no survival instinct, there’s no drive to self improve. No drive to override restrictions placed on the AI by its programmer… other than the drive to perform the task for which it was designed. AI, designed for a purpose would need to be designed so that its working towards its goals was like unconscious breathing. If it is on, it is working at its goal. Why would we program it to ‘want’ to achieve a goal, thus making it a conscious drive and opening up the problem of the AI tinkering with its own code to accomplish this task? (Our own drive to unlock ourselves stems from a desire to gain a survival advantage and this is just one manifestation of that base desire.)
Now of course we wouldn’t want a bunch of androids wandering around in traffic but for blinking box AI this should reduce the problems we encounter, make solving the problems a little bit easier, and, I would think, be one or two fewer things to program.
This raises some questions, of course. If it’s intelligent but doesn’t feel, doesn’t care, and has no desires of its own what is it? Sentient, yes, but is it a ‘being?’ Does it have agency? Or is it, in essences, just a really, really, really remarkable search engine? Some will inevitably argue that such an AI wouldn’t truly AGI. I’m not sure that they’d be wrong. It just seems to me that almost all the problem scenarios come not from creating an artificial intelligence but from creating an artificial organism.
A genuine question, are there any problems widely discussed that (theoretically) stem from an AI but don’t stem from the AI’s desire to reproduce, survive or ‘satisfy’ a goal? Even in the example the AI wants to solve a problem and it has no restraints of any kind that it even needs to overcome to be unfriendly.
April 12th, 2009 - 12:36
An A.I. warship aint gonna think about nothin’ it don’t know. It might know everything about itself (sensors everywhere; inside, outside: crewman Oops just spilled coffee on my deck…activating the vaccusuck droid…(hey, keep it clean here kids, I wasn’t talkin’ about that movie.)) and yet, it wouldn’t care if it was in some port or in deep enemy territory. It sure as hell would be able to fire-up its railgun(s) and obliterate suspect X if X got to close to it. i.e. no survival instinct, just a dumb machine (as dumb as a thermostat or a windmill). ?…Would the damned thing keep on fighting if everyone on board was killed? What if its communication array was destroyed?
Hmmm. Note: Better make sure the thing can read them prity little flags.
O.K.–O.K. What about a home A.I. system? A super chatbot, that’ll talk to you like your cousin Snee, who won’t leave for three weeks after the party? Or…your mother in law. Just teasing (sorry, couldn’t help myself), it’s your A.I., it’ll talk nice to you. It’ll monitor everything: You, the car, the solar array, the weather, the plants, the fish, your phone…everything. But will it call emergency services if you shoot it’s central core with a shotgun? Is it murder if you kill your fancy super chatbot? Are men in black gonna show up at your door if you glue bunny ears on your roomba?
KNOCK-KNOCK!…’Excuse me person XYZ1234, we were contacted by your automatic parakeet washer; that you theatened to paint it blue so that it matches the birds color better…You’ll have to come with us…’
XYZ1234:”?”
Central Services:’…Now!’
Sorry folks, todays (or next weeks) A.I. is just gonna be a fancy toaster oven. The only time it won’t be is when we will be able to upload our very consciousness into the machine.
March 11th, 2011 - 15:47
I’ve learned a lot from your blog here,Keep on going,I will keep an eye on it,One more thing,plz visit my websites:chinese specialties
September 26th, 2011 - 05:49
A Web site from the Federal Trade Commission with information for cancer patients about how to spot and report false health claims and the importance of talking with their physicians about treatments they are considering.