Thank you to all the bright people who participate with great enthusiasm in the comments on Accelerating Future. The quality of discussions here is often very high — thank you!
In the “future superintelligences indistinguishable from today’s financial markets?” thread, DMan and Mitchell Porter engage in a discussion about emotions and their relevance to AI.
Correction: Mitchell Porter not Howe. Sorry to Mr. Porter, both him and Mr. Howe have insightful comments but I usually find myself reposting the latter.
DMan says:
If I understand Rinesi correctly, he is equating distributed networks of information processing with AI. This model has been used by some neuroscientists as a model for consciousness, without much success in my opinion.
The problem, it seems to me, is that such a system would always be too diffuse to ever have an internal model of self that’s required for a useful form of consciousness. As far as I can see, there has to be a central point to which information flows and is interpreted by the overseer of the ‘self’.
It’s really like saying that a flock of birds flying in formation, schools of fish, or the coordinated activities of a termite nest are ‘intelligences’ in of themselves. In many ways the cooperative activities of social animals and hive insects resemble the financial market metaphor – would you call them an ‘intelligence’?
It’s kind of an ‘emergent hive mind’ theory with more in common with magical thinking than science.
As to AI being no existential threat:
I would (again) refer you to the excellent book by science writer Rita Carter, ‘Consciousness’ that collates evidence from neuroscience, cognitive studies and psychiatry in an attempt to understand the only working model of consciousness we have – us.
In it, research appears to indicate that:
a) There can be no ‘intelligence’ without ‘consciousness’
b) Consciousness – no matter human or otherwise – requires some form of embodiment to function (so, again, Rinesi’s hive mind is not embodiment).
c) Consciousness requires an emotional component in order to function. The very act of making even an abstract judgement requires a ‘feel’ for what is ‘right’ or ‘wrong’ – that’s emotion.
My point in mentioning this is that an AI (perhaps better name is AC – Artificial Consciousness?) will in all likelihood have to be able to experience emotion to be useful to us. Without emotion there is only a pallid facsimile of creativity – and without creativity what could it invent?
So if it can – or must – have emotion, it most certainly can and will be dangerous. It could feel resentment, jealously, ambition and hatred.
This is why,I suspect, many so-called AI ‘experts’ (experts on something that is currently not a reality doesn’t really warrant the title does it?) are uncomfortable with the idea of emotion being necessary, and go out of their way to dismiss it. Because it implies a potential existential threat to us.
But if they’re wrong and emotion is indeed needed for AI, then Rinesi’s telling us not to worry is – at best – irresponsible.
Mitchell replied:
“Without emotion there is only a pallid facsimile of creativity – and without creativity what could it invent?â€
Computers can invent in the way that evolution does – through trial and error. They can also employ smarter search algorithms than that, e.g. to explore a space of possible designs.
The whole significance of computing is that it provides a mathematical theory and a material technology capable of duplicating every mental function – goal-directed behavior, pattern recognition, communication – in a way which makes no reference to mind, thought, meaning, consciousness, emotion, etc. It’s all just physical “state machines†interacting with each other.
And since neuroscience is analyzing the human brain and human behavior in the same way, this raises serious questions about the relationship between subjectivity and the world of mindless physical cause and effect. That debate has been around for centuries now, but the theory of computation gives it a new twist. Lots of people now like to think that e.g. emotion or consciousness occurs wherever a certain type of computation occurs. Whether you believe that or not, the significance for AI is that emotion and consciousness are not a necessary part of AI theory or practice. You don’t need to think of a thermostat’s operation in terms of liking and disliking in order to design and make it, and the same goes for the far more intricate feedbacks and calculations which would make up an artificial “intelligence†capable of rivaling the human mind.
There’s a bit more back-and-forth, then Mitchell posts the following, which I think is especially on-target:
The creative limits of chess computers and story grammars don’t tell us about the limits of unconscious computation in general. Consider any cognitive process which involves emotion, creativity, consciousness, compassion, etc. It’s going to have a cause-and-effect description, in which there are transitions between various psychological states. To produce the same outputs, all that’s required is a process with the same cause-and-effect structure. The “states†involved don’t have to be psychological or conscious – unless functionalist philosophy of mind is right, and all those psychological properties really are present whenever you have the right sort of causal structure.
Even if I take that view – suppose I want to make a creative AI, and I decide to follow your advice and make it “emotionalâ€. How do I even do that? If I adopt a particular software design, how do I know whether or not it corresponds to the existence of emotion in the AI?
The ultimate reason that this doesn’t seem like very useful advice (for someone who wants to make an intellectually powerful AI – we’ll get to the ethical issue in a moment) is that emotion itself doesn’t solve problems, even in humans. If the problems themselves involve emotions, then an emotion can *be* the answer – happiness might be the answer to unhappiness, just as a glass of water can be the answer to thirst. But if you’re a monkey in a room trying to get at a banana on the ceiling, emotion itself does not tell you that the answer consists of stacking boxes and climbing on top of them. Or rather, emotion is not the process which will materialize that possibility in your mind. Emotion may motivate you to devote cognitive resources to the problem, and your mind may be wired to produce an emotion (excitement) when an imagined solution looks like it will work. But the consideration of possible actions – visualization, combinatorial explorations – all that is more “computational†than “emotionalâ€, and that’s the process which generates possible solutions. (Embodiment also plays a role here, because it permits, not just formal trial and error, but also a more formless experimentation which will suggest possibilities and components of possibilities.)
If a process is to produce the solution to a problem, it has to generate possible solutions and then evaluate whether they are useful. It’s a psychological fact about human beings that emotion and consciousness play a role in this evaluation of possibilities, and they even play a role in determining what we will think of as a problem. But from a computational perspective, it doesn’t have to be emotion or consciousness which performs the evaluative function. There just needs to be a sub-process which discriminates or guides appropriately, and that can be yet more unconscious computation. If you look at problem-solving algorithms searching a space of possibilities for solutions, they typically alternate between the formal generation of new possibilities, and the formal evaluation of the newly generated possibilities – do they offer progress towards a complete solution.
Summing up this stage of my argument: The intellectual power of an AI would not reside in the existence of emotions or the existence of an emotionlike structure of cognitive control and guidance. It would depend on the quality and power of its basic problem-solving algorithms. I am trying to finesse the hard problems associated with consciousness and subjectivity by being agnostic, in this discussion, about their relationship to material and computational reality. Further down the page, David Pearce and “Continuously Computed†have given us statements of the two main approaches, namely, choice of material substrate matters for the existence of subjectivity, and, only causal structure matters; consciousness reduces to substance, or consciousness reduces to function. Of course it’s a complex and very important issue, but I do want to emphasize just how far we can expect to go in the creation of general-purpose AI, employing only computational concepts.
The other topic you bring up is whether the creation of emotional AI is a way to achieve what our blog-host would call “Friendlinessâ€: rather than trying to engineer the functional equivalent of friendliness in an emotionless AI, you make an emotional AI and start it off compassionate. But that strategy requires that you begin to solve the hard problem of consciousness, as it pertains to emotion and compassion; you would need to say *how* to make an AI emotional or compassionate. And you would need to understand something of the developmental dynamics in an artificial emotional system. I assume you don’t want it *going mad* out of extreme sensitivity.
Even people who want to make an emotionless but Friendly AI have to find solutions to those problems anyway, because, even if emotions are not part of the AI’s mechanism, they have to be part of its supposed domain of competence. A general-purpose AI could not know how to treat human beings ethically or even safely, without having a highly refined understanding of emotion and all these other aspects of conscious subjective experience. Part of SIAI’s current thinking about the achievement of Friendliness seems to involve outsourcing some of these problems to the proto-friendly AI, which will engage in neuroscientific studies aimed at identifying what real-world material phenomenon or attribute is intended by all this vague human talk about emotions and consciousness and so on. It’s an interesting idea but it still requires as a starting point some minimal idea, on the part of the programmers or design theorists, about how to tell the AI what to investigate and how to value it, e.g. something like “We want the world to be optimized according to the criteria that are used by the part of the brain responsible for the judgements which ultimately produce confident assertions of happiness with the overall situation.â€
Does anyone else believe that phenomenological consciousness is necessary for general problem-solving? I’m sort of confused why anyone would think that. This line in particular is confusing to me:
And how do they judge, from ‘trial and error’ which is the best solution? Other than something blowing up of course? Without the ability to judge – which according to cognitive studies, requires emotion?
This statement sort of implies that “judge” is a natural category distinct from trial-and-error, and that the two are naturally separated and distinct clusters in algorithm-space, rather than two points on a continuum. It doesn’t make sense because there’s obviously a million shades of competence between the most simplistic trial-and-error algorithms and human judgement, to refer to them as two natural categories is most confusing. Surely animals have solved numerous “judgement” problems without human-level “emotions”.
The thinking seems to be that the universe can grant a special gift, “consciousness”, on certain hallowed beings, then they get magical judgement powers. But phenomenological consciousness doesn’t seem particularly related to judgement capabilities, and thinkers like Chalmers never suggest that it does in their papers on consciousness.
Checking out the reviews for Consciousness on Amazon, I see that Carter is a science writer rather than a scientist, and makes basic errors like thinking that an atom becomes positively charged when it gains an electron. Still, I don’t think that being a science writer rather than a scientist should ruin someone’s scientific credibility. Some science writers have a much more thorough interdisciplinary knowledge of science than many scientists.
It’s pretty plausible that phenomenological consciousness is such a wrong concept that ‘evolution is conscious’ is not even wrong, rather than being wrong, but evolution is pretty clearly a pretty general problem solver. OTOH, the idea that phenomenaological consciousness is necessary for *existence* isn’t nearly a silly, but has no bearing on AGI except conceivably as weak argument for something like the Omega Point.
I take the liberty to collectively thank you for seeding these discussions and for being such a bright transdude yourself!
Blog is one hell of a druug…
My girlfriend displays emotions … but I’m not sure that she’s sentient. Her emotional displays are manipulative, part of all that ancient sexual gamesmanship. I think she’s just faking it, but I’m not sure. How much should I risk to find out?
A sufficiently capable AGI could display emotional content as needed in order to elicit the appropriate response. There is no need to posit the existence of internal emotional states nor of consciousness. AGI self-awareness would exist purely to monitor and improve the efficacy of its own emitted behaviors.
Thanks for responding Michael.
Just a note on Rita Carter – in my post I do actually identify her as a science writer, not a scientist. As such I don’t doubt she occasionally has made errors in her writing – I’ve seen some whoppers in New Scientist – but she’s also made some perceptive observations and had some deep discussions with the leading lights in AI research.
I also quoted her because she explains neuroscience and cognitive research in a way that’s accessible to the non-scientist, and is refreshingly free of dogma. I could provide links to research papers, but much of it may be greek to those outside the field – including myself, despite trawling through as much of it as I can.
As to the sentence that confused you:
“there’s obviously a million shades of competence between the most simplistic trial-and-error algorithms and human judgement”
No doubt, but I’m not talking – or arguing – about most simplistic trial-and-error algorithms.
An amoeba ‘decides’ to engulf its prey through a series of chemical reactions without a limbic system or a brain. So it requires neither emotion nor consciousness as we understand it to function and act.
It’s when you move into the high cognitive realms, such as inventing, synthesizing and creating that I think trial and error alone can’t do it. The ability to decide – to judge the best solution – is not always cut and dry. We can deal with ambiguity – the ‘yes’ ‘no’ and ‘maybe’ I mentioned – and I believe an inventive AI would need to be able to do that as well.
Kurt Godel demonstrated that some of this conceptual ambiguity is fundamental, and can happen at the algorithmic level, so an AI dealing in astrophysics would need to be able to account for aspects outside ‘simplistic trial-and-error algorithms’ or an internal array of pre-programmed symbols.
In response to:
“Surely animals have solved numerous “judgement” problems without human-level “emotions”
and
“The thinking seems to be that the universe can grant a special gift, “consciousnessâ€, on certain hallowed beings”
Like the amoeba example above, that depends on how complex an animal you mean. Again, in keeping with quoting sources that are accessible, the March 2008 edition of National Geographic made a deep impression on me.
In the cover article, ‘Inside Animal Minds’ animal behavioural researchers showed that much of the traits that human chauvinists (and specieists) assert as uniquely human are shared by a much broader cross section of the animal kingdom than previously thought.
Traits like ‘a theory of mind’, toolmaking (a New Caledonian crow was filmed in a lab making tools to get at food), mirror studies, synthesizing new concepts (the famous African Grey Alex vocalizing ‘ban-erry’ from ‘banana’ and ‘cherry’ to identify an apple, he hadn’t been taught ‘apple’), longwool sheep able to recognise and remember individual faces of other sheep, ‘fast reasoning’ tests on dogs, chimpanzees with better short term memory than humans.
The list is enormous an varied, so no, I in no way think consciousness is conferred on ‘hallowed beings’, it’s quite clearly an emergent property of on any system of highly organised, self-referential complexity. And some of the research mentioned seems to show that our emotions differ from other encephalised animals only in levels of complexity.
Signs of an emotional life seems to emerge in far less complex creatures than previously thought.
As Mitchell said, perhaps an AI with emotions may even emerge spontaneously. Emotions may be a natural consequence beyond a certain threshold of complexity.
But like the replicants of Blade Runner, without foundational developmental design, if these emotions do suddenly emerge, would the AI know how to deal with them? Wouldn’t it be like a human emerging fully grown from a vat?
Perhaps I’m talking at cross purposes here. I’m envisaging an artificial mind, but perhaps others are aiming for something much simpler. Something simpler than us, or lacking something we have would, it seems to me, be not able to function unattended in every conceivable scenario.
Having being in charge of a large scale project, I can tell you how useless and hindering the individuals are who need constant assistance and monitoring. The project only moved forward when those individuals were ejected.
I personally think for it to be a superintelligence it must be ‘human +’ not ‘human + but – ability xyz’.
Time will tell who is right, but until then we’ll probably have to agree to disagree, since the jury is out prior to the emergence of the first true AI (even then we’ll probably be arguing what constitutes ‘true’!).
But I suspect we’ll all want more than what is essentially a smart zombie.
An amoeba ‘decides’ to engulf its prey through a series of chemical reactions without a limbic system or a brain. So it requires neither emotion nor consciousness as we understand it to function and act.
How do you know that there is a difference in nature between you and the amoeba?
More precisely, at which point along the complexity scale does consciousness occur and what difference does it makes such that you can tell it from the outside (i.e. from you own perspective)?
You choose to assign some universal significance to your own consciousness (the only one you truly know about) others choose not to.
You are both wrong, you don’t know, these are just opinions.
Mark in comment #3 gives some rehash of Searle’s chinese room argument but Searle is wrong too, he doesn’t prove the “impossibility of true artificial intelligence” he just proves that we cannot tell the difference between what we believe to be intelligence and consciousness and faking it.
So, what is exactly the purpose of these discussions?
Perhaps a more penetrating question would be, since you appear to think all these discussions are pointless, why are you here?
Wait, perhaps a clue here:
http://www.acceleratingfuture.com/michael/blog/2010/11/future-superintelligences-indistinguishable-from-todays-financial-markets/#comment-141843
..Because every blog or forum has to have at least one.
DMan
why are you here?
I am here because I am truly interested in strong AI and I feel that all these pointless discussions, ditherings and unwarranted paranoia are counterproductive and sometimes even on purpose.
SIAI people are actually against strong AI not just wary about AI unfriendliness, AI will rob them of the “social status” they derive from their cleverness.
They would be as hapless in front of a superhuman AI as the most idiotic morons they can happily sneer about today.
Your observations about cleverness, social status, and sneering, accurately represent the motivations of smart people. Spot on.
Even the idea that SIAI wants to stop strong AI is absurd – let alone the idea that they want to stop it so they can continue to preen from the top of the intellectual totem pole. SIAI just want to do it right, that’s all.
Let me tell you what real opposition to AI looks like. First, it looks like the Unabomber: terrorist acts directed against researchers, carried out by intellectually isolated fanatics. Then, it looks like the Tea Party movement: a diverse mass of technocratically disenfranchised people trying to stop what they regard as change that is bad and out of control. And finally, it looks like the war on terror: the governments of the world in permanent emergency mode, trying to crush the hydra-headed monster before it’s too late.
In case you hadn’t noticed, AI is potentially the end of the human race. And while there’s a technophile sub-sub-subculture which is quite OK with that prospect, most people are going to feel differently about it, and they won’t be swayed by hopes that a nice AI would provide them with a transhuman cornucopia, or by remonstrations that they’re being racist towards the “mind children”.
In case you hadn’t noticed, AI is potentially the end of the human race.
No, plain no, this is just a paranoid projection of the idiotic monkey brain.
A tech flavored Eschatology, THIS is the real danger and the idiotic monkeys don’t need AI to wipe themselves off.
Anything from nuclear war to cult like collective suicide to destabilizing complex societies by decreasing marginal returns (re Tainter’s Collapse of Complex Societies).
The Roman Empire crashed but there were other societies around, if the globalized world crashes there will be nothing left, AI or no AI.
“We can deal with ambiguity – the ‘yes’ ‘no’ and ‘maybe’ I mentioned – and I believe an inventive AI would need to be able to do that as well.”
Yes, we can and must deal with ambiguity (real-life systems nearly always are such, only probabilities and approximations) but in the end, to accomplish anything at all, to invent and create something, you must make a choice, 0 or 1. Inventions are produced by a series of these choices, and even though none of those choices is optimal, we can build systems that work well enough.
There’s fuzzy logic and all sorts of algorithms that deal with uncertainties.
The relationship between judgement and emotion in humans is simple to me. Imagining doing something really bad in the sense of something that has high negative personal utility or in the sense of something I think is morally wrong makes me feel bad and this helps decide not to do it.
Humans do use emotion to make decisions and I assume that animals do the same thing more. My cat probably doesn’t visuallize being eaten if he goes toward a big dog he doesn’t know and rubs his head against it’s forelegs but dogs scare him so he never tries something that dumb.
Emotion seems to me to be one way animals judge utility. I can see several reasons why an AI might be created with emotion or some state resembling it but I have no idea whether it’s necessary.
Does anyone else believe that phenomenological consciousness is necessary for general problem-solving?
Isn’t that pretty-much the “could there be zombies?” question?
“Is the unconscious philosophical zombie possible?”
– http://www.takeonit.com/question/316.aspx
It’s not the same. Human zombies could be impossible but there could still be AGI zombies. Insofar as I understand the mission of SIAI, we’re trying to create an AGI zombie.
I think very humanlike zombies could be possible, but not atomically identical to humans, so the philosophical hand-wringing sort of evaporates.
Funny to see Eliezer and Chalmers on opposing sides of the argument. :)
Kevembuangga says: ” … he just proves that we cannot tell the difference between what we believe to be intelligence and consciousness and faking it.”
A difference that makes no difference is no difference. Occam’s razor is a tool not a law; you can posit any jumbled mess you want, as long as the result is the same.
We identify consciousness in others when the stimulus-response cycle of our interaction fits certain parameters. That is the definition of consciousness.
There is no need to posit a ghost in the machine.
If a machine passes the test, then it is conscious no matter what the internal mechanisms may be.
There is no need to posit a ghost in the machine.
Not at all what I am saying.
You choose to be deliberately obtuse, not a problem for me but I won’t waste time trying to explain.
I don’t need to make a point beside the fact that I don’t share the common opinion(s).
How about donating it for the Studebaker Countrywide Museum in South Bend, IN wherever it absolutely was manufactured? You can contact Andy Beckman at 888-391-5600. Normally, deliver it to South Bend during the week of July 29 through Aug 4 should the Studebaker Driver’s Club International meet will probably be in city.
Be aware that emeralds have become tricky along with is often brittle. These kinds of stones are by natural means hardy, nevertheless usually effortlessly have got blemishes, helping to make these individuals at risk of casino chips. These kinds of jewelry usually are not proposed in case you regularly help their particular arms or perhaps whom carry out difficult or manual labour .
Respect to post author, some excellent selective information . “It’s always too early to quit.” by Norman Vincent Peale.