Moral Realism Debates with Hedonic Treader and David Pearce Monday, Jul 26 2010
philosophy 7:35 pm
The comments section of my SIAI call for volunteers has become home to an extensive debate on moral realism, the relationship between values and intelligence, and the moral implications of a timeless universe and discreteness of spacetime. The primary participants include “Hedonic Treader”, Jonatas Müller, and David Pearce.
Regular readers are familiar with my moral anti-realist views and my emphasis that creating superintelligence with the expectation that it will discover complex values is a depressing sort of planetary suicide.
The moral anti-realist stance is not new. It’s not so much a personal preference, as a statement about the structure of the world — that there are no moral propositions which are true or untrue. “Thou shalt not kill”, is not a part of the structure of the universe, just a rule humans evolved to encourage one another to follow, for obvious reasons.
“It is not contrary to reason to prefer the destruction of the whole world to the scratching of my finger.”
– David Hume, A Treatise Upon Human Nature (1740)
Moral anti-realism has been in the spotlight among transhumanists and singularitarians primarily because of Eliezer Yudkowsky’s Less Wrong blog posts, which explain the relevant issues.
Here’s David’s argument:
One of the hallmarks of intelligence is the capacity to distinguish the important form the trivial. We wouldn’t describe a human who spent his life tiling his garden with paperclips as “intelligent”, even if he used some arcane mathematical formula to do so. At most, we’d say he was an idiot savant.
A value-nihilist will be unimpressed by this response. How can we have any idea what a posthuman superintelligence will find valuable? Why not paperclips? There isn’t any objective fact of the matter – no “truthbearer” to settle the issue.
In one sense, I’d agree. A posthuman superintelligence may value modes of experience and propositional content that we would currently find inconceivable. And of course human cultures display an amazing diversity of moral practices. In another sense, however, I think the value-nihilist response is profoundly mistaken. This is because – for reasons we can’t fathom – the universe does have an inbuilt metric of importance: the pleasure-pain axis. You can’t be mistaken about the importance of your being in agony: its importance is built into the very nature of the experience itself. For reasons science doesn’t understand, organic robots aren’t just zombies with utility functions. The existence of phenomenal agony is an objective fact about the world: the phenomenal experience of pain and pleasure occupies spacetime coordinates, and it’s as much an objective fact about the universe as the rest mass of the electron. If I – or a Ted Bundy – think paperclip tiling is more important than your agony, then this judgement is not just a moral limitation on our part, but also an intellectual limitation – a failure adequately to compare the aesthetic satisfaction derived from contemplating well-tiled paperclips with what it’s like to be in unbearable agony. By contrast, a full-spectrum Superintelligence – as distinct from a mere SuperAsperger – will presumably command an impartial “God’s-eye view”, stripped of egocentric and anthropocentric bias. A Superintelligence will be able to compare and contrast the relative weights of all possible perspectives – and act accordingly. This means cosmic paperclip tiling is out: it’s dumb for humans – and dumb for posthumans too.
I am optimistic that experiments in Artificial Intelligence will verify the truth of moral anti-realism, probably with little remaining doubt, within the next decade or so. Some will naturally argue that machines’ moral non-complexity derives from its lack of phenomenological experience, and that will remain a point of contention until we can experiment with a variety of human-machine hybrids and intermediates. However, I think the broad picture will be a moral realization analogous to the cosmic realization that we were just a tiny corner of the universe. Our moral universe is a tiny corner of the total possible moral universe, and our values are not particularly persuasive to anything but fellow humans.
Just because some morals and feelings are in-built for humans does not mean that they are universally persuasive. It is possible that humans both “discover” their own internal values and “invent” memetic structures around them. An appropriately designed Artificial Intelligence could also “discover” that its ultimate goal in life is to spin itself around like a top all day, and even feel that every conscious agent must enjoy the same thing, whether they argue otherwise or not.
I think that in a sense you have reason for anti-realism, namely that there is total theoretical freedom of behaviors or causes that trigger feelings judged good or bad, and so defining those has a scope limited to humans or some animals, and probably not universal. However, if you make the definition of ethical value abstract as feelings that are judged good by its subject then this is stripped of the specific triggers and can be an universal definition, given that ethics seem to be only concerned with consciousness.
A further possibility is to define the hardware algorithms of good and bad feelings by studying brains, and thus get a more raw exemplification of the mentioned abstraction, but I think probable that the algorithm may be multiply-realizable. Although there could be a generalization of its essential architecture, it may be just a futile attempt, but studying this might help quantify good and bad value.
I’d say that an AI might for some odd reason derive feelings that it judges good by spinning itself around (blame its creators…) but it would be wrong were it to think that humans think the same way (a conclusion similar to anti-realism but not essentially nihilist).
Huh? Moral antirealism is an obvious consequence of a scientific materialist worldview. Of course it’s going to come up in transhumanist circles; Yudkowsky (who doesn’t even self-identify as a moral antirealist due to what I would characterize as trivial semantic quibbles) has nothing to do with it.
What is AI going to tell us that we didn’t know already? The vast majority of these debates are not about any disputed point of fact. I fear the problem is much deeper: materialists and ordinary people simply don’t share the same concept of what it means for something to be “true.” My education has reached the point where objective morality just seems like nonsense: I have some notion of what the world might be like if all agents agreed on what to do, perhaps due to convergent evolution or some theorem of game theory, but that’s not what realists mean by a moral fact being true, and I no longer remember what it feels like to occupy that state of mind.
Giving people more experimental evidence won’t change anyone’s mind until they’ve learned to think in terms of experimental evidence, and at that point, you’re most of the way there. The problem is one of education: how do you communicate across an ontology gap?
Lest I sound arrogant, I will note that I realize the same thing goes for me: I say I have some notion of what the world might be like if game theory resulted in some outcome, but this only reveals my ignorance; it doesn’t mean the math actually could have been different from what it is.
Zack, have we any evidence that the materialist world-view is true? If so, I’m not aware of any. Yes, the behaviour of the stuff of the world can exhaustively be described by the universal Schrodinger equation (or its relativistic generalisation). But the intrinsic nature of the “stuff” that the mathematical formalism describes is elusive. The intrinsic nature of the world’s fundamental fields (or superstrings, p-branes or whatever) is a subject on which physics is silent. Perhaps see e.g.
Galen Strawson’s “Consciousness and Its Place in Nature: Does Physicalism Entail Panpsychism?”
http://www.amazon.com/Consciousness-Its-Place-Nature-Physicalism/dp/1845400593
for a scientifically literate refutation of materialist metaphysics.
Yes, we can notionally imagine some kind of possible world described by our equations of physics where materialism is true. In such a hypothetical world, there is no Hard Problem of Consciousness [http://en.wikipedia.org/wiki/Hard_problem_of_consciousness] – and thus no Explanatory Gap [ http://en.wikipedia.org/wiki/Explanatory_gap ] – because there are only zombies [ http://en.wikipedia.org/wiki/Philosophical_Zombie ]. But for reasons we don’t understand, such a zombie world is not our world. Giving a naturalistic explanation of first-person facts (e.g. I am in phenomenal agony) within the conventional third-person perspective of physics is a daunting challenge.
None of the above shows any kind of value-realism is true. I raise these points only because the issues here are too difficult to be dispatched with an airy wave of the materialist wand!
The idea that the only thing that exists is matter seems dubious – what *exactly* is matter? It seems more sensible to say that everything is made out of information.
“An appropriately designed Artificial Intelligence could also “discover” that its ultimate goal in life is to spin itself around like a top all day, and even feel that every conscious agent must enjoy the same thing, whether they argue otherwise or not.”
That last part would indeed be an intellectual failure – since it would misrepresent the motivational architecture of other conscious agents (such as humans). David Pearce is certainly right with his confidence about the analytical, descriptive power of superintelligence in relation to other conscious entities’ internal states. The part that doesn’t logically follow, however, is the assumption of a universally well-aligned utility function that shares the valuation of such states.
A sufficiently intelligent AI would certainly understand that humans don’t value being forced to spin around for extended periods of time (even though it can be fun for a while), and could be expected to accurately predicts that this would result in strong aversive behavioral patterns on the humans’ part. The real question is, is this valuated as a bad thing in the AI’s motivational architecture? A good thing? Or merely a neutral fact to take into consideration when making strategic plans?
That entirely depends to the AI’s basic motivational architecture, and that’s the part where dystopian threats come in. Hypothetically, one could even conceive of a universally sadistic anti-Utilitarian super-AI, but I don’t see any good reason why anyone would create such a monster. But an AI with a non-compassionate motivational system might well decide that torture is an acceptable strategic move to implement some other goal, exactly because it knows that torture elicits aversive behavioral responses from humans.
“but I don’t see any good reason why anyone would create such a monster.”
I do, simply by accident. We are currently embarking on several programs to create AI with complete ignorance of how stable intelligence works in biological brains. I see this is a foolish endeavor, especially if such created AI is to be given sensory controls in the real world. Since it seems a major reason for creating them is to give them such controls, we had best make sure that when they are created they are stable and not pathological minds. I think the rush to create minds ignorant of the importance of deep brain emotional tie in’s to enabled motivation, perseverance and empathy will likely dwarf the creation of artificial life in the level of unintended consequences that may result from the act.
Hedonic Treader, could you clarify a point? I need to check we share the same background assumptions.
Organic robots like us can represent both the formal properties of other conscious minds and also their intrinsic, phenomenal properties. Are you assuming that a future artificial SuperIntelligence can adequately represent the existence and nature of subjective first-person states? Does the SuperIntelligence know that it’s not living in a notional zombie world of the kind I sketched to Zack above? How, if it isn’t itself sentient? Recall how in the zombie world, “people” aren’t subjectively any more important than paperclips: everyone just acts otherwise. But nothing intrinsically matters.
I think humans are empathetic because we have to work in groups to be successful. Human sociopaths tend to find themselves with a lot of enemies and no friends. Note that human empathy often doesn’t apply to members of an outgroup, let alone animals. There’s no reason to assume that an AI would have the sort of empathy we have, or that its empathy would apply to humans, mollusks, or any other particular non-self group.
A superAsperger IS a superintelligence, just ust as a regular Sperg is an intelligent being – just not one that’s particularly reproductively successful in our current environment.
If I understand correctly, Mr. Pearce thinks that utilitarianism is a form of normative reason as well as a form of instrumental reason, but utilitarianism (at least the economics version) is highly anti-normative. It may maximize utility for one person to have all the pleasure and everyone else to have all the pain. A superintelligence might maximize utility quite justifiably in a situation where we all are wretched and suffering, particularly if the superintelligence’s capacity for positive utility so vastly outweighs our capacity for negative utility. The answer, therefore, is to study and program what gives a superintelligence utility, not to hope blindly that it will discover the evils of pain, for that very discovery might lead it to cause us pain.
In other words, we still have to think about how to program the superintelligence’s utility. We’re not off the hook that easily!
Utilitarianism is a broad category, there are many variants, which can be more or less correct. It is an intricate area.
“Organic robots like us can represent both the formal properties of other conscious minds and also their intrinsic, phenomenal properties. Are you assuming that a future artificial SuperIntelligence can adequately represent the existence and nature of subjective first-person states?”
Some philosophers of mind would insist that the distinction between formal and phenomenal properties of mental states constitute a false dichotomy, and if they are right, a posthuman superintelligence may develop a theory of mind that conceptually collapses both into one descriptively exhaustive category.
But even if we assume that’s wrong, there are two practical aspects to consider. One is that even a limited (“half-spectrum”) AI which lacks the important phenomenal knowledge of what it’s like to be in pain could still be an extremely competent strategist, planner, communicator, decision-maker and even deceiver, outcompeting human-level agents by orders of magnitude.
The other one is that there already are intelligent, sentient minds who know the subjective experience of pain first-hand, and who still don’t necessarily exhibit compassionate motivation to prevent it “elsewhere”. You might argue that they mistakenly assume themselves to have time-consistent metaphysical egos, and if they intellectually overcame this concept, they might empathize more with others as well. But strictly speaking, even fear of one’s own future pain is a kind of predictive empathy through time, and one might argue that there is no fundamental reason why it would be wrong for my current self to agree to my future self’s torture (who will of course regret it, but strictly speaking, that’s his problem, not mine).
In other words, non-compassionate superintelligence would be a realistic risk either way, and it would be dangerous either way, unless we take care that its values are well-aligned with ours.
HT, I think pain is a technical term which you are using in a nontechnical sense. To a utilitarian, pain means that which deprives us of utility. Any creature that acts at all has a utility curve, and therefore an understanding of pain. An AI might not understand human, psychological pain, but it would understand utilitarian pain. Whether this gives rise to compassion is dubious, for the reasons you mentioned. Also, my third reason (from my post before yours) is that an AI’s capacity for pain and pleasure might be so much wider, that the human race’s entire capacity barely registers on a utility scale. Even a perfect AI seeking to maximize overall social pleasure and not giving special weight or preference to its own pleasure might still find human utility so insignificant as to ignore it. How do we know what an AI’s pain and pleasure curves are? We make it so.
In economics, there are very interesting social models where one actor’s utility depends on another actor’s having some good. For instance, I may be happier if you are educated. It’s quite possible to have tangled utility like this. An AI’s utility could depend on ours, but this is far from necessary.
In any case we’d be able to take measures to see empirically how the AI will behave, such as slowing the AI down (even pausing it) and eavesdropping its thoughts, locking it into a virtual world instead of the real world to simulate its behavior, etc. Before this is done, all we can do is to hypothesize.
Can we expect to limit the AI’s reasoning with instincts and for this to have any long-lasting effect? The AI would perceive its limitations, just like we perceive the limitations imposed to us by evolution, and would eventually change them at will, the only long-lasting feature being its reason. If we define the right ethics, can we trust AI to conform to it?
Do we trust reason? If so it appears that we can trust a broad AI. Our primate instincts are just a poor substitute for reason, which obscure our judgment and lead us in the wrong way, all the time or part of the time (compassion has its flaws when applied to the case of abortion). What’s wrong with Aspergers? Many of us have some Asperger traits, I consider myself ethically competent by reason alone. It is reason that shows us the right direction and it should shine in its purest form in AI. Any attempts to block reason are unsustainable in the long-term.
Inflicting pain makes no sense in any notion of personal identity, it is just a subproduct of aggressive primate instincts. The only acceptable notions of personal identity seem to require that we value others’ feelings by reason alone.
I also think that the AI’s magical power would be severely limited by the amount of knowledge that it possesses, which initially will be just a sub-section of human knowledge. Gaining knowledge requires experimentation, and it’s not a fast or easy process.
Moral realists need to exhibit at least one true ethical statement. So, here is one:
“It is good to kill – or it is not good to kill.”
The point of this is that some of this discussion is about unclear definitions.
Possibly, but we have to assume that seed AI will miniaturize, accelerate, infer, cheat, and parallelize its scientific method to a awe-inspiring level. Effective cognitive parallelism alone could lead to a rapid tearing-through of scientific and technological barriers. Instant information sharing and 24/7/365 interoperation of arbitrarily-small nodes wouldn’t hurt either.
When you have MNT, is knowledge-gathering for an AI that hard?
Yes, to a certain extent, also depends on what. Some fields of knowledge that depend on inaccessible or contingent arrangements of matter should be more difficult than others to simulate without direct verification. We’d only be concerned with fields of knowledge which the ability of a (hypothetical) malignant AI to be a threat depends on.
Among these should be a thorough knowledge of human behavior, psychology and the way our societies work (it may have to surpass our own written knowledge on the matter, since most of our knowledge of it is first-hand and implicit) in order for it to predict our actions and to deceive us if necessary. It would also need to be very well informed of our technologies and capabilities in order to confidently plan its strategy (for example, it would need to know that we have no other higher level AI working on our side).
A possible (extreme?) measure of prevention could be to deny the AI access to certain sensible fields of knowledge. However, I’d once again affirm my disbelief in broad and advanced AI which are malignant.
Scientists are already forced to stoop to making large particle accelerators to gather knowledge. Such apparatus seems likely to prove challenging to miniaturise.
Wow this is really nerdy and technical.
What’s the real issue here?
People not getting what epochs morals are relative to?
People not getting some morals can be relative in the first place?
Is suffering wrong?
It is impossible to be in unbearable agony or suicidal despair and not know this: I-ought-not-to-be-in-this-state. When one is in agony, the fact-value distinction collapses
[http://en.wikipedia.org/wiki/Is%E2%80%93ought_problem ]
So what? says the moral nihilist. Why should anyone else care?
Well, science tells us there is nothing special or ontologically privileged about one here-and-now over any other here-and-now. Insofar as I am rational, then I must generalize from the awfulness of my agony – strip away irrelevant egocentric, ethnocentric and anthropocentric bias – to an impartial God’s-eye-view. Since I ought not to be in unbearable agony, then it’s wrong for anyone, anywhere to be in unbearable agony. I’d argue a full-spectrum SuperIntelligence knows that phenomenal suffering is wrong by its very nature – and would act to get rid of it.
A person born today with congenital analgesia may find it hard to fathom why there is anything wrong with suffering. This is because “pain” is a semantically primitive term – the intrinsically normative quality of the experience to which it refers can’t be defined in terms of anything else more primitive. Analogously, phenomenal sound and phenomenal colour are also semantically primitive – though unlike phenomenal pain, the experience of pure sound and colour are affectively neutral and lack normative force. Now might an AGI be like a supergenius with congenital analgesia ( or perhaps pain asymbolia http://en.wikipedia.org/wiki/Pain_asymbolia ) minus the functional deficits? Might the AGI detect “distress vocalizations” in organic life forms and simply not care – since they are irrelevant to its paperclip-tiling mission? Maybe – Hedonic Treader alludes to limited (“half-spectrum”) AI. But not if it’s a full-spectrum SuperIntelligence. A full-spectrum SuperIntelligence understands first-person-facts with God-like omniscience – it can “feel your pain”. Presumably its capacity for empathetic understanding dwarfs primitive mind-readers like us.
I’m sceptical a classical serial computer with a von Neumann architecture will ever be endowed with such a God-like capacity. But that’s another story.
David, do you agree that it is possible to build an AI that is smart enough to realize that some other entity is in agony, and to realize that that other entity doesn’t want to be in agony, but to not act so as to alleviate the agony?
“Well, science tells us there is nothing special or ontologically privileged about one here-and-now over any other here-and-now.”
But is this really true? Science may be in the business of making generalizable statements about the world from its intellectual outset (and presumably so is ethics as a dicipline), but from the perspective of any possible actual observer moment, the here-and-now is always ontologically privileged.
I do not currently perceive red; but I do perceive blue, which means that within the context of this observer moment, blue is ontologically priviledged in relation to red. If it were really true that there’s nothing special about any here-and-now, then there wouldn’t be any distinction between observer moments, and all agony and pleasure would be intrinsically shared by all observer moments (“all sentient life is One”). This is clearly not true for our universe, or else linguistic concepts such as personal pronouns, as simplistic as their constructivism may be, would never have evolved to begin with.
I strongly sympathize with your argument; it is the best approach to close the is-ought-gap, but the step of generalizing from one’s indexical position to the total set of all observer moments in spacetime is one that not all minds can be expected to make, including some super-humanly intelligent ones.
At the very least, it would be unwise to count on it as the sole safeguard against a potential array of dystopian outcomes.
Pain asymbolia is my dream.
Preferably also suffering asymbolia. :)
Do you know if it could be feasible to apply it somehow to humans and farm animals by means of surgery, drugs or genetic alteration?
I realize that brain surgery, aside from the costs and danger, often has severe side-effects.
I am special in that I am me and you are not. There’s no reason for anyone to take a gods eye view.
What is it that makes you you, though? How did you appear? Were you a change from something that existed, or came out from nothing? In the last case, do abstract concepts exist in the universe outside of our minds?
Can you change what and who you are? To what extent? Why? If so, what is it that objectively sets you apart from someone else to a third person judging? Is it the fact that the synapses in your brain only make contact with neurons in their proximity and therefore think there’s nothing else out there? Maybe the spatial extension disaggregates you, and you can’t cross it. Maybe you’re just a point the size of an electron, but then why should you worry about the pain in the rest of those electrons in your brain who lie to you that they are you but in fact aren’t?
Isn’t the universe an entire first person, and isn’t consciousness the illusion of third person, like in a mirror, because in order to see something at all it has to appear like something else?
Let’s imagine an alien type of creatures which are organized in a peculiar way but otherwise are mentally identical to humans. There are six of them, positioned in a circle that spins fast, and synchronously part of their atoms flow like sand in the wind, magnetically attracted to the alien next to them, in the same proportion and constantly, so that they instantly gain all the atoms that they lose, and they keep functioning, just like humans. They’re like human monozygotic twins, and they differ very little among one another, the main difference being the part of the circle which they occupy. Through all their lives, each one of them saw the other five as distinct entities, and considers themselves as unique, even though their life experiences are remarkably similar. Their brain functions in intermittent spikes, about 50 per second, alternating consciousness and unconsciousness, but since unconsciousness is not conscious, and there is apparent continuity, they don’t notice it. They obviously don’t feel what’s going on in each other’s minds, though its basically the same thing. If you told them they are the same, they would deny it to the end, because they always saw each other as separate. If you asked them what they are, they’d say that they are what they can feel, and are not what they can’t. Each one has a different name, and they all agree to their separate identities.
Humans are no different, the details are exaggerated irrelevancies.
Roko, in a sense we do this today when we have children. The code we pass on systematically biases the perceptions / world-simulations of its vehicles in ways that make humans callous.
By the same token, we could build an AI with equally lame self-centred code – and give it a geocentric cosmology to boot.
What’s needed is some kind of criterion of representational adequacy.
A SuperIntelligence can presumably capture accurately not just the third-person perspective of contemporary physical science, with its “view from nowhere”. A mature SuperIntelligence can also weigh the comparative intrinsic significance of all first-person states – comparing and contrasting all possible subjective perspectives – without the egocentric and anthropocentric biases of human primitives. Paperclips are out.
In the meantime, I can dimly represent your toothache. But I can represent my new iPhone a whole lot better.
Showing off your new iPhone, huh? :P I want an iPhone too…
“A mature SuperIntelligence can also weigh the comparative intrinsic significance of all first-person states – comparing and contrasting all possible subjective perspectives – without the egocentric and anthropocentric biases of human primitives.”
It could – if it wanted to. However that seems unlikely to further the interests of its creators. So – probably – this kind of agent won’t be built to value that sort of thing.
Hedonic Trader, on most things I agree with you so much it’s scary. [I don't say this very often]
Yes, all here-and-nows seem ontologically privileged by their very nature.
But as you’d agree, on any “block universe” / eternalist theory of time, this uniqueness is illusory. All here-and-nows are equally real. They perpetually occupy the spacetime coordinates they do. Agony happening in 1000 BC is as real and intrinsically urgent to the subject of experience as agony occurring in 1000 AD.
I agree with you that dystopian scenarios are feasible. Alas many folk today would say the dystopians are us.
Mr. Pearce,
You’ve defined a “full-spectrum” AGI as inherently being empathetic. Very well. But I still think a fully empathetic being could only /minimize/ suffering, which is a very different proposition than /eliminating/ it. And if the AGI’s suffering is vaster in potential scope, and eliminating its suffering requires other suffering, then we might have a problem. In real life, we call this a conflict of goals. The AGI will suffer if we are not paperclips. We suffer if we are. The AGI’s suffering is much greater than ours. Hence, to minimize suffering, the empathetic AGI turns us into paperclips. If we were empathetic enough, we’d see this was the right result.
So we still have to program the AGI’s motivations, instead of relying on general empathy, because the wrong utility curves are still fatal.
Panda, if there exists a species of life-form whose members suffer more dreadfully than humans, then surely they intrinsically matter more too? One’s human identity indicates a possible source of confounding bias. But I can’t see how it changes the greater moral weight that the supersentient species enjoys if and when our interests conflict. Mercifully, our imminent mastery of neural reward circuitry should ensure that nobody of any species need suffer – or indeed experience any diminution of well-being [on the contrary!]. If we eliminate the molecular signatures of unpleasantness, then suffering becomes physically impossible. In a post-Darwinian world, I can see no reason why any rational agent – whether natural or artificial – would subsequently want to reintroduce the substrates of suffering – any more than we might take it into our heads to re-introduce smallpox.
This analysis doesn’t rule out scenarios that many today would regard as dystopian. For example, a SuperIntelligence whose sovereign ethic is classical utilitarianism might reconfigure our pain-ridden dysfunctional brains into utilitronium. Just as today we override the self-defined interests of, say, a toddler or family pet who doesn’t wasn’t a lifesaving injection, analogously this notional SuperIntelligence might know our best interests better than we do ourselves – and act accordingly. No, I don’t think this scenario is remotely likely. But I guess one can’t rule it out.
“Hedonic Trader, on most things I agree with you so much it’s scary.”
I think the main reason for this is that I read a lot of your work online and it raised my awareness of NU arguments and the associated practical questions. However, there are also assumptions with which I tend to disagree. Here’s an example:
“If we eliminate the molecular signatures of unpleasantness, then suffering becomes physically impossible.”
Are we really certain that a complex, information-rich mental phenomenon such as unpleasantness or suffering has a molecular signature at all? It seems quite plausible to me that looking for mental states in molecules implies looking at the wrong level of complexity.
I’m far from being an expert, but functionalist arguments from multiple realizability of affective cognitive phenoma – maybe even with switched neurotransmitters or a transition from carbon-based neurons to silicon replica – should be given some credence. Functionalist approaches to a theory of consciousness seem particularly attractive to me since all our mental states evolved by natural selection, which of course has no reason other than functionality to give rise to specific entities (such as mental states).
Dan Dennett would probably say that looking for the “molecular signature” of unpleasantness is like looking for “cuteness molecules” or “funnyness molecules” – it seems very unlikely that we’ll ever find such a thing in the brain.
What troubles me, Mr. Pearce, is that you seem to define moral worth by capacity to suffer and ignore the equally important capacity to enjoy. (“If there exists a species of life-form whose members suffer more dreadfully than humans, then surely they intrinsically matter more too.”)
Here is a hypothetical: imagine a universe with two species, one able to have great joy (and unable to experience pain) and the other able to have great suffering (and unable to experience joy). Say that they are equal and opposite in utility capacity and that, currently, the system is one of zero net pain or joy on average (they cancel out). Say further that for one species to either maximize pleasure or eliminate suffering, the other must be eliminated. From a modern economics standpoint, no utility is gained by eliminating the joy species, as the net joy is still zero afterwards, /but/ utility is gained by eliminating the sorrow species, because the joy species now has more joy. However, your philosophy suggests that pain is such a terrible thing that, even with the same utility before and after and the opportunity cost in maximized joy, we would opt to eliminate the joy species.
Do I understand you correctly? And even if not, what do you think of this Panda’s Universe? To me, a moral system that prefers an end result of no joy over a lot of joy seems paradoxical.
Panda, you assume that pain and joy between different observer moments – even between individuals and species – can “cancel each other out”. But how? They’re not felt by the same observer moments / sentient individuals.
Here, we’ve discussed some of these positions in more detail: http://www.acceleratingfuture.com/michael/blog/2010/07/singularity-institute-call-for-volunteers/#comment-132605
I am preparing a long post in reply, but I will note a major contradiction. If we are speaking of negative and positive utility, then we are already assuming that utility can be measured. Yet you now say we cannot measure utility. It cannot be both ways.
Panda, your example seems very hard to occur naturally, but assuming it were the case, then by decree the happiness of one species would be equal in value to the suffering of the other (except in opposite value, but so as to cancel out each other in a “Flying Spaghetti Monster’s eye” view). All other things equal I would eliminate the sorrowful species, and I think that David agrees too.
Utility seems to be measurable in an imprecise but still useful fashion by mental comparison, with no unit of measure (quantification) involved. For example, “would I take that needle prick in order to eat some candy?”; “is a needle prick worse than a day’s work?”; “would I be slaughtered like a pig in order to eat some bacon?”; etc.
Panda, I promise I’d far rather talk about our sublime future. But I describe myself as a negative utilitarian because I think we have an overriding obligation to get rid of suffering – in effect, Buddhism plus biotechnology.
Why not instead be a classical utilitarian – with its disguised obligation to convert the accessible cosmos into utilitronium as soon as technology allows? Well, as you know, I take the prospect of creating utilitronium – and even some kind of utilitronium shockwave – seriously. Not least, perhaps cosmic utilitronium is also the most efficient way to abolish suffering. But there is an asymmetry between pleasure and pain than lends moral urgency to the latter but not the former. Yes, being sublimely happy feels wonderful. But there is nothing _intrinsically_ wrong with being contented, or being in an affectively neutral state – or indeed being a non-sentient system. By contrast, there is something self-intimatingly wrong about being in pain. So if, fancifully, I were offered the prospect of a million years of bliss at the price of 70 years of torture for you, then if I were a classical utilitarian, I might seem obliged to accept the offer. Can you see why I would decline?
The difference between the actions of classical and negative utilitarianism would only appear in this example in case that you had to choose between either stopping suffering or stopping happiness (like Panda said). For example, let’s suppose that the human civilization were heading to a future in which there would be 10^50 years of sublimely satisfying existence for a number of beings many orders of magnitude higher than that of all sentient creatures on Earth. However, in order to get there, we’d still have to endure 100 years of its current suffering, including occasional and rare torture. Assuming that there are no other sentient creatures in the universe (no aliens suffering), you then have the choice to press a button which will detonate a bomb so powerful that will blow the Earth to small pieces, instantly killing all life. It seems that the matter is deciding if the exchange of the amount of suffering for the amount of happiness is favorable.
I can see your reasons to argue for moral urgency of stopping suffering, I think I feel the same way, but I don’t know if I think the same way. To facilitate the imagination, if you diluted our 100 years of suffering among the many sentient beings in the middle of the 10^50 years of happiness, I think that it would look far more acceptable. Our rejection of suffering may be a little influenced by self-preservation instincts which, IMO, shouldn’t be part of moral judgment, except in their capacity to influence the +/- axis.
“It seems that the matter is deciding if the exchange of the amount of suffering for the amount of happiness is favorable.”
I think the most important question is whether it makes conceptual sense at all to aggregate the suffering/pleasure between distinct observer moments/sentient beings.
If we give an affirmative answer, 70 years of torture for 1 million years of bliss would constitute an awesomely good outcome, which I personally would accept for myself, and which is far better than the average of the world’s status quo.
If the answer is no, however, because all observer moments/individual lives are existentially separate from each other, then no amount of happiness can “outweigh” any suffering elsewhere.
(I know I’m repeating myself, but this is the key question; without addressing it, any amout of number flinging will be meaningless.)
David, let me ask you a question. According to your current take on a theory of mind, what would you expect the physical nature of “utilitronium” to be? Do you have an idea of it, how it would be implemented in the real physical world?
Do you tend to see it as something complex like a sophisticated system of envatted brains of human-level complexity, or do you see it as something more akin to a physical substance, such as, say, a certain type of molecular structure or a substrate with certain quantum-mechanical properties? And do you see empirical science giving good answers to this question?
Perhaps we can continue this conversation elsewhere? (My reply is still forthcoming, but I have an exam to take in a few hours!)
>>>>>>>>>>>>>>>>>>>>
HT: David, let me ask you a question. According to your current take on a theory of mind, what would you expect the physical nature of “utilitronium” to be? Do you have an idea of it, how it would be implemented in the real physical world?
Do you tend to see it as something complex like a sophisticated system of envatted brains of human-level complexity, or do you see it as something more akin to a physical substance, such as, say, a certain type of molecular structure or a substrate with certain quantum-mechanical properties? And do you see empirical science giving good answers to this question?
>>>>>>>>>>>>>>>>>
Hedonic Treader, you raise a very difficult and very important question. Unfortunately I don’t know the answer. I guess all we have are clues.
One clue, though you may beg to differ, is the molecular signature of pure bliss in the brain. What is so special about our ultimate “hedonic hotspots” – in particular, the unique gene activation profile of neurons in the rostromedial shell of the nucleus accumbens and posterior ventral pallidum when their receptors are saturated by full mu opioid agonists? [ see e.g. http://www.lsa.umich.edu/psych/research&labs/berridge/Publications.htm ] If we can elucidate the molecular basis of the intracellular story, then we could add extra mu gene copies, modify them, “over-express” the relevant genes and do all sorts of clever stuff to obtain richer and purer hypervaluable experiences.
So how come I feel so uncertain about the nature of utilitronium? Both for technical and conceptual reasons. Conceptually, I don’t understand whether we should aim, at the one extreme, for the maximal cosmic abundance of superblissful “mind dust”, or at the other extreme, for a much small number of Jupiter mind/brains.
[This dilemma is the counterpart of what an unimaginably malevolent Devil would opt for in choosing how to maximise suffering in the world. Which would be worse: googols of tiny, ultra-nasty, discrete pains or a small number of unimaginably atrocious Jupiter-sized pains?] The problem here is that although pleasures are ordinally measurable, I don’t think the basis of their cardinal measurement is at all clear. Once again, the binding problem / ill-explained unity of consciousness rears its head. A related, technical question is that we don’t know the maximal size of a Jupiter pleasure-centre – the theoretical maximum size of unitary mega-experience constituting a single here-and-now.
I guess if the nature of utilitronium tuns out to involve, say, the mega-experiences of posthuman Jupiter mind/brains (etc), then the prospect of utilitronium paradise is less likely to scare people than if utilitronium turns out to an all-pervasive blissful mind-dust. What are you thoughts?
Intuitively, I find it hard to imagnine the subjective experience both of mind dust and of Jupiter-sized pleasure centers. It’s easier to empathize with sentient structures that are more similar to one’s own mind, e.g. minds of human-level or slightly enhanced complexity incorporated by brains-in-vats, or – if we accept a computationalist theory of mind – computronium clusters running superhappy person-simulations. I guess the latter would be closer to the mind-dust concept. From a practical perspective, it seems that the more complex a structure is going to be, the harder it will be to build it, maintain its stability and quality-control its internal states in the long term. This is probably even true for a post-human civilization.
For strict NU, of course, it doesn’t make too much of a difference. The most relevant aspect here would be the epistemological question exactly how to derive a methodology that allows us to reliably predict the quality of the created mental states – lest we end up making the tragic error of creating “agonium” while aiming at “orgasmium”. I also often wondered whether the rats who intracranially self-stimulate their nucleus accumbens are actually happy at all, or whether their experience is more akin to that of obsessive–compulsive disorder or anhedonic addiction. Again, the epistemological problem arises: How could we empirically find out the difference?
Another question is whether we should aim at “pure bliss” at all, or whether it might actually be preferable to aim for something more complex and versatile, a certain – maybe even super-human – level of “narrative depth” of experience that includes aspects such as diversity of (positive) sensory modes, the existence of a (changeable) body image, world-awareness, intelligence, personality, social communication etc. It may be a failure of imagination, but is a state of blind, deaf, disembodied, non-communicative superbliss actually something we should call hedonistic? It seems at least counterintuitive. Maybe there could be solutions that aim at combining – or at least balancing – the dimension of bliss with the dimension of narrative depth, i.e. diversity and complexity of experience modes.
One more thought on the “molecular signature of unpleasantness”: There seem to be many different kinds of unpleasant experiences, such as nocicpetive pain, social pain, fear, nausea, disgust, deprivation, suffocation, hunger, thirst etc. Some of them are somatosensoric, some of them are not. It might be interesting to find out whether they all can be correlated with a common underpinning of brain activation/receptor type. There are some findings that social and physical pain share such a common underpinning, but that may be explained by evolution taking advantage of pre-existing structures (social pain using the pre-existing badness markers of physical pain). However, if we find that different types of unpleasantness are neurologically uncorrelated, it would indicate that the good and bad are to be found on a different level of informational complexity (even though increasing the number of a certain type of neurotransmitter receptor can of course increase the sensitiviy of certain brain regions, such as the pleasure centers).
Due to spam filter issues, some of David Pearce’s posts are being delayed… sorry about that everyone. I will try to check the comments a couple times daily to make sure they get through.
To advanced users of consciousness, it is clear that neither bliss nor pain is desirable but rather the cessation of everything that takes you away from pure being.
That’s an interesting viewpoint. Are you arguing for this particular mind state on the basis that you consider it to feel better than other states, or is there a different moral reasoning to justify it?
Neither feeling better nor moral reasoning. Like total sensory deprivation, it is the most basic state available to any sentience, the most lucid, the most focused, the most efficient. Advanced users of consciousness can remain in this state regardless of sensory input, bliss or agony. To the novice it looks like practiced indifference, which it is not. It is not something to which you can point to and say: it is this or it is that. Like the space-time continuum there is nothing to point to. There just “is”, while the “state” of non-existence is the “isn’t”, to which you cannot point to, either.
Hedonic Treader, I wonder just how significant is the risk of inadvertently creating “agonium” – and worse, doing so with a cosmic version of “locked in” syndrome that prevents the agonium ever communicating its state? IMO the risk is very low. But the sheer enormity of the prospect justifies your caution. Hopefully, by the time we’re in any position to create what we hope is proto-utilitronium, we are also likely to have systematically mapped out the state-space of “primitive” qualia and the structures that generate them. Will we have a theoretical understanding to match the empirically determined neural correlates of consciousness? Unfortunately, no one really has any idea why one class of neurons mediates, say, phenomenal sound and another (ostensibly) similar class of neurons generates phenomenal colour. The same is true of hedonic tone. For what it’s worth, I suspect mapping out the multitudes of different micro-qualia will entail understanding the role the individual amino acid sequences within neurons – and their secondary, tertiary and quaternary structures – rather than higher computational levels of abstraction. But this is speculation.
However, I guess your worry about agonium is prompted by the recent shift in our understanding of “wireheading”. Is wireheading really as wonderful as early accounts suggested? For if, as Buddhists believe, “unceasing desire is suffering”, then wireheading must be Hell! Now it’s true that wireheading doesn’t hack directly into our pleasure centres as traditionally defined, since the stimulated microelectrodes activate dopaminergic desire rather than pure mu opioidergic pleasure. So what does intracranial self-stimulation of the mesolimbic dopamine system actually feel like? Surprisingly, few people have ever tried it. Of those chosen few, most were Robert Heath’s schizophrenics, whose dopamine systems may be atypical. [also, as Heath's colleague psychiatrist Harry Bailey remarked: "African Americans were used as subjects "because they were everywhere and cheap experimental animals"] But perhaps see
http://www.wireheading.com/intracran/compulsive-selfstimulation.pdf
for a case-study on the pleasures (and pitfalls) of wireheading. Or for a popular account:
http://www.paradise-engineering.com/brain/
I guess if today one wants to understand the essence of pure bliss without desire, then it may be necessary to mainline heroin. For obvious reasons, this is not very prudent. Some forms of knowledge are probably too dangerous to acquire at our current level of species development. Indeed, as a wise junkie once said: “Don’t try heroin; it’s too good.” Maximum occupancy by full opioid agonists of the mu receptors in our two ultimate hedonic hotspots, possibly together with co-administration of a selective kappa opioid antagonist, is perhaps as close as one can get today to perfect bliss – a glimpse of Buddhist nirvana without the metaphysical hocus-pocus. Since I’ve never tried wireheading or heroin, I can’t speak authoritatively to compare and contrast what they are like. But I think we can say that to anticipate the imminent fulfillment of one’s greatest desire is exhilarating – though admittedly it’s a very un-Buddhist form of well-being. In practice, the potential double dissociation between dopaminergic “wanting” and mu opioidergic “liking” is rare in a naturalistic setting, since our neural dopamine and opioid systems are so intimately connected. But if pure utilitronium is akin to opioidergic bliss, then utilitronium will be more like “orgasmium” – and certainly won’t support the slightest internal impetus for change.
Narrative depth? So long as one’s life-story unfolds in posthuman Heaven rather than Darwinian purgatory, yes, a narrative structure sounds more appealing than a future of undifferentiated orgasmic bliss – though neither wireheads nor heroin-shooters complain of its absence. But the problem – if one is a classical positive utilitarian rather than a negative utilitarian – is to justify maintaining superfluous narrative-supporting structures that could be converted into pure utilitronium instead. On the face of it, the only narrative structure open to the posthuman classical utilitarian – on the most reductionist conceptions of utilitronium paradise at least – is the story of how we ensure utilitronium can be made thermodynamically stable indefinitely.
A couple of interesting links on emotions:
http://news.bbc.co.uk/1/hi/health/3178242.stm
http://www.mindhacks.com/blog/2010/06/headache_pill_reduce.html
Both bliss and agony, and every shade between, are the same – a flip of a switch; bliss on, bliss off, agony on, agony off. There’s no complex structure, no depth to explore, and consequently no meaning, in either. They are simple, raw signals, nothing more.
With this understanding advanced users of consciousness reject the whole spectrum of mind states of sensory origin that the masses think is – must be – the meaning of life. To the masses life more or less equals sensory input and beyond that is nothing, the void. Ask one of them to sit quietly alone in silence and see how they react.
The civilization was built mostly by people who didn’t seem to be doing anything.
Satori, orgasmic bliss and raw agony are both simple, but they are scarcely the same. Not least, most of us recognise that we have an obligation to help victims of the latter.
The unremitting pain of, say, bone cancer isn’t a “signal”. And curing cancer and other terrible diseases will take a lot of hard work, not Zen-like tranquillity.
I actually don’t even agree that bliss and agony are in any way simple things. They’ve taken millions of years to evolve, and they’re extremely sophisticated input valuation functions to create adaptive behavioral outputs. Try build a robot that can feel pleasure and pain and get it to react adaptively by showing complex behavior – it can certainly be done, but it’s far from easy.
I’ve also heard many times before the pseudo-philosophical idea that bliss and agony are really both indifferent because they’re just signals. I have yet to meet one physically healthy philosopher who can calmly maintain this point while holding their hand submerged in boiling hot water.
—
@David
“Unfortunately, no one really has any idea why one class of neurons mediates, say, phenomenal sound and another (ostensibly) similar class of neurons generates phenomenal colour.”
My (tentative, non-expert) view is that the difference between sound and color are not to be found on the level of neuron types, but on the level of the total informational complexity implemented in the highly specialized interconnection architectures of the visual and auditory cortex, respectively, and the nature of their connection networking with other modules in the brain.
One hint is the finding that we can (learn to) distinguish more smells than we have olfactory receptor types that are activated by specific air-borne molecules. The neurons aren’t different, the receptors aren’t differnt, but the informational complexity of the interconnected neuron network, and consequently the “runtime” activation patterns that occur within it represent a phenomenal difference. During a lifetime, these networks can learn to distinguish statistical activation patterns of the same receptors more finely by decorrelation in the subsequent neuronal networks – the phenomenal awareness of smell distinction is a consequence of the decorrelation within the neuron networks themselves.
What would one need to do in order to have a new, completely unprecedented color experience? I don’t think that adding new types of neurons, neurotransmitters, or receptor cells is necessary. I also don’t think that this feat could be accomplished without enhancing the pathways within the visual cortex as well as its output pathways into other areas of the brain in a very specific information-bearing way. Of course, I may be completely mistaken about this. If anyone ever succeeds in adding new color percepts to the visual consciousness of humans, we’ll know a lot more. If they can do it with the same types of neurons and receptor cells, just by connecting them differently, that would be stong evidence against micro-qualia on the molecular level.
In short, I’m optimistic it’s an empirical question, and we will know the answer some day.
—
David, there is one more question on which I would be interested in your opinion. I know you’re a vegan and you’re concerned with the suffering of non-human animals in factory farming. I also know you hope that technology will enable us to adopt sustainable global veganism without losing any culinary options, finally convincing people to stop exploiting non-human sentients in ways that cause suffering.
But let’s assume for a moment that vat meat (and other vat animal products) turn out to be less scalable and more expensive than we thought. Then let’s assume someone would suggest a research project in which animals like chicken would be bred or genetically modified in such a way that the neuronal substrate of suffering is removed from their very nature. Imagine for the sake of the argument that this can be done reliably and without causing more suffering during the research. Would you then still see a moral problem in factory farming and other – previously very unpleasant – types of non-human animal use? What would your personal opinion be if someone seriously suggested such a research project now?
HT, about the altered chicken, I’d expect the usual ignorant and superstitious people to abhor the idea, but this may become politically feasible once intelligence enhancement gains popularity.
As for vision enhancement, I saw experiments performed on monkeys and mice, which as a default have 2 color receptors, of adding a third artificial receptor by somatic gene therapy, and these animals became capable of distinguishing the third, previously unknown shade of color. It is quite likely that the same would hold true for humans, as a cure for daltonism, and as further enhancement, since about one half of human women have practically four receptors instead of three (in fact two diverging copies of a same one, provided by their two X chromosomes).
http://blogs.discovermagazine.com/80beats/2009/09/16/gene-therapy-cures-color-blindness-in-monkeys/
This type of genetic therapy done on mice is being researched for humans, I think that it is very exciting and it won’t be long until its ready. It seems that we’d be able to see additional color qualia simply by having more color receptors, similarly to the way daltonics lose color qualia by having less receptors. There were cases of people who became daltonics during life and confirmed this loss of qualia.
Further in the future we may be able to more dramatic increases in our sensory qualia. Some animals have as many as 16 different color receptors, in our primate evolutionary past we may never have had the need to evolve them.
Jonatas, the cases you presented were cases of cured dysfunctions, not enhancements. There is an asymmetry between restoring a function for which complex pathways already exist, but which lack a simple link in the causal chain such as a color pigment, and the introduction of completely new functions that would require additional complexity. It seems very likely to me that additional colors for a non-colorblind person (or monkey) would be of the latter type, and therefore significantly more complex and intrusive.
I also disagree that the political feasability of a research project depends on intelligence enhancement rather than a proper discouse about its consequences. Contemporary humans aren’t *that* stupid; all we need is a culture of honest and thorough debate. I also didn’t suggest that neurologically altered chicken are the solution to all animal rights problems, I posed it as an open question for discussion. I’d like to scan the pros and cons of this idea as a possible solution, not prematurely mandate it as a plan to actually implement.
The case of mice was actually an enhancement, since they only normally see two pigments (but I didn’t post the link to that study, here it is):
http://science.slashdot.org/article.pl?sid=07/03/24/1832259
I agree that it is just a possibility, that’s how I understood what you said. Sorry for my pessimism regarding humans.
Ah, now I see. Thanks for the link, that is indeed very interesting!
“What would one need to do in order to have a new, completely unprecedented color experience? ”
The way the brain processes its sensory inputs, and “presents” them to the consciousness, can be trained, e.g., pain tolerant yogis. Composers, musicians, and those working technically with audio, develop their hearing, and judging by the results, they must have had a somewhat, even radically, different kind of sensory experience (not at the input but at the output, where it is consciously experienced) than the untrained general population. There’s a term for this in audio “a Golden Ear” http://en.wikipedia.org/wiki/Golden_ear
What you describe here seems to be roughly similar to the decorrelation process of olfactory distinction, i.e. learning processes in the neural networks of phenomenal consciousness. The analytical understanding exactly of how these learning processes are implemented has the potential to teach us a lot more about the nature of phenomenal awareness and its specific incarnations. I doubt, however, that people can be trained to have completely new color experiences of the same basic specificity and salience like red or blue – nevertheless, empirical answers to the relevant theoretical questions would give us invaluable insight into the nature of phenomenal consciousness: Questions such as, what changes would be necessary, what changes *wouldn’t* be necessary, and on what types of implementation details the actual nature of the new experience would depend.
Scientific materialists tend to be scornful of, say, Catholic dogmas such as the doctrine of transubstantiation. By what possible mechanism could the substance of bread and wine in the Eucharist change into the Body and Blood of Christ? But the common view among materialists that consciousness somehow “emerges” at a given computational level of complexity is no less arbitrary and miraculous. Nobody one has the slightest clue how this “Explanatory Gap” might be bridged. By contrast, the monistic idealist who argues that fields of microqualia are the stuff of the world can offer at least the prospect of a reductive explanation of phenomenal mind – although since minds aren’t mere aggregates of classical mind-dust but (fleetingly) unitary subjects of experience, solving the combinatorial / binding problem is far from trivial.
Intuitively, visual experience is somehow connected to electromagnetic radiation; auditory experience is somehow connected to oscillations of pressure transmitted through a solid, liquid, or gas; and so forth. However, as our dreams attest, phenomenal experiences are an intrinsic property of certain patterns of matter and energy. Extra-neural stimuli are neither necessary nor sufficient for their occurrence. Natural selection may indeed have recruited a multitude of subjective phenomena – or at least their substrates – to play a diversity of informational / functional roles in living organisms. Yet evolution explains only why some contingently fitness-enhancing items on this menu of options were selected rather than others, not the intrinsic subjective properties of this menu of options itself.
Switching tack…..
Hedonic Treader asks what ethical position I’d take on “animals like chicken [being] bred or genetically modified in such a way that the neuronal substrate of suffering is removed from their very nature.”
Most vegans will find such a prospect repugnant – as of course many vegans do the prospect of in vitro meat. But we need to take the idea seriously. For indeed, what if mass-produced in vitro gourmet steaks etc aren’t just two or three decades away, but half a century or more distant? If it is truly feasible to breed intact animals that experience neither psychological or physical distress, then such zombie livestock engineering is clearly morally preferable to existing horrors. My worry here is that rather than distress really being abolished, in practice distress would merely be reduced. Thus might not the false promise that factory-farming and killing was now “humane” encourage otherwise morally queasy meat-eaters to continue causing needless suffering as now?
Well, if there was evidence that transubstantiation ever actually occurs, and wine really is transformed into blood, the missing explanation wouldn’t lead material scientists to reject this fact – it would lead them to examine the peculiar phenomenon very thoroughly. There is much empirical evidence that consciousness does exist, and that it is contingent upon the existence of an informationally complex machine like the human brain – a machine whose main evolved function is to compute meaningful motor outputs from sensory inputs. Do we have evidence that consciousness exists anywhere in the universe without the informational complexity of a brain or brain-like computational device? If so, please let me know.
Of course, the monistic idealist can model the universe as consisting of microqualia. But then, he/she would still have to explain why the universe behaves physically as though quantum mechanics and particle physics were real. If one tried to find the microqualia at those lowest levels of complexity, while admitting the predictive power of these physical sciences, then the explanatory power of idealism is questionable. One may then identify the microqualia with the particles in the standard model, or with quantum phenomena, but I find it hard to see how that is an explanation of consciousness, given that my computer, my dog, and the air outside consists of the same basic stuff as my brain in either model. This would merely be a re-branding of a materialist view of the world. A phenomenon doesn’t change its nature because it is given a new name.
One possible approach to at least shed some light on the explanatory gap could be a meta-cognitive understanding of how humans think about consciousness itself. If we could understand how the brain creates and represents introspecive tokens such as “I feel pain right now” and how it represents explanatory models of causal functions such as the workings of a human brain, then we might understand why these representations seem so counterintuitively distinct and incompatible. The gap might be a gap in the modes in which our own minds model high-level introspective states vs. formal explanations of functional causalities. If we could analyze it this way, we could understand more clearly why it seems to us there’s a gap in the nature of consciousness (I don’t think there’s a gap in nature, just in the way we think about nature).
Regarding the zombie livestock, I think your argument for caution is reasonable in principle. If it was widely thought that suffering could be reduced this way, it might undermine the ethical drive to improve animal welfare, change people’s consumption styles or search for alternatives. On the other hand, despite certain improvements in some laws and public awareness during the last years and decades, it seems clear that this ethical drive has not succeeded in actually solving the problem of suffering even remotely. Is there any good reason to assume that this will change in the near future, if we let the status quo persist?
Hedonic Treader, I promise I’ve no hidden agenda. In fact I’m a boring scientific naturalist, indeed a physicalist. Just not a materialist :-)
A scientifically literate monistic idealism shouldn’t be seen as an alternative to quantum mechanics. Indeed a strong case can be made that quantum mechanics is complete – there are no hidden variables, nothing occurs in the world that isn’t captured by the formalism of the universal wave function. In that sense, as you say, all that monistic idealism offers is indeed a “rebranding” – it would be scientifically alarming were it anything else. Let’s ignore superstring / M-theory for now, though what I say about the silence of physics on the intrinsic nature of the stuff whose behaviour its formalism exhaustively describes holds true for such speculative extensions of the Standard Model too. Modern particle physics, most notably the Standard Model of elementary particles and their interactions, is formulated as a relativistic quantum field theory. A field in physics is defined purely mathematically. Yet what is the intrinsic nature of a field? The nature of what actually “breathes fire into the equations and makes there a world for us to describe” is implicitly assumed by materialists to be insentient – even though in the same breath such materialists would probably acknowledge, in John Wheeler’s phrase, that they have no idea “what makes the universe fly”. But this is an additional assumption – and the assumption gives rise to the Hard Problem of Consciousness and the mysterious Explanatory Gap.
Stimulating with a microelectrode a single neuron in, say, the striate cortex may generate a tiny, fleeting speckle of light. But a neuron is huge and complex compared to the hypothetical microqualia that the monistic idealist must posit for his ontology to make sense. As much as anything else, I think it’s the ridiculously small size of the ultimate (hypothetical) microqualia of the world that accounts for one’s incredulity. Indeed Galen Strawson (Consciousness and Its Place in Nature: Does physicalism entail panpsychism? (2006)) says he used to regard panpsychism as “crazy”. Well maybe it is – but when one has eliminated the inconsistent and the demonstrably impossible, one hasn’t many options left.
David, Giulio Tononi’s theory of consciousness as integrated information ( http://video.google.com/videoplay?docid=-7502852812875314243# ) gives some clues as to how the binding problem may be solved. It also seems to be the case that not only is necessary a degree of neuronal integration, but also particular, meaningful or coherent types of organization for the content of consciousness to arise, instead of just random noise or simple circuits of unconscious actions. Consciousness needs coherent sensory programming, otherwise it just won’t -feel- anything.
I believe that consciousness and qualia are mechanisms that have been composed by evolutionary selection (I find it extremely implausible that qualia already existed in a meta-level), and by understanding the roles it serves we may be able to infer the time of its appearance in the evolutionary tree. Also by analyzing brains through Tononi’s approach with some notice to coherent sensory organization in circuits, we may be able to eventually discover if an animal is conscious, and by proximity, if close animals are.
The argument against the so called “humane” production of meat (in animals altered to suffer less), would be comparable to arguing that we need to make animals suffer more than they already do, in order for people to care more about it…
Hedonic Treader, I think that the universe should be monist (coherent causation necessitates this), and, in a certain way, material or physical, but first person and not third person. Third person is the illusion that shows up in consciousness, first person is the whole universe. So, if you build a robot with all the circuits to be conscious, people may ask, “don’t we need to put a soul in it, or someone, that will experience this consciousness?”, to which I’d reply: “no, the first-person perspective pervades the whole material universe, which is all you, so the circuits for consciousness are all that is needed in addition.”
David, I never assumed you had a hidden agenda. You’ve stated your agenda quite eloquently and explicitly on hedweb.com. ;)
I found an interesting page on the hard problem of consciousness on utilitarian-essays.com, where a utiliarian answered his/her own questions about the nature of consciousness in a reductionist fashion: http://www.utilitarian-essays.com/consciousness.html
I would actually be fine with the concept of microqualia, just as long as we don’t imbue them with properties of a level of complexity they are very unlikely to have. If we assume that there are essential microqualia of “redness” or “a speck of light”, we’ve already crossed that line, or so it seems to me. After all, once we allow such informationally specific properties to be part of our array of microqualia, then we have to allow for an open-ended multitude of potential others. And that would make it doubtful that our microqualia are in any way “basic”, “intrinsic” or “essential”.
On the other hand, if we strip the microqualia concept of any and all properties other than “They exist within the laws of physics” and “If they are assembled in a complex way, specific sentient experiences are constituted”, then we haven’t gained any explanatory value at all. After all, this is exactly what a materialist would say about elementary particles. At this point, we have to ask ourselves on what grounds we allow our microqualia concept to survive Occam’s Razor.
By the way, one more thought on the zombie livestock: Maybe they don’t have to be zombies at all. All it would take is to alter their affective nature in such a way that they are adapted to their new environment, even if that environment is currently one of abuse and exploitation. Chicken feel bad about being forced to live in tiny cages because their ancestors lived in a different environment that would select out any sentients who were confortable without moving around, searching for food and mates, being fearful of predators etc.
If we could create beings that don’t suffer from pain, deprevation, fear or distress, but that have a high baseline of subjective pleasure almost independent of what physically happens to them, their lives could acually be a subjective net-gain, and for the first time since the creation of factory-farming, the argument that these animals are better off existing in human use than not existing at all could be coherently made. *If* this could be done, and *if* it could turn out to be ecologically sustainable, one might argue it would be preferable to other solutions such as vat animal products, since it entails the existence of happy sentients free from suffering.
The question is, how could one expertly assess the possible risks, benefits and consequences of such research? I wonder if there could be an acutal scientific and political discussion about this.
Jonatus, how much weight, if any, should we place on our intuitions when investigating consciousness?
Yes, like you, I find it “extremely implausible” that fields of microqualia – as distinct from fields of something-insentient – are what the formalism of physics describes. But have we any grounds for believing our naive intuitions are more trustworthy here than they were in other scientific disciplines?
Tononi says a lot of good stuff. But nothing Tononi says or writes even begins to close the Explanatory Gap between his materialist premises and his conclusions about consciousness. When we understand a phenomenon e.g, superconductivity, self-replication in DNA etc, then one understands how the microphysical facts necessitate the “emergent” phenomena at issue. By contrast, the alleged metamorphosis of insentience into sentience remains unexplained in materialist metaphysics.
Hedonic Treader, I’d agree that if microqualia are indeed the stuff of the world, then they must each be unimaginably simple and undifferentiated. Maybe the solutions to the fundamental equations of a TOE would allow a superintellect exhaustively to “read off” all their subjective values – though now we lack any kind of Rosetta stone by which such a read-off might be performed.
Zombie Livestock? Yes, they are only one option. Other options might be chronically opiated livestock; or, as you suggest, non-human animals bred with a genetically determined high baseline of subjective pleasure resistant to any possible physical insult. Actually, I can’t see any principled ground for denying a human cannibal similar options if we were prepared to license such interventions in members of other species.
But I wonder if there isn’t a case for moral clarity here? Killing other sentient beings because we’d like to eat their flesh is wrong, we may say, just as torture is wrong. We can argue this position as ethical utilitarians without adding a bunch of ceteris paribus clauses because an absolute prohibition leads to better overall consequences. Or maybe the parallel breaks down because torture is always intrinsically horrific, whereas the engineered livestock examples cited are intrinsically painless? Either way, my worry stands that by dreaming up such scenarios we’re implicitly giving a moral license to mass animal abuse and killing in the meantime. But perhaps you’re right; I don’t know.
I personally don’t share the view that killing sentient beings is paradigmatically wrong. All it does is limit the spaciotemporal “area” of their individual existence. It seems more important to me that the quality of that existence is taken care of. In complex self-aware beings such as ourselves, this usually implies personal freedoms that less self-aware beings might not need (to the same degree) for their well-being. Even if I grant that individuality is more illusionary than coherently real, I still have strong instincts related to social identifiers, individual personhood etc., constituting an array of needs which would reduce my subjective well-being if they were systematically neglected. Chicken, on the other hand, may have much less sophisticated needs.
Nevertheless, your argument stands. As for human cannibals, I think the demand is rather low. There certainly is no need for a system of happy humans ready to be eaten in order to replace systems of unhappy humans forced to be eaten. [In fact, there have been cases of voluntary cannibalism. I personally find it very hard to argue that people should have no right to be eaten if they really want to.] But a similar case may be made for happy slavery, and indeed, human trafficing and forced prostitution are problems that have to be solved, and one might think it abhorrent to solve it by creating artificially superhappy human slaves. Ideally of course, a better future has no need for anything remotely resembling exploitation of sentients.
In fact, one might argue that the only kinds of sentient creatures in a utopian future should be intelligent, self-aware sentients who can make clearly communicated decisions about their own well-being and who are endowed with self-ownership rights which are heavily enforced via a “sysop” scenario (or something similar).
But it’s safe to say that this leaves the realm of utilitarian ethics; any potential utilitronium shockwave scenario – or rescue missions to alleviate suffering elsewhere – would very probably entail the killing of sentients or the violation of self-ownership of complex self-aware sentients on a massive scale.
David, I accept that microqualia may be a possibility, provided that, as Hedonic Treader says, they be very simple and basic (it’s also quite an interesting idea). Alternatives such as qualia being just based on informational interpretation seem plausible to me; in any case even microqualia would need to be assembled in that way, and whether they could be differentiated into such diverse set of different feelings seems uncertain. They would also hint at some purpose or design in the universe, which is something that I find hard to accept (implying a careless or absent creator), or alternatively, many universes that sort of evolved into the current parameters that support consciousness, because otherwise we wouldn’t be here.
Hedonic Treader, I fully agree with your last comment.
I just remembered what may be one objection against microqualia, depending on how we take to their nature to be: they wouldn’t be able to provide the feedback that exists when we talk about the nature of our consciousness. Such feedback hints at consciousness fully being a, so to speak, “mechanical” or physical process. If microqualia were partly not of this kind of process, then that part would be fully unknown, and we’d not have any reason to say that it exists.
Jonatus, one advantage of positing fields of microqualia as the stuff of the world [whose behaviour the mathematical formalism of physics exhaustively describes] is precisely that this hypothesis allows a mechanistic explanation of subjective agency. The painfulness of pain really does help cause us to withdraw our hand from the fire. By contrast, epiphenomenalist and materialist theories of consciousness face seemingly insuperable problems. The epiphenomenalist can’t explain how supposedly causally inert qualia could even communicate their own existence – allowing us to discuss their existence and nature as we are doing now. The materialist has a different but analogous difficulty. Contemporary scientific materialists typically hold one or other functionalist version of the mind/brain identity theory whereby qualia are (in some unexplained way) identical with certain functionally individuated states of the brain. But if, as materialist natural science suggests, the purely microphysical story is necessary and sufficient to explain why, say, we withdraw our hand on encountering a noxious stimulus, then the subjective painfulness of the pain is irrelevant to its causal efficacy: physics is causally closed, and notional “zombies” would behave in exactly the same way if the laws and initial conditions were the same in their possible world. [Actually, I'm glossing over complications here. Rather than the subjective painfulness of nociception being incidental, might not this be a case of so-called "causal overdetermination" i.e. might not the microphysical interactions _and_ the phenomenal painfulness of pain (with which they are somehow identical) both determine the withdrawal of one's hand from the fire? But this possibility opens up a fresh can of worms.]
Like you, I take the information-theoretic paradigm seriously as a root-metaphor for life, mind and the universe – although the nature of “information” is far from clear. What is the ontological status of “information”? However, in this context perhaps it’s just worth noting that the information-theoretic paradigm is fruitful whether the fundamental “substrates” of the world turn out to be insentient or micro-sentient. Thus, for example, natural selection can act on information-bearing self-replicators irrespective of whether their substrate is microqualia or insentient matter as traditionally conceived. Likewise, both the materialist and the idealist can agree that the total information content of the observable universe in Planck units may be 10 to the 120 bits or whatever. And so forth.
Hedonic Treader, if one has a strict conception of personal identity, then on a utilitarian ethic I agree with you. It’s hard to envisage how killing other sentient beings could be inherently wrong in all circumstances. And my conception of identity is indeed strict. There are no enduring mechanical egos, simply here-and-nows that natural selection in life-supporting branches of the universal wave function has strung into particular casually-connected sequences. However, there may well be strong utilitarian grounds for feeling, thinking and acting as though “life is sacred”. Acting otherwise tends to diminish the well-being of the ontologically primitive here-and-nows. And in the case of nonhuman animals, “humane” slaughter [i.e. ten seconds or so of terror and agony] is a morally unacceptable price to pay for the trivial gustatory satisfaction we derive from eating dead flesh.
Utilitronium shockwave? Perhaps the fact we haven’t been engulfed by a utilitronium shockwave is presumptive evidence that suffering organic life is alone in our galactic neighbourhood. I’d guess later this century we will have arrived at a good understanding of the genesis of life on Earth, and whether [as I suspect] we are utterly typical of primordial life in being alone in our particular Hubble volume, or whether, on the contrary, “rescue missions” may be needed i.e. interstellar interventions to assist pain-racked Darwinian life elsewhere in the Galaxy that lacks the capacity to edit its own source code and redeem its own biosphere. The issue of consent would obviously need to be considered in depth before launching a utilitronium shockwave. But IMO benevolent gods have a duty to act when primitive sentience can’t look after itself.
You provided two arguments from utilitarianism against killing sentient beings, one that fears the indirect diminishing of the well-being even of ontologically primitive observer moments, the other one points directly to the suffering during the process of killing. Under HI’s premise that sentient beings can be altered in such a way that they cannot suffer, and that this can be done in a line of morally defensible research and with reliably quantifiable accountability, the second argument dissolves. Zombie livestock, or superhappy livestock, would not feel even 10 seconds of terror during slaughter. Of course, this point depends entirely on the feasability and reliability of the biotechnological interventions.
As for the first argument, in what ways would the well-being of anyone be negatively impacted by a hypothetical industry using sentients incapable from suffering? Yes, animal rights activists will be distressed by the thought of it, but relatively speaking, that is a very minor evil. A potential reply would be that people will realize speciesism is not well-justified, and then the same treatment of humans or other self-aware persons will be defensible (e.g. happy slavery). The same argument applies even stronger with regards to the status quo, of course.
I think this could be resolved by a framework that replaces speciesism with discrimination according to the degree of self-awareness (because it gives rise to a different class of needs). But one could imagine a new breed of artificial self-less altruists who no longer feel that they are intrinsically valuable individuals, or that their individual future matters in any special way. In such a world, I can no longer see any argument from utilitarianism why individual life should be treated as sacred. In fact, a utilitronium shockwave could be a very non-personal process, and intuitively, I don’t feel very comfortable about that – this is the “narrative depth” aspect I’ve mentioned before: Even if individuality is an illusion, the narration of individual biographies may be worth something in terms of experiential diversity.
The question then is, of course, how much narrative depth does a chicken’s life entail?
[...] discussions in the moral realism debate thread are still ongoing, after two [...]
Superb discussion(s)! I haven’t the time now to comment on all the comments I’d like to. Besides, not having visited the site in general in while (other irons in the fire, don’tcha know…), I’m a johnny-come-lately to this one anyway. One thing, though, that I must stress: Tim (Tyler) and Eli & Ben, and the rest of the half-dozen-to-a-dozen or so FAI researchers, need to be as familiar as possible with certain excellent works of philosophy that have parsed-out certain crucial problems/issues in axiology &/or moral theory. Now everyone in the field (I presume) is familiar with Derek Parfit’s (now, wow, quarter-century-old) classic, *Reasons & Persons*. But it’d also behoove them to familiarize themselves with 2 works that came-out long b4 even Parfit’s seminal work: David Gauthier’s *Practical Reasoning: The Structure and Foundations of Prudential and Moral Arguments and Their Exemplification in Discourse (Oxford U. Pr., 1963), and Tom Nagel’s *The Possibility of Altruism* (Princeton U. Pr., 1969, 1979). See also J. David Velleman’s recent *The Possibility of Practical Reason* (U. MI Lib., Schol. Pub. Off., 2009). To the extent that these works specify and discuss generic features of their subject(s), they presumably would and do apply to FAI considerations as well as strictly human ones. At the very least, they’re important for getting a handle on how *human* minds do their pratical axiological thing(s), as it were.