Roko at Good.is: The Utopia Force
Roko has written an article on the human benefits of the Singularity, "The Utopia Force" at Good.is. Here it is.
How intelligent machines could make being human unimaginably better.
Part six in a GOOD miniseries on the singularity by Michael Anissimov and Roko Mijic. New posts every Monday from November 16 to January 23.
Think of this note as if it were an invitation to a ball—a ball that will take place only if people show up. We call the lives we lead here “Utopia.â€
– Nick Bostrom, Letter from Utopia.
Why should you care about the singularity when studies show that material possessions and technology beyond a certain point don’t actually make people any happier? Two weeks ago, I spoke about the possibility of giving a superintelligent AI the goal of doing whatever the human race would, after careful consideration, decide was best. This is known as the CEV algorithm. The outcome of this process would be very much unlike the technology, gadgets, and consumerism of today.
As Nick Bostrom has so eloquently reminded us, humanity’s biggest problems aren’t what we think they are: the most insidious and hard to notice, according to Bostrom, is that life is not nearly as good as it could be. This problem is really difficult for us to see; what could possibly be substantially better about our lives, even here in the developed world?
To start with, one has to realize that we’re not built for our own good. Evolution built us caring only about our ability to pass our genes on. We are easier to hurt than to pleasure, and we have been built with happiness set-points that are near impossible to significantly move away from without altering our biology. Studies have shown that giving a person $1,000,000 doesn’t actually make them happier in the long run, because of the hedonic treadmill effect: The human brain gets “used to†your circumstances, so that if your circumstances improve, your happiness goes up at first, but then returns to average. Our genes “calculate†that our bodies are worth keeping in good shape for 50 or so years, but after that, we are of little use, so our genes allow us to fall apart.
The society around us is also not built entirely for our benefit; it is a set of self-sustaining institutions that are, to a lesser or greater degree, influenced by the whims of a capricious electorate. Corporations can survive by hiring marketing departments to make us want things that we don‘t really need, and by hiring lobbying departments to make sure that the democratic process doesn‘t get in the way (see, for example, the tobacco industry). The challenges that work presents to us are often stifling and tedious; working in an office is not the natural human environment, which is why so many people ask for (and never get) a job that involves being outdoors.
Perhaps most importantly, the social dynamics that emerge from the interaction of many people who are each individually seeking status, power, and happiness often results in zero and negative-sum interactions. People are mean to each other, argue, fight, cheat, lie, and frequently make each others’ lives a misery—and this is ultimately a result of our evolved psychology, which was designed to deal with situations in which humans were forced by scarcity to kill each other in order to survive.
A world fashioned by the CEV algorithm would, at the very least, fix all of these fairly obvious flaws. Human psychology and biology could be altered to make us kinder, happier, healthier, and free from involuntary death or aging, and to remove the hedonic treadmill effect. New and better institutions could be developed from the ground up, and complex yet nourishing intellectual and physical challenges could be designed to replace what we today call “work.â€
Iain Banks has described such a world in his science-fiction books about a future society called “The Cultureâ€: enhanced humans live for thousands of years, and do exactly what they want with their time; they create art and science, they socialize, they enjoy a selection of customized virtual reality and real-world experiences and safe recreational drugs. They are all permanently young and attractive, with bodies and brains that have been altered in beneficial ways, they rarely argue with each other or have significant or prolonged negative interactions, and they have lots of sex.
If we consider all the possible ways that the universe could be arranged, and rank them in terms of how good they would be, Banks’ utopia certainly gets a very high rank. But it seems unlikely that it is the very best—or even close to the best. Banks’ utopia represents the limit of good experiences that we can currently think of and realize are good according to our complex values. Just how much better could it get?
In order to really make an accurate guess about the limits of goodness of the world, one must think about the problem indirectly or by analogy, because there are some states that are both so good and so complex that we cannot even imagine them yet. For example, what are the limits of goodness of subjective experience? What is the limit of the level and degree of mutual respect, friendship, passion or love that is possible?
At the risk of severely embarrassing myself forever over the internet, I’ll illustrate this with a personal example. Before I had ever kissed anyone, I didn’t actually know that the subjective experience of a passionate kiss was possible. I knew that kissing was possible, but not what it would feel like, or even that feelings that good were possible. Not only was I missing out, but I was unaware that I was missing out.
Humanity as a whole may have the same problem. We haven’t realized that a level of life that far surpasses what we currently experience might be possible. Imagine a human raised by animals who did not have the ability to speak, or the very real South American Indian tribe who are unable to learn, even when given food incentives, to count up to 10. Imagine a person who spent their entire life without experiencing romantic love, imagine a human who had neither hearing nor sight. These impoverished human beings are to us as we are to humans living in a post-positive singularity world: There are very probably intellectual, aesthetic, and emotional enhancements that would put their recipients as far beyond the citizens of The Culture as we are beyond a blind, solitary ignoramus.
A benevolent superintelligent AI would drastically and precisely alter the world, but do so in a direction that was dictated by your preferences. It would be like a new physical force that consistently pushed life towards our wisest utopian ideal. This ideal, or something very close to it, really is attainable. The laws of physics do not forbid it. It is attainable whether we feel that it is “unreasonable†that life could get that good, whether we shy away from it for fear of sounding religious, whether we want to close our eyes to the possibility because it scares us to believe that there is something greater out there, but we might let it slip through our fingers.
And indeed we might. As Carl Sagan puts it, “Our descendants, safely arrayed on many worlds throughout the solar system and beyond, will marvel at how vulnerable the repository of raw potential once was, how perilous our infancyâ€
.
Roko Mijic is a Cambridge University mathematics graduate, and has worked in ultra low-temperature engineering, pure mathematics, digital evolution and artificial intelligence. In his spare time he blogs about the future of the human race and the philosophical foundations of ethics and human values.
December 23rd, 2009 - 01:22
The Culture is too facile. Without the prospect of death, however unreal it seems to us, there would be no art. What meaningful beauty man creates is not despite our mortality, but because of our mortality. We tinker with the system at our own peril. The price of immortality will be our loss of humanity.
December 23rd, 2009 - 02:11
“Without the prospect of death, however unreal it seems to us, there would be no art”
No art at all? Surely you mean, “there would be art but it wouldn’t be as good?”. Obviously people would still create art.
December 23rd, 2009 - 08:21
Young children who stack legos and play “make-believe” are being creative because of death? How does that work, when they don’t understand the concept of death? Art is about learning, socializing, and communicating when words won’t do. We make art for many reasons, long before our short lifespan begins to really trouble us. In fact, a short lifespan often means we must focus on our job and our children during our most productive days of life instead of working on art.
Besides, no one mentioned immortality until you did, Jim. Wherever did you come up with that? It’s much easier to argue against immortality than against longer lifespans, so I see why you did it, but you sort of failed to discuss the actual article.
December 23rd, 2009 - 10:30
@Panda:
Yes, people really do like criticizing immortality. I wonder – perhaps Jim can tell us – what is his true rejection?
http://lesswrong.com/lw/wj/is_that_your_true_rejection/
He probably doesn’t actually know what his true rejection is – what it was about immortality that actually caused him to reject it. Of course, I will readily admit that I suffer from the same post-hoc justification problems.
December 23rd, 2009 - 16:05
Removing the hedonic treadmill effect isn’t necessarily a good idea. It works the other way as well, a person who becomes blind may not after a while be sadder than a sighted one. Unless you think an algorithm can be so godlike as to remove any need for human resilience not being miserable when you lose something important can be good.
I feel that the SAIAs assumption that an advanced AI could solve all problems is close to a religious belief. Shouldn’t one assume that it would have limitations and sometimes not know the right thing to do? As an example I would not know what to do if my dog had incurable cancer but her life could be extended by radiation and chemotherapy which would cause her possibly painful side effects. This is despite having cognitive powers she doesn’t have and couldn’t even understand.
December 23rd, 2009 - 18:00
“I feel that the SAIAs assumption that an advanced AI could solve all problems is close to a religious belief”
Well I know of no-one who has said that a superintelligent FAI could solve *all problems*, however I did say that it would probably make short work of the particular problems that we worry about, e.g. global warming or poverty.
Also, when you say “is close to a religious belief” do you mean “it sounds like religion, therefore, reasoning by analogy, it must be false”, or do you mean something else? How am I or Michael supposed to respond to criticism by analogy or association? It feels rather like being the victim of racist discrimination:
“you’re black, and the last 2 black people I met were ignorant, therefore you are also ignorant”
- in both cases this reasoning is useful only as a rough heuristic, and if you’re going to engage in a debate with someone, you may as well skip the stereotypes.
December 23rd, 2009 - 18:24
We can acknowledge that an advanced AI would have limitations and sometimes not know the right thing to do. Still, it would be better having one around rather than not, in the same way it would be nice to have cognitively enhanced humans around. The ideal Friendly AI would probably be functionally similar to a benevolent enhanced human without an ego.
December 23rd, 2009 - 18:30
I think it sounds like an article of faith that a CEV is possible and will produce an outcome that we here and now would think better than what’s happening now if we could see it.
Maybe, what’s really motivating my comments is the fear of something new and, by definition, without empirical testing behind it, dictating how I live. I’m not sure if this is a rational fear but it is one I have.
I have young children and I have to decide how they should live. For instance, I decided, since one is chubby, that they should take walks in the morning and be given toys to facilitate physical activity and enrolled in dance and sports classes. Will they thank me for this when they’re older? I’m guessing they will but I could be wrong.
What could a CEV know and what would it be forced to guess? Also what problems could a created intelligence not solved owing to the problem not really making sense, physical limitations that one might assume it would have, or limits because the problems are too complex to solve? I think that the characterizations I’ve read of CEV and CI make the assumptions that all our visible difficulties are due to misunderstanding or incomplete understanding. I can easily imagine that their could be problems that we would like solved, that given any extrapolation of our capacities that would make sense to us we’d still like solved but nevertheless cannot be solved.
December 23rd, 2009 - 21:14
This is a pretty interesting discussion I’ve never considered before.
“I can easily imagine that their could be problems that we would like solved, that given any extrapolation of our capacities that would make sense to us we’d still like solved but nevertheless cannot be solved.”
It seems to me that it’s less of a difficult equation/problem for AI to solve, and more of a neutral situation that is not what humans consider ideal – which I dare say, there is a solution to, even if it isn’t obvious. For example, if two humans both want to eat the same candy bar – well, no matter how smart AI is, it can’t enable both people to eat the candy bar. What it can do is take away the peoples’ desire to eat the candy bar, or change their desire to eat half a candy bar. It’s a roundabout, and instinctively unsatisfying and unexpected solution, but I really can’t think of a problem/situation that couldn’t be solved in this counter-intuitive way. Utopia can be achieved by meeting all reasonable needs/wants, or (somewhat frighteningly) by removing them from our biological software.
December 24th, 2009 - 03:59
“I can easily imagine that their could be problems that we would like solved, that given any extrapolation of our capacities that would make sense to us we’d still like solved but nevertheless cannot be solved.”
– sure. I can even give you an example: bringing dead people back once they have been buried for a long time or cremated may be physically impossible, and therefore beyond the capability of a super-smart AI, even though we would dearly like it.
But … this is not exactly an argument against building a CEV superintelligence. Just because it isn’t perfect doesn’t mean it isn’t GOOD.
July 22nd, 2010 - 18:52
The trouble is that there is no way to get to CEV. It is effectively what “we” would decide if we were smarter and more rational than we actually are. So how do you get that and vet that it is correct if some theoretical FAI attempts to compute it?
What if you vehemently disagree with the conclusions of the CEV? What if the process concludes that what is best for us are things that the majority of actual humans very much disagree with? Why is having an AI dictatorship effectively any better for humanity?
At what point is a FAI rewriting humans to be better effectively destroying real humans and replacing them with something it reasons would be better and happier beings? How do we ensure that it “grows” us from where we are toward better at the proper pace for each of us rather than simply replacing us?