Michael Vassar on RPOP ‘Slaves’, AI vs. Human Uploads Friday, May 18 2007
AI and singularity 4:23 pm
While browsing the SL4 mailing list archives, as I am wont to do, I ran across this post by Michael Vassar that I thought made a lot of good points in a small space. It was in response to a couple people voicing ethical concerns that the AI boxing (sandboxing an AI from the outside world for testing purposes) is always unfair to the AI. Vassar, myself, and many others, believe that it should be entirely feasible to create an AI that is a self-improving optimization process in a general sense – something that manipulates matter into a target state – not requiring consciousness, the experience of pleasure or pain, or the like. In this same sense, evolution is one particular optimization process, without anthropomorphic qualities. In the future, it may be worthwhile to create AIs that are consciousness and human-like, but the point here is that they don’t need to be.
Onto the post:
~~~
Robin and Phil: I know it feels liberal, reasonable, fair, logical, unselfish, unbigoted, and in every way moral to extend ethical consideration to a GAI. I also know that as a species, our greatest ethical regrets are the countless times when we withheld ethical consideration from our fellow human beings, and that we have a long way to go before we overcome the tendencies which make us vulnerable to such regrettable actions. However, concerns about mistreating an AI, enslaving it or whatever, reflect deep anthropomorphic confusion.
We are not talking about containing an organism with an evolutionary past, selected from the search space by the removal of trillions of non-ancestors who failed to crave freedom. We are not even talking about an organism composed of countless agents, where belief is the interaction of excitatory “reward” and inhibitory “punishment” on many levels of organization. We are talking about an organism with no cognitive structures onto which to attach concepts of “reward”, “punishment”, “disappointment”, “pain”, “suffering”, “frustration”, “freedom”, “injustice”, or any of the other evolved salient patterns which we call values. These terms are no more properly attached to the sort of transparent AI SIAI favors than they are to “evolution”, “the economy”, or “the government”. We are talking about a Really Powerful Optimization Process, and it seems possible to me that this is a case where using that language, RPOP, rather than AI, will greatly improve thinking.
The universe is FULL of things which may merit ethical consideration and do not yet recieve it, from children to animals to lower level mind-like processes taking place in our own brains possibly including structures very loosely analogous to Freudian concepts, or to our models of other human beings and of ourselves. It is concievable that when we better understand ourselves we will identify other such things which I do not yet even suspect warrant such consideration, but to guess that a RPOP is one of those things makes no more sense than to guess this of existing software, and is in fact somewhat less justified than moral consideration given to the discarded programs produced by directed evolution, especially direct evolution of neural nets.
I am not at all suggesting that all AI development strategies can be pursued without the risk of causing harm to digital beings. The construction of an AI by reverse engineering of the human brain, as Kurzweil advocates, would be almost certain to be preceded by numerous aborted attempts at its goal prior to success. Partial minds would be built and studies, and their evolved structures would interact with their simulated environments in ways which corresponded to thousands of different exotic varieties of suffering. AIs of this sort would be, in many ways, far less dangerous than the transparent AIs recommended by SIAI. When thinking about them, anthropomorphic thinking would work. They would not suddenly display dazzling and unexpected new abilities which could be fully utilized with mere gigaflops of processing power. They would not be natives to the world of code, nor naturally enabled to modify their own workings. Unfortunately, they would not, ultimately, solve our problem. The fact that they can be built would not make normative reasoning systems impossible. The singularity would still beckon, and AIs modeled on our minds would be no more likely to make the ascent in a controlled and Friendly fashion than we would. Less actually, for many reasons including reasons analogous to those discussed here. There is also the substantial risk associated with any such AIs being terribly insane for biological and environmental reasons.
~~~
Many relevant concepts in this post.
I have no confidence that an un-self aware RPOP of sufficient capability to address the existenstial crises facing humanity can be designed. How precisely can an optimization process optimize well over criteria it does not understand and its creators do not fully understand and cannot fully specify? Without a deep understanding of human beings, deep enough a model to understand how we feel, how will such a process optimize human happiness and well-being? How will it optimize its acceptance by humans or its effects upon them without any concept of self? I believe the RPOP is a fruitless path. It is little more than another glorified software program modifying its own code with no deep model of what it is modifying or the consequences of its modifications beyond what its designers thought of.
I can’t disagree more. I even think, that this attitude of Samantha, which most people share with her, is a big obstacle for them to see, how things really are.
A self improving process _can_ be enough to conquer the Universe in a storm.
How to do it, I’ll not tell. Even if I knew.
Humans regularly optimize over criteria we don’t understand except in a very vague way. Our internal models are possibly 0.00000001% as complex as a high-resolution real world model, yet we manage to get things done. It is conceivable such a being can thoroughly understand “how we feel” simply by asking a few thousand people initially and then having the intelligence to make very educated guesses thereafter.
Say aliens and humans had different feelings of love and hate, or whatever. Would that mean that human and alien societies couldn’t work together, that we couldn’t come to an understanding and mutual assistance anyway? I think not.
Surely a RPOP can have a concept of self without itself engaging in egocentric reasoning. It is possible to understand anything by building a sophisticated enough model – you need not always be what you are observing. Newton did not need to become a celestial body to model gravity, Galileo did not need to become a cannonball to understand trajectories, and Hawking did not need to become a black hole to understand black hole physics. :)
There’s nothing so special and complex about aspects of our awareness and desires that they can’t be modeled by an outside agent, if that agent is sufficiently intelligent and has enough gigaflops of crunch power and a desire to understand us.
There’s some weird human instinct that causes us to put our brains and conscious experiences outside the realm of the natural, but this is just silly. We’re made of the same matter and energy as everything else. Understanding the details of human consciousness is no different mathematically than understanding the details of any other highly complex physical system.
Of course, if it really did turn out that a RPOP couldn’t model humans effectively, then it should be programmed to shut itself down.
“When thinking about them, anthropomorphic thinking would work.”
Not necessarily. Building up partial brain models inside a computer would probably entail modifying them significantly from their original form, which would be enough to wreak havoc with our ideas of what minds are like. Human beings are very, very similar to each other genetically, and they all have been raised and socially conditioned from birth in the company of other humans. They aren’t even as different from each other as other primates- our genetics are unusually close due to a bottlenecking event around 60,000 years ago. I agree, however, that these models will be more anthropomorphic than an AI created entirely from scratch.
“I have no confidence that an un-self aware RPOP of sufficient capability to address the existenstial crises facing humanity can be designed.”
An non-self-aware RPOP could not, by definition, modify its own source code; there is therefore no risk of a runaway recursive self-improvement. I am very skeptical as to whether humans could build an RPOP from the ground up, putting each component into place with only human intelligence. And at some point, wouldn’t a sufficiently smart RPOP become self-aware? Itself, after all, is the easiest thing for the RPOP to access.
“How precisely can an optimization process optimize well over criteria it does not understand and its creators do not fully understand and cannot fully specify?”
Through experiment. As long as you have a process which says “this fits criteria X” or “this does not fit criteria X”, you can reconstruct a definition without it existing beforehand by trying out rules for seeing whether something fits criteria X and deciding which to keep by Bayes’ Theorem. This is how we define anything; after all, when we were fetuses, we had no pre-existing definition of a computer in our heads. We had to build one up from what kind of object was referred to as a “computer”.
“I believe the RPOP is a fruitless path. It is little more than another glorified software program modifying its own code”
What?! I thought you just said the RPOP was non-self-aware! How exactly can you modify source code without being aware of what you’re modifying? Or is “self-aware” just codename for all the philosophical mumbo-jumbo that is associated with the idea of human consciousness?
“There’s some weird human instinct that causes us to put our brains and conscious experiences outside the realm of the natural, but this is just silly. We’re made of the same matter and energy as everything else. Understanding the details of human consciousness is no different mathematically than understanding the details of any other highly complex physical system.”
For a better idea of this, it would probably be helpful if you imagined “switching off” different parts of consciousness; sight, hearing, tactile sensation, the voice in your head, mental imagery, logical reasoning, emotions, and so forth. Once you get the idea that consciousness is an entity composed of parts, like a dashboard is composed of the odometer, speedometer, tachometer, and so forth, thinking about it analytically follows fairly easily. Also consider how animals behave- they still have emotions, visual, tactile, and auditory systems, but they lack higher reasoning functions.
I’m with samantha on this one. I think that there is no way to make a piece of software that is recursively self-improving and not self-aware. I think that the effort to create a digital slave of incredible power but no sense of self will lead to failure.
Of course this is all informal reasoning, and to settle the debate we need a mathematical theory of mind, which we don’t have. I’ve argued this on the transhumanist network a few weeks ago.
Samantha – what makes you think along these lines? You’re the first person who seems to agree with me!
Oh and, Thomas, you use, too many, commas, in the things, that, you, write,.
You may be right about commas, otherwise you are wrong.
Self awareness, yes, but subjective feelings of pleasure and pain, or a human-like “who am I?” philosophical tangent complex? No.
Michael Anissimov Says: “Self awareness, yes, but subjective feelings of pleasure and pain, or a human-like “who am I?” philosophical tangent complex? No.”
Ok, so you don’t think self awareness is what makes human being special. You say that what is really important is
(1) Pleasure and pain , and
(2) Our tendency to ask philosophical questions
As to (1), I’m not sure what you mean by pleasure and pain. Taken at face value, what you are saying sounds very odd. Animals have pleasure and pain, but we quite rightly regard ourselves as much more important than them. There are people whose nervous system is broken so that they don’t feel the obvious sorts of pain, like from touching hot things or cutting themselves. There must surely be people who cannot taste or smell, and there definitely are people who claim to be asexual. We don’t seem to regard them as worthy material for slavery. We also afford full human rights to autistics, who don’t derive pleasure from relating emotionally to people in any way.
So since you are clever guys here, I must presume that you are all talking about the “higher pleasures”. The pleasure of scientific discovery. The satisfaction of achieving a goal or helping people. It seems that a lot of these things would be prerequisites for an AI. It would be self-aware, so it would know that it liked these things. You could argue that an AI would be motivated to do all these things, but wouldn’t actually get any “pleasure” out of them; however this is dangerously hair-splitting. I might ask whether you actually get any “pleasure” out of your higher goals, or whether you are simply motivated to do them.
Now for (2). Presumably an awful lot of an AI’s work would be philosophical. He would have to understand human philosophy, correct and improve upon it, ask questions of his own initiative. He would then have to explain it all to us. He would be a better philosopher than any human.
All in all, it seems that you want grossly mistreat a person who, if instantiated on biological hardware, would be your best friend and role model. I smell rationalization.
Of course this is all provisional; I don’t have a mathematical theory of mind so I can’t prove what I’m saying.