Somehow I missed this back in February: The Artificial Morality of the Robot Warrior. (Blog post by Nicholas Carr, who in case you’re not familiar, is somewhat of a big deal.) The Ethics & Emerging Technologies Group at Cal Poly wrote up a report on roboethics for the US Navy. This was covered by the Times of London.

Why is it necessary we explore the possibility of engineering robot morality?

Perhaps robot ethics has not received the attention it needs, at least in the US, given a common misconception that robots will do only what we have programmed them to do. Unfortunately, such a belief is sorely outdated, harking back to a time when computers were simpler and their programs could be written and understood by a single person. Now, programs with millions of lines of code are written by teams of programmers, none of whom knows the entire program; hence, no individual can predict the effect of a given command with absolute certainty, since portions of large programs may interact in unexpected, untested ways … Furthermore, increasing complexity may lead to emergent behaviors, i.e., behaviors not programmed but arising out of sheer complexity.

Related major research efforts also are being devoted to enabling robots to learn from experience, raising the question of whether we can predict with reasonable certainty what the robot will learn. The answer seems to be negative, since if we could predict that, we would simply program the robot in the first place, instead of requiring learning. Learning may enable the robot to respond to novel situations, given the impracticality and impossibility of predicting all eventualities on the designer’s part. Thus, unpredictability in the behavior of complex robots is a major source of worry, especially if robots are to operate in unstructured environments, rather than the carefully‐structured domain of a factory.

In a section of the report titled “Programming Morality”, they say:

Engineers are very good at building systems to satisfy clear task specifications, but there is no clear task specification for general moral behavior, nor is there a single answer to the question of whose morality or what morality should be implemented in AI …

The choices available to systems that possess a degree of autonomy in their activity and in the contexts within which they operate, and greater sensitivity to the moral factors impinging upon the course of actions available to them, will eventually outstrip the capacities of any simple control architecture. Sophisticated robots will require a kind of functional morality, such that the machines themselves have the capacity for assessing and responding to moral considerations. However, the engineers that design functionally moral robots confront many constraints due to the limits of present‐day technology. Furthermore, any approach to building machines capable of making moral decisions will have to be assessed in light of the feasibility of implementing the theory as a computer program.

More research is needed.

I’m continuing to enjoy the term “artificial morality” for promoting the concept. It features prominently in Moral Machines by Wendel Wallach and Colin Allen, which I call “the only published book on Friendly AI”.

I’m in somewhat of a confusing situation, because I find myself automatically drawn to using Eliezer Yudkowsky’s terminology when discussing the subject, because he has written the most of anyone on the topic, but his work is not considered to be part of the academic mainstream, which I want to speak in the language of. As pointed out by Peter McCluskey in his Amazon review of The Singularity is Near (rated #1 out of 136 reviews):

I’m bothered by his complacent attitude toward the risks of AI. He sometimes hints that he is concerned, but his suggestions for dealing with the risks don’t indicate that he has given much thought to the subject. He has a footnote that mentions Yudkowsky’s Guidelines on Friendly AI. The context could lead readers to think they are comparable to the Foresight Guidelines on Molecular Nanotechnology. Alas, Yudkowsky’s guidelines depend on concepts which are hard enough to understand that few researchers are likely to comprehend them, and the few who have tried disagree about their importance.

I blame two memes: the Blank Slate and Moral Realism. The blank slate fallacy can be dispelled by reading the book of the same name, and moral realism by reading Joshua Greene’s Ph.D thesis. When these two ideas are removed, we are left with the hard task of building a morality, which promises to be mighty difficult, possibly as difficult as building intelligence itself.

Moral realism seems particularly hard to get rid of, though. Perhaps Hume can help, who, like a diabolic unfriendly AI, said, “Tis not contrary to reason to prefer the destruction of the whole world to the scratching of my finger.”