I just finished Joshua Greene’s paper, “The Terrible, Horrible, No Good, Very Bad Truth About Morality and What To Do About It” and loved it. Best paper I’ve read since Omohondro’s “Basic AI Drives”. If people just read and understood both those papers, then far more would probably drop everything they’re doing and do something about unfriendly AI. I hope that SIAI can build some sort of relationship with Greene, or get him to say a few words on how moral revisionism (in the sense of moving past moral realism) applies to AI morality.

Though the first half of Greene’s essay was the most intellectually serious and useful, the latter half was more amusing and interesting. I want to comment on it, but I don’t want to ruin it by having you reminded by the part I mention for the whole essay, so I’m just going to post the quote he concludes with:

“If only it were so simple! If only there were evil people somewhere
insidiously committing evil deeds, and it were necessary only to separate
them from the rest of us and destroy them. But the line dividing good and
evil cuts through the heart of every human being. And who is willing to
destroy a piece of his own heart?”
—Aleksandr Solzhenitsyn