Great article by Patri Friedman at Less Wrong. He turned my thinking around on the issue, in less than 15 seconds of reading.

Other interesting posts include “Can we create a function that provably predicts the optimization power of intelligences?” by whpearson and Do Fandoms Need Awfulness? by Eliezer Yudkowsky.