At the SIAI blog, Joshua Fox has provided a list of writings about risks and moral issues associated with recursively self-improving intelligence. Here is the list:
Stuart Armstrong, “Chaining God: A qualitative approach to AI, trust and moral systems,“ 2007.
Nick Bostrom, “Ethical Issues in Advanced Artificial Intelligence,” Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003.
Tim Freeman, “Using Compassion and Respect to Motivate an Artificial Intelligence,” 2007-08.
Ben Goertzel, “Thoughts on AI Morality,” Dynamical Psychology, 2002.
Ben Goertzel, “The All-Seeing (A)I,” Dynamical Psychology, 2004.
Ben Goertzel, “Encouraging a Positive Transcension“ Dynamical Psychology, 2004.
Stephan Vladimir Bugaj and Ben Goertzel, Five Ethical Imperatives and their Implications for Human-AGI Interaction.
J. Storrs Hall, “Engineering Utopia”, Artificial General Intelligence 2008: Proceedings of the First AGI Conference, Volume 171, Frontiers in Artificial Intelligence and Applications, ed. P. Wang, B. Goertzel and S. Franklin, 2008.
Bill Hibbard, “Critique of the SIAI Guidelines on Friendly AI,” 2003.
Bill Hibbard, “Critique of the SIAI Collective Volition Theory,” 2005.
Steve Omohundro, “The Nature of Self-Improving Artificial Intelligence,” Singularity Summit 2007.
Steve Omohundro, “The Basic AI Drives”, Proceedings of the First AGI Conference, Volume 171, Frontiers in Artificial Intelligence and Applications, ed. P. Wang, B. Goertzel and S. Franklin, 2008.
Feel free to add to this list in the comments if anything was missed.