Comments on: More Subtle Objections to Jamais Cascio’s Views http://www.acceleratingfuture.com/michael/blog/2009/11/more-subtle-objections-to-jamais-cascios-views/ Transhumanism, AI, nanotechnology, the Singularity, and extinction risk. Mon, 13 Feb 2012 02:08:02 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Easy Web Site Builder http://www.acceleratingfuture.com/michael/blog/2009/11/more-subtle-objections-to-jamais-cascios-views/#comment-156285 Easy Web Site Builder Sat, 07 May 2011 07:46:32 +0000 http://www.acceleratingfuture.com/michael/blog/?p=1454#comment-156285 Top article Accelerating Future » More Subtle Objections to Jamais Cascio’s Views absolutely encouraged everyone In addition did you hear Egypt remarkable news flash. Thanks Easy Web Site Builder Top article Accelerating Future » More Subtle Objections to Jamais Cascio’s Views absolutely encouraged everyone In addition did you hear Egypt remarkable news flash. Thanks Easy Web Site Builder

]]> By: Steven Hales http://www.acceleratingfuture.com/michael/blog/2009/11/more-subtle-objections-to-jamais-cascios-views/#comment-128664 Steven Hales Wed, 18 Nov 2009 09:38:06 +0000 http://www.acceleratingfuture.com/michael/blog/?p=1454#comment-128664 Thanks Michael I appreciate the explanation it is a bit clearer now, I think. In his book, The Diffusion of Innovations, Everett Rogers details some examples of when diffusion fails. One story is about adoption of boiling water to prevent water borne illness. In a remote Peruvian village ingrained belief systems of when it was appropriate to consume hot or cold foods (not necessarily actually hot or cold in temperature) got in the way of adoption. The adoption rate was only 5% and this was only in the families who were "outsiders" in some sense. But even their adoption was based on their belief systems about lowland illness and how to fight this unfamiliar thing and they were happy that the health worker had guided them to a means to fight this. Adoption for them did not affect their social status or violate any internal rules of behavior. But the inconsistencies or misalignments of goals and behavior from non-adopters is only apparent to an objective observer. An AI must be able to handle this kind of situation and understand the dynamics of belief systems by being that objective observer. This is a very tough problem but one an AI must be able to do in some future scenario to save lives when belief systems cause misalignment between goals and behavior. Thanks Michael I appreciate the explanation it is a bit clearer now, I think. In his book, The Diffusion of Innovations, Everett Rogers details some examples of when diffusion fails. One story is about adoption of boiling water to prevent water borne illness. In a remote Peruvian village ingrained belief systems of when it was appropriate to consume hot or cold foods (not necessarily actually hot or cold in temperature) got in the way of adoption. The adoption rate was only 5% and this was only in the families who were “outsiders” in some sense. But even their adoption was based on their belief systems about lowland illness and how to fight this unfamiliar thing and they were happy that the health worker had guided them to a means to fight this. Adoption for them did not affect their social status or violate any internal rules of behavior. But the inconsistencies or misalignments of goals and behavior from non-adopters is only apparent to an objective observer. An AI must be able to handle this kind of situation and understand the dynamics of belief systems by being that objective observer. This is a very tough problem but one an AI must be able to do in some future scenario to save lives when belief systems cause misalignment between goals and behavior.

]]> By: Michael Anissimov http://www.acceleratingfuture.com/michael/blog/2009/11/more-subtle-objections-to-jamais-cascios-views/#comment-128657 Michael Anissimov Wed, 18 Nov 2009 01:44:14 +0000 http://www.acceleratingfuture.com/michael/blog/?p=1454#comment-128657 Steven, I anticipate a steeper transition curve, but if a slow takeoff occurs, it would make sense to implement what Jamais Cascio calls "just-in-time socialism" and basically start giving away free money to humans by "taxing" the machines. Even a miniscule tax could probably fulfill most human needs and desires. The first steps towards an AI are things like decision theory and algorithms that model complex causal networks. Human-friendly morality may actually represent 10% or less of the effort needed to get to Friendly AGI, but it is quite essential. You're right that building an AI that doesn't always defect in Prisoner's Dilemma-type situations is necessary. Decision theory formulations given by Eliezer Yudkowsky and Gary Drescher represent progress towards desirable decision theories, according to my current view. The idea of building an AI that begins by abstracting regularities in human behavior and using them as evidence for potentially useful strategies seems helpful, but it has yet to really be implemented in any substantive way, to my knowledge. In <em>Creating Friendly AI</em>, a similar notion is called "unity of will". Unity of will has its limitations because human motivations are inconsistent under reflection (we aren't always the people we wished we were, as Michelle points out), while we'd want a seed AI's motivations to be consistent under reflection. The challenge is immensely hard, but comments like yours and Michelle's are exactly the type of intelligent steps we need to make progress on it. Steven, I anticipate a steeper transition curve, but if a slow takeoff occurs, it would make sense to implement what Jamais Cascio calls “just-in-time socialism” and basically start giving away free money to humans by “taxing” the machines. Even a miniscule tax could probably fulfill most human needs and desires.

The first steps towards an AI are things like decision theory and algorithms that model complex causal networks. Human-friendly morality may actually represent 10% or less of the effort needed to get to Friendly AGI, but it is quite essential. You’re right that building an AI that doesn’t always defect in Prisoner’s Dilemma-type situations is necessary. Decision theory formulations given by Eliezer Yudkowsky and Gary Drescher represent progress towards desirable decision theories, according to my current view.

The idea of building an AI that begins by abstracting regularities in human behavior and using them as evidence for potentially useful strategies seems helpful, but it has yet to really be implemented in any substantive way, to my knowledge. In Creating Friendly AI, a similar notion is called “unity of will”. Unity of will has its limitations because human motivations are inconsistent under reflection (we aren’t always the people we wished we were, as Michelle points out), while we’d want a seed AI’s motivations to be consistent under reflection. The challenge is immensely hard, but comments like yours and Michelle’s are exactly the type of intelligent steps we need to make progress on it.

]]> By: Steven Hales http://www.acceleratingfuture.com/michael/blog/2009/11/more-subtle-objections-to-jamais-cascios-views/#comment-128656 Steven Hales Wed, 18 Nov 2009 01:13:32 +0000 http://www.acceleratingfuture.com/michael/blog/?p=1454#comment-128656 What if through bayesian analysis and bounded rationality the AI could follow complex rules to achieve the appearance of acting in line with your volition. It doesn't have to "know" or understand just that it appears to be acting in line with your volition. If the goal is to do things consistent with what you want and to detect deviations from some abstract moral code won't the AI's simulated subjective probabilities of acting correctly be more accurate since its rationality would be less bounded than a human's. Isn't heuristics the first step toward an AI? Also, won't teaching the AI about non-zero sum games and how to play them give it the tools to craft say trade treaties and reciprocal aggrements that maintain a positive non-zero sum game? Again bayesian subjective probabilities of success or probabilities that one or the other parties won't cheat or otherwise violate the agreement. What if through bayesian analysis and bounded rationality the AI could follow complex rules to achieve the appearance of acting in line with your volition. It doesn’t have to “know” or understand just that it appears to be acting in line with your volition. If the goal is to do things consistent with what you want and to detect deviations from some abstract moral code won’t the AI’s simulated subjective probabilities of acting correctly be more accurate since its rationality would be less bounded than a human’s. Isn’t heuristics the first step toward an AI? Also, won’t teaching the AI about non-zero sum games and how to play them give it the tools to craft say trade treaties and reciprocal aggrements that maintain a positive non-zero sum game? Again bayesian subjective probabilities of success or probabilities that one or the other parties won’t cheat or otherwise violate the agreement.

]]> By: Michael Anissimov http://www.acceleratingfuture.com/michael/blog/2009/11/more-subtle-objections-to-jamais-cascios-views/#comment-128655 Michael Anissimov Tue, 17 Nov 2009 22:14:33 +0000 http://www.acceleratingfuture.com/michael/blog/?p=1454#comment-128655 You're absolutely right, Michelle. Read <a href="http://singinst.org/upload/CEV.html" rel="nofollow">Coherent Extrapolated Volition</a> for some proposed mechanisms to minimize incoherence and increase consistency under reflection. You’re absolutely right, Michelle. Read Coherent Extrapolated Volition for some proposed mechanisms to minimize incoherence and increase consistency under reflection.

]]>