Future Current

The study of reproducible errors of human reasoning, and what these errors reveal about underlying mental processes, is known as the heuristics and biases program in cognitive psychology. This program has made discoveries highly relevant to assessors of global catastrophic risks.  Eliezer Yudkowsky, who writes on the subject of cognitive biases at Overcoming Bias, presented at the  Global Catastrophic Risks conference in Mountain View on the subject of cognitive biases in the assessment of risk.

Read the rest of this entry »

Ben Goertzel, Eliezer Yudkowsky, and Melanie Swan at the Artificial Intelligence and Society event

In everyday life, we underrate the importance of intelligence because our social environment consists only of other humans, yet it is our real trump card as a species and the foundation for everything else we do. The Bayesian statistician I. J. Good proposed that an “intelligence explosion” brought about by an artificial intelligence improving the design of its own intelligence could be expected to reshape the universe more than all human actions up to this point. In January 2008, Eliezer Yudkowsky examined this arguments in an informal presentation of his talk “The Human Importance of the Intelligence Explosion” at the Artificial Intelligence and Society event hosted by the University of Santa Clara and Singularity Institute for Artificial Intelligence.

Read the rest of this entry »

Eliezer Yudkowsky has two papers forthcoming in the edited volume Global Catastrophic Risks (Oxford, 2007), “Cognitive Biases Potentially Affecting Judgment of Global Risks” and “Artificial Intelligence as a Positive and Negative Factor in Global Risk.” At the 2007 Singularity Summit, he described how shaping a very powerful and general AI implies a different challenge, of greater moral and ethical depth, than programming a special-purpose domain-specific AI. The danger of trying to impose our own values, eternally unchanged, upon the future, can be seen through the thought experiment of imagining the ancient Greeks trying to do the same. Human civilizations over centuries, and individual human beings over their own lifespans, directionally change their moral values.

Read the rest of this entry »

Eliezer Yudkowsky is one of the world’s foremost researchers on Friendly AI and recursive self-improvement. He created the Friendly AI approach to AGI, which emphasizes the importance of the structure of an ethical optimization process and its supergoal, in contrast to the common trend of seeking the right fixed enumeration of ethical rules a moral agent should follow. At the 2007 Singularity Summit, he introduced three schools of thought currently associated with the word “Singularity,” their core arguments and bolder conjectures, while noting where they support or contradict each other.

Read the rest of this entry »

On October 10, a Facebook group composed of Silicon Valley residents met for an informal dinner conversation with Singularity Institute Co-Founder and Research Fellow Eliezer Yudkowsky. The topic was the future of intelligence: Before we can have any understanding of what to expect if the future contains minds far more intelligent than we are, we might consider what it is that makes a mind intelligent in the first place.

Read the rest of this entry »

SIAI Interview Series - Eliezer Yudkowsky

Read the rest of this entry »

Eliezer Yudkowsky and Peter Thiel at the 2006 Singularity Summit at Stanford

Eliezer Yudkowsky is Research Fellow and Director of the Singularity Institute for Artificial Intelligence, a non–profit research institute dedicated to increasing the likelihood of, and decreasing the time to, a maximally beneficial singularity. He is one of the world’s foremost experts on the subject, and frequently speaks on artificial general intelligence (AGI), rationality, and the future. He created the “Friendly AI” approach to AGI, which emphasizes the importance of the structure of an ethical optimization process and its supergoal, in contrast to the more common trend of seeking the ‘right’ fixed enumeration of ethical rules a moral agent should follow. In 1996, he wrote “Staring into the Singularity,” one of the first singularity analyses. In 2001, he presented the first technical design for a self–improving AI that takes safety into consideration, with his book–length Creating Friendly AI: The Analysis and Design of Benevolent Goal Architectures. “Coherent Extrapolated Volition” (2004) is a recent written update to his Friendly AI theory.

Yudkowsky’s research into the workings of the human mind and minds–in–general led to a paper on the evolutionary psychology of human general intelligence, “Levels of Organization in General Intelligence” (2002), forthcoming in a volume on Artificial General Intelligence, as well as papers on human rationality and philosophy of science. He also has two papers forthcoming in an edited volume on Global Catastrophic Risks from Oxford University Press, entitled “Cognitive Biases Potentially Affecting Judgment of Global Risks” and “Artificial Intelligence and Global Risk.” His presentation at the Singularity Summit in 2006 was entitled “The Human Importance of the Intelligence Explosion.”

Read the rest of this entry »

Eliezer Yudkowsky is one of the world’s foremost researchers on Friendly AI and recursive self-improvement. He created the Friendly AI approach to AGI, which emphasizes the importance of the structure of an ethical optimization process and its supergoal, in contrast to the common trend of seeking the right fixed enumeration of ethical rules a moral agent should follow.

In 2001, he published the first technical analysis of motivationally stable goal systems, with his book-length Creating Friendly AI: The Analysis and Design of Benevolent Goal Architectures. In 2002, he wrote “Levels of Organization in General Intelligence,” a paper on the evolutionary psychology of human general intelligence, published in the edited volume Artificial General Intelligence (Springer, 2006). He has two papers forthcoming in the edited volume Global Catastrophic Risks (Oxford, 2007), “Cognitive Biases Potentially Affecting Judgment of Global Risks” and “Artificial Intelligence as a Positive and Negative Factor in Global Risk.”

In his Transvision 2007 talk on the power of intelligence, called “Mind Is All That Matters: Reasons to Focus on Cognitive Technologies,” he reminded us that the human brain more than any other development in history has changed the face of the earth, and logically one can expect that improved cognitive technologies will likely have the same dramatic impact on human conception and control over the physical universe.

Read the rest of this entry »