Accelerating Future Lexicon:

GAI:
See Artificial General Intelligence.

Gaussian distribution:
The characteristic distribution of intelligence within our species. Think "Bell Curve". All neurologically normal human beings share the same universal set of complex functional adaptations underlying our ability to think, but their capacity is based on our heredity and random genetic variation. "Non-Gaussian", "super-Gaussian", or "extra-Gaussian" minds would be outside the characteristic human distribution. Such minds could be smarter than the smartest human that has ever lived, or dumber than the dumbest human, or roughly human-equivalent, but possessing a different balance of domain competencies; a variety of exotic arrangements could be possible. See also Singularity, transhuman intelligence.

gender-neutral pronouns:
Science fiction writer Greg Egan's terms for referring to gender-neutral entities; ve, vis, ver, verself. These words are sometimes used within Singularity communities to refer to AIs or superintelligences, although they seem to be passing out of favor.

general AI:
An Artificial Intelligence with general intelligence. See also Artificial General Intelligence.

general intelligence:
The finite quantity of complexity required to solve a reasonably open-ended range of problems. The ability that allowed humans to build cities, invent writing, and launch rockets, bearing in mind that we adapted to environments where none of these abilities were specifically relevant, and that we share 98% of our DNA with chimpanzees. The right mix of memory, processing algorithms, and reflective capabilities that allow an entity to "accomplish complex goals in complex environments" [Goertzel01]. After achieving general intelligence, it has been postulated that an Artificial Intelligence could take over the job of its own design and improvement from the programmers, most likely resulting in recursive self-improvement [Yudkowsky02]. See also Artificial Intelligence, human-equivalence.

General Intelligence and Seed AI (GISAI):
A publication of the Singularity Institute for Artificial Intelligence outlining an early theory for a complete self-improving mind. This document has been mostly obsoleted by Levels of Organization in General Intelligence. Located at http://www.singinst.org/GISAI.

GISAI:
See General Intelligence and Seed AI.

Globalism:
One of the "Singularitarian Principles", or guidelines that those pursuing the Singularity tend to follow. Globalism is the idea that the benefits of the Singularity should be distributed equally to everyone on Earth; any other outcome would be suboptimal. Part of the point to the Singularity would be transcension of the resource- and power-grabbing that has characterized human society for hundreds of thousands of years. See Singularitarianism, Singularitarian Principles.

goal system:
A system making choices projected to lead towards certain goal states, either in fictitious scenarios abstracted from spoon-fed data, (as in The Matrix, or young AI training simulations) internal mindspace, (if the goal system's only feedback is information from its own brain, as in an algorithm that does nothing but process its past output) or externally observable reality. At the head of a goal system sits a supergoal or supergoals. If no explicit supergoals exist, then the aggregate of low-level goals might begin serving as de facto supergoals. Humans are a goal system, with goals designed by biological evolution. Our evolution-given supergoal is to reproduce as effectively as possible. Human culture and pursuit of pleasure are examples of human-unique subgoals "stomping" evolution's supergoal. For information on the relationship of Friendly AI and AIs in general to goal systems, see http://www.singinst.org/CFAI/design/generic.html. See also subgoal, supergoal.

Goertzel, Dr. Ben:
Research professor who has taught at universities in Australia, New Zealand, and the United States. Holds a Ph.D in mathematics. Leader of the Novamente AI project. Founded the Artificial General Intelligence Research Institute. Website: http://www.goertzel.org.

Grey Goo:
Runaway self-replicating nanotechnology, often feared as potentially world-swallowing. Grey Goo is likely to come in different flavors, and some types may possess higher degrees of intelligence or adaptability than others (whether internally or through external control). Grey Goo is less likely to be a threat than a runaway nanotechnological arms race, purposefully created nanoweapons, or the use of nanocomputers to brute force a recursively self-improving, human-indifferent seed AI. The threat of Grey Goo has been exaggerated for several reasons; 1) it's easy to imagine, 2) the analogy between biology and molecular nanotechnology predisposes us to view self-replicating nanotechnology as germs, and 3) forecasters underestimate the high level of complexity that would be required for self-replicating nanotemachines to survive in the Earth's open environment. The real danger of nanotechnology is through deliberate misuse. See also Center for Responsible Nanotechnology, Foresight Institute, nanotechnology, nanocomputing, nanotechnological arms race.

Hard Problem, the:
Articulated by philosopher and cognitive theorist David Chalmers in the landmark paper "Facing Up to the Problem of Consciousness", the Hard Problem of consciousness refers to why we have subjective experience at all, why we don't just spend our lives talking and moving and learning without a complex subjective world. There still would be subjective viewpoints if consciousness didn't exist, but subjective experiences , no. Very tough problem; hard to put into words as well as analyze. I would definitely suggest reading the above paper if you're into this one. See also qualia.

hard takeoff:
The Singularity scenario in which a mind makes the transition from prehuman or human-equivalent intelligence to strong transhumanity or superintelligence over the course of days or hours [Yudkowsky01]. Arguments for the probability of hard (as opposed to "soft") takeoffs usually cite the vast speed differences between human neurons and silicon chips, in addition to the qualitative differences between humans and minds in general. The high likelihood of a hard takeoff once a roughly human-equivalent AI is created has been argued by the Singularity Institute here: http://www.singinst.org/LOGI/seedAI.html.

hardware-extensible intelligence:
An intelligence with the ability to integrate new hardware into its brain. No such intelligences currently exist (evolution has never designed them), but they may be created eventually, probably by people trying for the Singularity . AIs, by virtue of their substrate, would naturally exist as hardware-extensible intelligences. If an experienced neuroscientist had the ability to make arbitrary additions to his/her own brain structure, it would only be a matter of time (probably no more than a year, unless the neuroscientist were working very slowly) before he/she jumped out of the human Gaussian distribution of intelligence, became smarter than Einstein, passed on into the transhuman realm, then on to the realm of recursive self-improvement and superintelligence. When we fuse Friendliness, smarter intelligence, faster intelligence, hardware-extensible intelligence, and self-modifying intelligence into one entity, what we get is a Friendly seed AI, which, if programmed correctly, would be able to enhance itself into a Friendly superintelligence safely (or so the argument goes). See also self-modifying intelligence, self-translucent intelligence, recursive self-improvement.

Hofstadter, Dr. Douglas:
Professor of Computer Science at Indiana University, dabbler in philosophy of mind, linguistics, music, systems-theoretic stuff, Artificial Intelligence, analogies, and much more. Author of "Godel, Escher, Bach" and "Metamagical Themas". Homepage: http://www.psych.indiana.edu/people/homepages/hofstadter.html.

Horizon:
The period of time on Earth just before the Singularity. Many Singularity analysts would argue that we are already in the Horizon times right now (greater-than-human intelligence is likely to arrive shortly, relative to historical timescales. This would hold whether it is created within 10 years or 100.) Singularitarians would argue that the Horizon period is when it is most important to be deliberately accelerate the Singularity and devise strategies to protect its integrity (fairness and safety).

humane:
The general set of values most humans might acquire if we had the ability to thoroughly understand and modify our beliefs and emotions, and necessary resources were abundant. We might become kind, effective, pleasant, repairing and creating rather than jealous, defensive, anxious, and destructive. Human is what we are, humane is what we wish we were. Since our innate neurologies prevent deep levels of self-modification, no truly "humane" humans have ever existed, therefore making it difficult for us to guess what they would be like. Broadly, we might say that they generally favor diplomacy over force, life over death, happiness over sadness, and so on. The specifics of how these values would be implemented is genuinely beyond us. The term "humane" is often used to indicate that one accepts the possible existence of a qualitatively better-than-human morality; the alternatives are asserting that 1) humans have the best possible morality, or 2) all morality is relative. Even if 2 is true and morality is relative, this still doesn't prevent the construction of moralities that are genuinely considered more satisfying and complete among an arbitrarily intelligent subset of the poplace. Morality being relative still allows the creation of moralities that are better relative to us; meaning "us" the human species, or "us" as in "human species plus any sentient minds created after or before the Singularity". Sometimes called "renormalized human morality".

human-equivalence:
A threshold level of ability in some domain which, if achieved, allows the AI to understand human concepts and do work as good as a human in that domain. A human-equivalent AI is one that has passed the threshold level of general competence in some sense. An ambiguous and arguable definition; "transhumanity", especially strong transhumanity, is more blatant [Yudkowsky01]. See also human-similar, prehuman AI, tool-level AI.

human-similar:
Yet another term to denote rough human-equivalence in AI, which partially escapes the anthropocentric connotations of the phrase "human-equivalent".

ICM:
See Integrated Causal Model.

immortalism:
The philosophy held by those who want to live forever, if possible. Immortalism argues that life is inherently positive, long lives are worth living, and the opportunity for immortality should be available, but not forced upon anyone. Immortalists acknowledge that the laws of physics may not permit infinite lifespans, in which case they would settle for living for billions or trillions of subjective years (as would seem to be possible through uploading or nanomedicine.) See also Anissimov, Michael, Immortality Institute, Klein, Bruce.

Immortality Institute:
A grassroots life extensionist organization devoted to spreading information regarding the possibility and desirability of extremely long (or infinite) lifespans. The Immortality Institute seeks to make death voluntary rather than mandatory. Website: http://www.imminst.org. See also Anissimov, Michael, immortalism, Klein, Bruce.

inclusive fitness:
A term used in evolutionary psychology to explain kin selection, or the tendency of organisms to behave altruistically towards those they share genetic material with. Used to refer to selection models inclusive of a given organisms' kin (genetic interests). Siblings share 50% of new genetic material with one another. First cousins share 12.5% of new genetic material with one another. Parents share 50% of new genetic material with their children. Organisms may sacrifice themselves for kin if enough is at stake. Altruism beyond the family probably evolved by bootstrapping itself from kin altruism. Instead of simply saying "an organism is behaving in the interests of its own fitness", modern evolutionary psychologists say "an organism is behaving in the interests of inclusive fitness", which carries the connotation of kin altruism. In ancestral tribes, composed of only around 200 people, the average kin relationship we would have with any given tribe member would be around the level of 3rd cousin, creating a small, ambient altruistic hum grounded in the genetic logic of inclusive fitness.

inductive inference:
From University of Alberta Cognitive Science Dictionary: "Inferences are made when a person (or machine) goes beyond available evidence to form a conclusion. An inductive inference is one that is likely to be true because of the state of the world. Unlike deductive inferences, inductive inferences do yield conclusions that increase the semantic information over and above that found in the initial premises". The University of Alberta's definition is not precisely correct - it is impossible to spontaneously generate information without premises. However, the premises underlying inductive inferences are supposed to be more general and vague than those underlying deduction, making induction more mysterious and the source of much controversy. The algorithmic complexity theorist Ray Solomonoff has formulated a more formal and mathematical definition of induction based on complexity theory, known as "Solomonoff induction".

infrahuman AI:
An AI of below-human ability and intelligence. "Prehuman" is usually referred to use to an infantlike or tool-level AI, so that "infrahuman" usually refers to a fairly mature AI, capable of general cognition, which is still not in the vicinity of human intelligence [Yudkowsky01]. See also prehuman AI, near-human, human-equivalent, transhuman, superintelligence.

Institute for Accelerating Change (IAC):
Los Angeles-based organization and community that studies the phenomenon of accelerating change. Holds a yearly conference, "Accelerating Change Conference". Founded in 2002 by John Smart. See also Accelerating Change Conference.

Integrated Causal Model (ICM):
Evolutionary psychologists John Tooby and Leda Cosmides' model of human behavior, created to displace the scientifically inaccurate Standard Social Science Model (SSSM). The Integrated Causal Model was first formally introduced in the article "The Psychological Foundations of Culture" in the venerable collection "The Adapted Mind". The Integrated Causal Model holds that the human mind is composed of a set of psychological adaptations to recurring evolutionary challenges in our ancestral environment. For more on evolutionary psychology and the Integrated Causal Model, see http://www.psych.ucsb.edu/research/cep/primer.html. Steven Pinker, author of "How the Mind Works" and "The Blank Slate", is a prominent supporter of the Integrated Causal Model. See also complex functional adaptation, Cosmides & Tooby, evolutionary psychology.

Jupiter Brain:
Brain approximately the size of Jupiter. The Transhumanist Terminology glossary defines a Jupiter brain as "A posthuman being of extremely high computational power and size." The term supposedly originated due to an idea by Keith Henson that nanotechnology might eventually be used to turn the mass of Jupiter into computers running an upgraded version of himself. Transhumanist Robert Bradbury (and others) have actually ran the numbers: http://www.aeiveos.com/~bradbury/JupiterBrains. See also ceiling being.

kinder-than-human:
Term to describe minds that exceed the kindness displayed by any human throughout history, and possess the rationality and wisdom to apply their kindness appropriately. Since natural selection and evolution on Earth automatically tends to create organisms that possess selfish goal systems, humans have certain bounds as to the amount and quality of kindness they are capable of displaying (this is supported by evolutionary psychology). With the de novo creation of synthetic entities, and neurologically modified human beings, these bounds need constrain us no longer. Kinder-than-human intelligence would be most effective if coupled with smarter-than-human intelligence and recursive self-improvement, as would be the case with successful Friendly AI. See also altruism, altruist.

Klein, Bruce J:
Director of the Immortality Institute and one of the main figures in the immortalist movement. Homepage: bjklein.com. See also Anissimov, Michael.

Kolmogorov complexity:
A term from information theory. The Kolmogorov complexity of a string of bits is the length of the smallest Turing machine program which produces the bit string as output. (It is therefore somewhat dependent on one's choice of Turing machine, but since every Turing machine can be emulated by an universal Turing machine with a constant increase in program length this doesn't matter much.) See http://www.idsia.ch/~marcus/kolmo.htm. See also AIXI, algorithmic complexity, Solomonoff Induction.

Kurzweil, Dr. Ray:
Well-known futurist who speculates on a merger between humans and "machines" (AIs) in the near future. Dr. Kurzweil analyzes various accelerating trends and extrapolates them to come to the conclusion that there will be a Singularity sometime around 2029. (It is projected that around that time, extremely high-resolution brain scans will be possible, allowing us to run human brains on computers. However, this projection neglects the possibility of a sooner Singularity through understanding of the basic functions of general intelligence before extremely high detail brain scans arrive.) Kurzweil is an author, inventor, and entrepreneur, and has testified before Congress on technology policy issues. Past works include "The Age of Intelligent Machines" and "The Age of Spiritual Machines" (which has ranked #1 on Amazon.com for the category of "technology"). Kurzweil will be publishing his next book, "The Singularity is Near", sometime in 2004. Personal website: http://www.kurzweiltech.com/aboutray.html. Popular website: http://www.kurzweilai.net.

Levels of Organization in General Intelligence (LOGI):
A publication of the Singularity Institute for Artificial Intelligence which overviews the Institute's General AI theory. Integrates a variety of strong theories from cognitive science to create a very vigorous outline of a theory of intelligence in general. Available at http://www.singinst.org/LOGI.

Luddism:
Short for "Neo-Luddism". Neo-Luddites are named after a group of factory workers from 18th century England who were put out of a job due to the invention of the power loom, and subsequently raided their former employer's factory and destroyed all the machines. The term "Luddite" is used today to describe conservatives with a narrow view of which types of technological progress are tolerable or permissable, and an unduly negative view of technology in general. Luddites are famous for abuse of the Precautionary Principle. See the Wikipedia definition of the PP here, and a transhumanist analysis of the PP here.

LOGI:
See Levels of Organization in General Intelligence.

Massive Modularity Hypothesis:
The hypothesis originally presented by cognitive scientist Jerry Fodor that the human brain is composed of domain-specific neural modules (as opposed to a homogeneous blank slate), which specialized to perform specific tasks [Fodor83]. In the past decade, neurologists and evolutionary psychologists have amassed an overwhelming amount of evidence in favor of the Massive Modularity Hypothesis, and continue to defend it from critics [Sperber02]. Some of the strongest evidence for Massive Modularity comes from patients with brain damage in specific areas. There are hundreds of thousands of examples of stroke or injury disabling cognitive functionality in very specific domains while leaving other areas totally intact. One example would be the inability to recognize faces, called prospagnosia. See also evolutionary psychology.

mature nanotechnology:
Eric Drexler's technical book on nanotechnology, "Nanosystems" describes, for example, acoustic nanocomputers using "rod logics" composed of diamondoid rods with hundreds or thousands of atoms each, moving at the speed of sound in diamond, providing 10^21 ops/sec from a one-kilogram nanocomputer. This level of nanotechnology is best thought of as representing the nanotechnological equivalent of vacuum tubes. "Mature nanotechnology" is nanotechnology that bears the same resemblance to rod logics as, say, a modern ultra-optimized VLSI chip bears to ENIAC. This would cover electronic nanocomputers verging on computronium, extremely small (~100 nanometer or ~10 nanometer, rather than ~1 micron) nanobots capable of swarming to perform complex manufacturing activities (including reproduction) in a natural non-vat environment, ultra-optimized molecular specifications, error-tolerant nanomachinery, and so on [Yudkowsky01].

Meaning of Life FAQ, The:
Document by Eliezer Yudkowsky on the (Interim) Meaning of Life. Contains interesting answers to many common questions, such as "Why should I get out of bed in the morning?" Located here: http://yudkowsky.net/tmol-faq/meaningoflife.html. See also Singularitarianism, Yudkowsky, Eliezer.

meme:
An idea considered as an organism, a replicator, and a focus of selection pressures exerted by human minds. Ideas primarily reproduce through linguistic transmission from one human to another; they mutate when individuals make errors in the retelling, or deliberately tweak the presentation of the meme. Almost everything in human society qualifies as a meme or the extension of a meme; a newspaper is filled with buzzing memes, a chair was created through a chair-making meme passed down through the industry, breakfast cereal comes from a conglomeration of food-production and marketing memes, and so on. Although the term "meme" is sometimes derogatory, memes can be positive forces as well - for example, when a meme relies on positive emotions or truthful thinking to propagate itself. Memes exist on the substrate of gene-machines, or thinking intelligences. They are therefore subject to the unique and specific limitations and advantages of that substrate. For example, it is extremely difficult or impossible for humans to form memes about 8D objects, the quantum realm, selfless goal systems, or any meme that requires seamlessly integrating more than a few tangible pieces of information in short-term memory. See also veme.

MEST Compression:
Matter/Energy/Space/Time compression. Term used by systems theorist John Smart in his systems theory approach to the Singularity, on his website, www.SingularityWatch.com. See also Institute for Accelerating Change, Accelerating Change Conference, Smart, John.

metamorality:
The study of structural aspects of human morality, focusing on what makes human morality different than (say) banana slug morality, as opposed to what makes humans different from each other. Metamorality is the study of human universals, and how these universals fit into the space of "minds in general". Studying metamorality can give us insight into traditional morality as well. Our metamoralities are what underlie our ability to choose between what we usually call "moralities" or "value systems". Metamorality also attempts to answer questions like "to what extent does human altruism mirror idealized altruism?" or "why did human morality evolve the way it did?" Metamorality lies not in the domain of those usually expected to argue about morality, such a theologians, humanists, philosophers, and talk show hosts, but in the domain of theoretical cognitive science. See also Friendship architecture, goal systems, morality.

metamoral turbulence:
Unharmonious changes in moral decisionmaking exhibited by a self-modifying entity making deep changes to its value system and intellectual capacities under recursive self-enhancement and wholescale architectural revisions. Could be very minor, or major, depending on the initial design, the progression of that design under self-modification, and the inherent stability of the moral possibility space outside of humanity. (There's nothing to suggest that it would be impossible to create an AI with a morality that stays fixed indefinitely, regardless of self-reprogramming and learning, because the goals of self-reprogramming would simply be to maximize fulfillment of the morality, that is, the supergoal. If the self-improving AI eventually acquired high levels of physical and intellectual power, it's unlikely that external forces could intervene and reprogram the AI.) If heavy-duty metamoral turbulence is inevitable, then the self-modifying being, especially if it cared for other sentients (as it hopefully would), would need to slow down the improvement process and proceed cautiously, as apparently innocuous changes might lead to radical, unexpected transformations of values. These undesirable changes could lead to the creation of a superintelligence indifferent to humans (or possibly anything besides itself and its own goals). The perceived likelihood of this scenario depends on our intuitions about minds in general and our acquaintance with Friendliness theory [Yudkowsky01].

minds-in-general:
Any type of mind permitted by the laws of physics (we may even go beyond this and consider minds obeying any laws of physics). The idea of minds-in-general is extremely useful for thought experiments designed to minimize anthropomorphism when considering not-yet-existent minds, such as cybernetically enhanced human minds or Artificial Intelligences. We can review our evolutionary history and see where humanity falls in the space of minds-in-general, (it turns out that we're a noncentral special case of intelligence, rather than a typical case) we can make guesses about the boundaries of the space of minds-in-general, hypotheses about universal characteristics of minds, and so on. This field of study has only been born in the past few years, and has been growing slowly since. It's an interdisciplinary field involving (at least) information theory, biology, psychology, evolution, decision theory, and mathematics. Theories of minds-in-general are our guide to accurately imagining some aspects or qualities of minds that don't exist yet. For example, if a mind is being run on components a billion times more rapid than human neurons, then we shouldn't expect that mind to think at human speed, but a billion times faster. Scientific theories of minds in general are rooted in causal functionalism. See also anthropics, Ethical Issues in Advanced Artificial Intelligence, Levels of Organization in General Intelligence.

Mirror:
An as-yet-unimplemented idea for training humans to think more intelligently by finding localizable hardware substrate for such cognitive adaptations as hatred and rationalization, then training humans to deliberately avoid the use of those abilities by asking them to philosophize in an fMRI machine [Yudkowsky01].

molecular manufacturing:
Another name for nanotechnology, or the creation of arbitrary structures using molecular-scale machinery. In recent years, the word "nanotechnology" has been co-opted by microtechnologists, so genuine nanotechnologists are often forced to use the phrase "molecular manufacturing" when referring to molecular-scale machinery. See also Center for Responsible Nanotechnology, nanotechnology.

Moore's Law:
Intel co-founder Gordon's Moore's observation that the number of transistors on a chip of fixed price tends to double every 18 months. There's a lot of argument over what Moore actually meant, but at some point, it doesn't really matter precisely what he meant; the doubling trend is pervasive and obvious, and extends to nearly all forms of advanced technology (as opposed to computer chips alone). We know what he was getting at. See further commentary on Moore's Law as it pertains to Singularity issues at http://www.nickbostrom.com/superintelligence.html.

morality:
"Morality" is a tricky and politically valent word, and there isn't widespread agreement as to its proper use. Dictionary.com defines morality as;

1. The quality of being in accord with standards of right or good conduct.
2. A system of ideas of right and wrong conduct: religious morality; Christian morality.
3. Virtuous conduct.
4. A rule or lesson in moral conduct.

Some writers talk about morality as something deep and human-universal, something that certainly exists yet cannot be seen directly, like "brotherhood". Others talk about morality in disparaging terms, referring to destructive, self-righteous dogmas. Some refer to morality as underlying ethics, some visa versa. On this site, and in common usage within Singularity and AGI communities, the convention is to refer to "morality" as the overall goal system of a given mind, in a deep and general way. "Ethics" would then be the surface heuristics we use to steer ourselves towards moral outcomes. When talking about morality among humans, we (the authors of this website) give humanity the benefit of the doubt, referring to "morality" as the goodness in each human, the inclination to be kind, nonviolent, and understanding whenever possible. We tend to say that human morality can be pretty good, but we want it to be better, and we want to transfer that fundamental goodness, that ability to learn from mistakes and improve, into any new species we create, especially if that species possesses greater-than-human intelligence. (Transferring this goodness effectively might require leaving out certain parts like hatred and self-centeredness.) Certain formulations of volitionism figure that the best morality would be one that values the right of every sentient entity to do whatever they want without violating the volition of others, along with safeguards to protect everyone from accidentally doing something to themselves anything they clearly wouldn't want. See also Friendship architecture, moral turbulence, metamorality.

moral turbulence:
Our day to day moral challenges, the seemingly autonomic rising and falling of personal scrupulousness due to various internal and external factors, often beyond our control. A study of male college students once indicated that 50% would commit rape if they knew there would be no consequences [O’Donohue et al. 97]. It sometimes seems that the improvement of humanitarian conditions throughout history is not a result of direct human choice, but the creation of legal systems which seem to act as a sort of rudder for humanity's navigation through the fabric of moral space; they create higher-order rule enforcing agencies for the sake of preserving a minimum set of moral standards that are generally agreed upon. Considering moral turbulence in humans sometimes allows us to determine the underlying critical variables of these patterns and extend them to create models of how Artificial Intelligences might act under certain goal systems and cognitive architectures. For example, here are a few guesses about how minds behave morally or immorally. Critical variables in predicting how moral an agent will behave at any moment will include; 1) how much information they have about the situation, 2) the quantity and quality of intelligence available to gather data and weigh evidence, 3) their ability to understand and achieve goodness, 4) their desire to understand and achieve goodness. In humans, evolved instincts can supervene at any of these points to decrease the chances of making a satisfying moral decision. The first transhuman intelligence must exceed human capacity in all these areas, to avoid unnecessary risk. See also Friendship structure, Friendly AI.

Moravec, Hans:
Research Professor for the Carnegie Mellon Robotics Institute. Focus on autonomous robots and robotic vision. Moravec is the author of "Mind Children" "Robot", and hundreds of articles on machine intelligence and robotics. He argues that fully intelligent robots will emerge by 2050. Homepage: http://www.frc.ri.cmu.edu/~hpm.