Accelerating Future Lexicon:

Sandberg, Anders
Neuroscience doctorate and Long-time transhumanist. Author of the gigantic Anders Transhuman Page. Also see his personal web page.

Sans Ceiling Hypothesis:
The speculative notion that there may be no upper limit to what sufficiently advanced intelligent life can accomplish, as opposed to the view that the universe possesses unbreakable limits in the form of physical laws. The rationale for this belief is that intelligence has overcome "unbreakable laws" time and time again - and it is uncertain if this process will continue indefinitely or eventually halt. Paul Hughes' term. Very speculative. See also ontotechnology.

Schmidhuber, Jürgen:
Researcher on Artificial Intelligence, seed AI, information theory, and sequential decision theory. Co-worker of Marcus Hutter's. Personal website at http://www.idsia.ch/~juergen/.

seed AI:
An AI capable of self-improvement, self-revision, and recursive self-enhancement. It is supposed that once a seed AI gains a certain degree of intelligence, it could entirely take over the job of programming and improving itself. This could result in an open-ended cycle of intelligence improvement, a Singularity. A strong case for the potential intelligence and speed of a typical AI has already been made, based on hardware advantages intrinsic to machine substrates. These advantages could be applied to the task of inventing additional cognitive enhancements, resulting in improvements to the quantity and quality of cognition, which could in turn be applied to devising still more enhancements. When viewed from human timescales, the self-improvement process might take place quite rapidly, resulting in a roughly human-equivalent AI one day and a full-fledged superintelligence the next. In fact, the notion of "human-equivalency" in AI is probably anthropomorphic; we shouldn't expect the threshold of recursive self-improvement to correspond to the exact pattern of domain competencies exhibited by Homo sapiens. This quick jump from human-equivalency to superintelligence is commonly labelled a "hard takeoff". See http://www.singinst.org/intro/seedAI.html. See also Friendly AI, recursive self-improvement.

self-translucent intelligence:
See also autosentience, recursive self-improvement.

shaper:
A piece of neurological machinery or cognitive regularity underlying the human ability to make moral decisions. Shapers stem from complex functional adaptations that are panhuman; they vary quantiatively rather than qualitatively. The spectrum of ways in which they respond to the environment and other human beings introduces another layer of variance, but the underlying cognitive hardware is qualitatively identical across the whole of humanity. It has been proposed that a Friendly AI learn to seek out these shapers and integrate them (wisely) into its interim model of altruism ("Friendliness"). See also Creating Friendly AI.

Shock Level Zero (SL0):
From Eliezer Yudkowsky's "Future Shock Levels" scale, someone at SL0 is "...comfortable with modern technology - not so much the frontiers of modern technology, but the technology used in everyday life. Most people, TV anchors, journalists, politicians." See also Future Shock Levels.

Shock Level One (SL1):
From Eliezer Yudkowsky's "Future Shock Levels" scale, SL1 is "Virtual reality, living to be a hundred, "The Road Ahead", "To Renew America", "Future Shock", the frontiers of modern technology as seen by Wired magazine. Scientists, novelty-seekers, early-adopters, programmers, technophiles." See also Future Shock Levels.

Shock Level Two (SL2):
From Eliezer Yudkowsky's "Future Shock Levels" scale, SL2 is "Medical immortality, interplanetary exploration, major genetic engineering, and new ("alien") cultures. The average SF fan." See also Future Shock Levels.

Shock Level Three (SL3):
From Eliezer Yudkowsky's "Future Shock Levels" scale, SL3 is "Nanotechnology, human-equivalent AI, minor intelligence enhancement, uploading, total body revision, intergalactic exploration." See also Future Shock Levels.

Shock Level Four (SL4):
From Eliezer Yudkowsky's "Future Shock Levels" scale, SL4 is "The Singularity, Jupiter Brains, Powers, complete mental revision, ultraintelligence, posthumanity, Alpha-Point computing, Apotheosis, the total evaporation of "life as we know it." See also Future Shock Levels.

SL4 mailing list:
Shock Level 4 mailing list. Futurist mailing list moderated by Eliezer Yudkowsky. From its website, sl4.org: " The SL4 mailing list is a refuge for discussion of advanced topics in transhumanism and the Singularity, including but not limited to topics such as Friendly AI, strategies for handling the emergence of ultra-powerful technologies, handling existential risks (planetary risks), strategies to accelerate the Singularity or protect its integrity, avoiding the military use of nanotechnology and grey goo accidents, methods of human intelligence enhancement, self-improving Artificial Intelligence, contemporary AI projects that are explicitly trying for genuine Artificial Intelligence or even a Singularity, rapid Singularities versus slow Singularities, Singularitarian activism, and more." See also Yudkowsky, Eliezer.

smartness:
A word sometimes used to refer to inter-species differences in intelligence, as opposed to variations within the human range only. For example, it seems a fair bet that humans are smarter than chimps, where humans would be less smart than superintelligences. "Smartness" determines the kind of problems you can notice or solve, and the elegance and effectiveness of the solutions. Although "ants are to humans as humans are to X" doesn't necessarily imply a solution for X, arguments from cognitive science and evolutionary psychology push a strong case for the cognitive nonoptimality of humans and likelihood of a physically possible X, superintelligence. "Staring Into the Singularity" is an interesting introduction to smartness.

sphexishness:
A tolerance for repetition. Artificial Intelligences, without biological limitations, will likely possess the ability to think and work continuously on an arbitrary task without becoming bored, a fantastic display of sphexishness relative to human beings. Douglas Hofstadter's term.

singleton:
In the near future, humanity's technological capacity and cognitive complexity may increase so greatly as to render traditional forms of political organization obsolete. A "singleton" would be a top-level decision-making agency, preventing "Tragedies of the Commons" and orchestrating other civilization-wide matters, possibly executing actions in accordance with the agglomerative will of a society, or a renormalized derivative thereof. Despite the name, a singleton doesn't have to be monolithic; a worldwide democratic government could qualify as a singleton. Part of the need for a singleton may result from the simple fact that as technology advances, small fringe groups will be capable of doing greater amounts of damage to society as a whole. Just as it was adaptive for collectives of cells in prehistoric times to evolve brains and become full-fledged organisms, our global civilization may need to create a stable and trustworthy central decision-making agency in order to preserve our safety and freedom to pursue happiness. The head decision-makers need not be egocentrically biased or even sentient - neither of these qualities seem required for the ability to make wise decisions that benefit everyone (which simply entails certain complex types of information-processing). The term "singleton" was originally coined (www.nickbostrom.com/fut/singleton.html) by Oxford philosopher Nick Bostrom. See also Sysop.

Singularitarian:
The philosophy that creating a benevolent smarter-than-human is the most leveraged way to address the problems humanity currently faces or will face, and a potential path to achieving a very large quantity of objective good. The Singularitarian Principles state; "Singularitarians are the partisans of the Singularity. A Singularitarian is someone who believes that technologically creating a greater-than-human intelligence is desirable, and who works to that end. A Singularitarian is advocate, agent, defender, and friend of the future known as the Singularity". See The Meaning of Life FAQ. See also Anissimov, Michael, Singularity Institute for Artificial Intelligence, Yudkowsky, Eliezer.

Singularitarianism:
The philosophy of Singularitarians. See "And Now for Something Completely Different" or "The Singularitarian Principles" for more information.

Singularitarian Principles:
A document by Eliezer Yudkowsky describing some of the common principles among Singularitarians. Yudkowsky possesses no particular authority over Singularitarianism, but someone had to write the document, and he was the first to do so. Most Singularitarians generally agree on these principles. Available at http://yudkowsky.net/sing/principles.html.

Singularity:
The transition in human history marked by the technological creation of greater-than-human intelligence; the end of the era of "strictly human intelligence" that has characterized the last fifty thousand years. If one were to draw a single line, dividing into two parts the history of Earth-originating intelligent life, the line would be drawn at the Singularity. See "What is the Singularity". See also neurohacking, recursive self-improvement, seed AI.

Singularity advocacy:
Advocacy of the Singularity as a humanitarian goal. Most, if not all, AGI researchers are Singularity advocates. Many transhumanists may be considered Singularity advocates. Granted, who qualifies as a "Singularity advocate" depends on your precise definition of "Singularity"; by some accounts there are thousands of "Singularity advocates", by others, less than a dozen. Singularity advocates argue for the importance of the Singularity in online and offline fora, and write documents conveying that view. See http://yudkowsky.net/sing/principles.html#activism and http://www.singinst.org/why-singularity.html. See also Singularitarianism.

Singularity analysis:
Objective analysis of the Singularity as a possible-but-not-certain future event. Singularity analysis often goes hand in hand with Singularity advocacy, because maximizing the likelihood of a benevolent Singularity occuring is easiest when the best available model can be used to select among potential goals. Even the most enthusiastic Singularity advocates tend to associate with the most sober Singularity analysts, conducting analyses side by side. The most prominent Singularity analyst (outside of Singularity activists) is John Smart, systems theorist and author of SingularityWatch.com and President of the Institute for Accelerating Change. All Singularity advocates are also Singularity analysts, although the inverse does not hold.

Singularity, cognitive interpretation of:
Interpretation of the Singularity as a primarily cognitive event, that is, the improvement of the underlying brainpower traditionally used to create improvements in the first place, rather than some "really big" technological or cultural event. The cognitive interpretation of the Singularity views the transition from Homo sapiens to "something slightly more intelligent than Homo sapiens" by analogy to the transition from Homo erectus to Homo sapiens, or the transition from chimps to humans, and makes no analogies for "something significantly more intelligent than Homo sapiens". The cognitive interpretation of the Singularity observes the characteristic limitations of biologically instaniated thought in comparison to the inherent terrain and possibilities of nonbiologically instaniated thinking processes. The cognitive interpretation of the Singularity is the central interpretation of Singularitarians, Singularity advocates, and the Singularity Institute. See also Singularity, sociotechnological interpretation of.

Singularity fundraising:
Gathering resources for the eventual creation of seed AI, (whether that be 10 years, 30 years, or 100 years away) continued research in the areas of AI morality and general intelligence, and interim efforts designed to convey the importance of AI and the Singularity to forward-thinking groups. In the few years it has been around, Singularity fundraising has actually been somewhat successful, but is still just getting started. Singularity donors often come from organizations like the Foresight Institute that commonly discuss futurist issues and the potential risks and benefits of advanced technology.

Singularity Fun Theory:
Paper by Eliezer Yudkowsky answering commonly-asked questions about fun and happiness in the indefinitely far future. Observes that the volume of "Fun Space" appears to increase exponentially given a linear increase in intelligence. Available at http://yudkowsky.net/essays/funtheory.html.

Singularity-ready:
An AI or enhanced human being trustworthy and robustly benevolent enough to enter into recursive self-improvement and spark the Singularity. We can never have 100% trustworthiness, but either a Singularity or a major event is eventually going to occur whether we like it or not, and it seems prudent to intentionally create benevolent smarter-than-human intelligence rather than let be created accidentally. There is also the worry that the power of our technology is increasing exponentially while our intelligence and wisdom are not, putting us in constant risk. Proposals to increase our intelligence or wisdom within the biological boundaries of Hoomo sapiens seem naive; for example, it has been shown that all human beings display a certain measure of selfishness whether they are aware of it or not. New minds could be designed lacking these cognitive inconveniences and biases.

Hopefully we can attain a pleasant outcome by transferring over the same worries we have about Singularity trustworthiness to the Singularity sparker it/her/himself.

Singularity, sociotechnological interpretation of:
Interpretation of the Singularity as a kind of culmination of traditional technological and social progress, with minimal emphasis on the qualitative intelligence advantages of transhumans and the collapse of human understanding in the face of smarter-than-human intelligence. The sociotechnological interpretation of the Singularity draws primarily from evidence in the field of accelerating change, whereas the cognitive interpretation draws from evidence in cognitive science and evolutionary psychology. Regardless of which interpretation is right, the former seems easier to understand and communicate to laypeople. This could have positive or negative effects depending on the differential validity of the models. Many would argue that Ray Kurzweil's interpretation of the Singularity is primarily sociotechnological, while the Singularity Institute's is not. See also Singularity, cognitive interpretation of.

SIAI:
See Singularity Institute for Artificial Intelligence.

Singularity Institute for Artificial Intelligence:
Nonprofit organization founded in July 2000 for the sole purpose of implementing a safe and swift Singularity. The Institute's name includes "Artificial Intelligence" in the title because the consensus was that AI is probably the easiest Singularity technology to accelerate and influence positively relative to other Singularity technologies. Website: http://singinst.org.

Smart, John:
Accomplished academic, former business owner, systems theorist, and Singularity analyst. President of the Institute for Accelerating Change. Author of the website www.SingularityWatch.com.

species AI:
An AI created to represent the wishes of its parent species, rather than some kind of etheral "AI species" or particular individuals within the parent species (such as its creators). If the species AI cannot perfectly represent the wishes of its parent species, then it should come as close as possible to doing so. A species AI should be fair. A species AI must be trustworthy and morally stable. A species AI is what we want the first real AI to be; any other path is almost certainly too large a risk. See also Friendly AI.

spirothete:
From the Latin spiro-, meaning to breathe, blow, draw breath; to be alive; to have inspiration; be inspired, and -thete, meaning "entity". A spirothete is a name for a synthetic mind. Originally coined by a man known only as "Hermit", from the Virus organization.

Standard Social Science Model (SSSM):
The standard model of human nature as used in popular culture and most social science. In the Standard Social Science Model, humans are blank slates primarily influenced by the patterns of culture and upbringing rather than their genes. The full definition of the SSSM is much more complicated and nuanced, and is explained by evolutionary psychologists Leda Cosmides and John Tooby in the article "The Psychological Foundations of Culture" in the book "The Adapted Mind". See also Cosmides & Tooby, Integrated Causal Model.

strong superintelligence:
Intelligence qualitatively smarter than humans, as opposed to "merely" faster-than-human intelligence. Here, the word "smarter" is only used to imply a superiority in areas such as pattern recognition, ability to plan, and problem-solving horsepower; it does not entail greater-than-human wisdom, social skills, or morality. It does, however, imply a powerful ability to achieve real world goals, including high-resolution modeling of the minds of other agents, the further enhancement of intelligence, and the consolidation of physical power. How superintelligences will behave towards other sentients will be dependent on their initial motivations and how those motivations transform and develop under recursive self-improvement. Superintelligence is likely to eventually emerge from either the field of Artificial Intelligence or that of Brain-Computer Interfacing, where incremental advancements will eventually snowball into a positive-feedback loop of cognitive self-enhancement.

At the moment, we can't really say how exactly how far below the theoretical limits of intelligence humans are. There are some factors worth thinking about, however. Humans possess about a hundred billions neurons that each fire about 200 times per second.

While the statement "you only use 10% of your brain" is a flat-out myth, it is true that the firing of each individual neuron is not functionally relevant to general cognition all of the time. A great number of neurons have to fire in unison to even have a chance at influencing our global cognition or behavior. Most of the brain evolved in the total absence of general intelligence, and is therefore poorly suited for implementing it. This fact means that the biological human brain can hardly be called an optimal instance of intelligence, in fact suffering deficits in the areas of efficiency, elegance, performance, and more. This is nothing to be terribly ashamed about per se; this is what we should expect from an instance of intelligence produced on the basis of evolutionary criteria alone. When general intelligence first evolved, it co-opted many preexisting structures within the brain; structures not necessarily optimal for the functions of intelligence. Neurons do not regenerate, they are not reprogrammable, they cannot be upgraded, repaired, rearranged, or added. All these reasons lead Singularity analysts to believe that humanity is hardly the final word when it comes to qualitative intelligence.

A strong superintelligence might be able to quickly generate satisfying solutions to problems that have plagued humanity throughout the centuries, including problems we have been too dumb to notice.

There have been objections to the idea of strong superintelligence, implying that there are those who view humanity as already being at a theoretical upper ceiling in terms of qualitative intelligence. The idea of strong superintelligence being physically possible is sometimes referred to as the strong superintelligence hypothesis or the singularity hypothesis. The notion of the "Singularity" is related to the strong superintelligence hypothesis, but not dependant upon it. See also Singularity, recursive self-improvement, weak superintelligence.

subgoal:
Servant goal created for a supergoal, based on a probabilistic confidence estimate that the creation of the subgoal will lead to greater fulfillment of the supergoal. Many subgoals will take the form of heuristics, rules of thumb for achieving goals in the real world. Some subgoals might not appear like subtasks of the supergoal at first glance, rather contributing something to the entire network of subgoals, serving the supergoal in that way. Some subgoals may be convergent, present across many or all classes of theoretically possible minds, such as "obtain a power source" or "gather perceptual information". "Protect myself" may be a convergent subgoal, but by no means is it universal. (We can imagine classes of AI systems whose behavior is basically selfless.) See also convergent subgoals.

supergoal:
In AI design, the top-level goal of an AI system. An AI will always have a supergoal, even if it is better described as the aggregate effect of its subgoals, or the programmers didn't explicitly mean for a supergoal to exist. In a "cleanly causal" goal system, subgoals are created solely in the service of supergoals. An AI with "supergoals" need not be monomaniacal or "obsessive" about them, despite the connotation of the name. For more on goal systems, see "An Introduction to Goal Systems".

superintelligence:
An intelligence with a level of smartness and problem-solving horsepower enormously in advance of that possessed by any human or group of humans. Despite common usage, there is probably no such thing as a "superintelligent AI"; a true superintelligence would no more resemble an AI than a human resembles an amoeba. Singularity analysis commonly distinguishes between weak superintelligence, which is human thought at enormously higher speeds, and strong superintelligence, involving an intelligence difference similar to that distinguishing a human from a chimpanzee, or a dog, or a possibly a rock.

It has been suggested (6) that "transhuman" should refer to a mind with perhaps three or four times the computing capacity (or cognitive capacity) of a human, that "superintelligence" should refer to a mind with perhaps a hundred or a thousand times human capacity, and that "Power" (a la Vinge) should refer to a mind with billions or trillions of times the computing power of Earth's entire current population. It is noteworthy that even a transhuman is still smarter than we are and thus fundamentally incomprehensible, and also that a desktop rod logic suffices for superintelligence under this definition [Yudkowsky01]. See also Power, transhuman intelligence hypothesis.

symbol:
An internal representation and simplification of an external or internal object, statistical regularity, or concept, for the purposes of cognition.

symbol grounding:
Giving mental symbols "grounding" by fleshing them out with internal complexity, connections to perceptual data, past experiences, and other symbols. The field calling itself "AI" has traditionally had problems with symbol grounding, creating vast knowledge bases consisting of English sentences without giving these sentences much internal structure or underlying meaning. The root of this problem probably lies in the fact that introspectively visible components of thoughts often consist of English sentences, so "AI researchers" assume that's what thought consists of. Relatively recent advances in neurology and other areas of cognitive science have made it very clear that these past strategies are obviously faulty, and symbols need to be grounded by multiple layers of complexity if they are going to work effectively.

Sysop:
A human or AI-derived superintelligence serving as an "operating system" for all the matter in the Solar System or beyond. A living peace treaty with the power to enforce itself. A friendly superintelligence ready to do whatever the "inhabitants" ask - including upgrading us to superintelligence ourselves - so long as it doesn't violate the citizenship rules derived from Friendliness. (For example, asking the Sysop to kill someone else, or to use more than your share of the Solar System's resources, or even to upload and upgrade someone against their will.) One of the possible outcomes of the Singularity; see Interlude: Of Transition Guides and Sysops [Yudkowsky01]. Sometimes called a "Sysop Scenario". A Sysop might be an agglomeration of superintelligences rather than a "single" one, and would certainly need to lack and selfishness or self-righteousness to fill its role appropriately (which probably wouldn't be too hard for a superintelligence to pull off; it looks like there's a good chance that even humans would be able to create selfless and non-self-righteous entities in the form of Friendly AIs.) See also Friendly AI, singleton.

Sysop FAQ:
Mitch Howe's insightful FAQ on Sysop Scenario, a possible outcome of a successful Singularity. Located at the Empowerment Academy website.

takeoff-safe AI:
An AI that has met or exceeded its programmers' standards for moral reasoning and possesses enough integrity to safely enter recursive self-improvement. It's tough to say when this decision should be made, but if the AI clearly exhibits superhuman moral reasoning, it would probably be unethical and adversarial to attempt to restrain it from pursuing real-world goals in the service of good. See also Friendliness, Singularity-ready.

Transcension Scenario:
A scenario whereby civilizations that undergo a Singularity transcend beyond this universe as a computational arena, leaving no trace behind. This has been proposed by Singularity analyst John Smart as a solution to the Fermi Paradox. Somewhat speculative. See Smart's paper at http://www.transhumanist.com/Smart-Fermi.htm.

terminal failure:
Final failure. In seed AI design, this class of failure probably indicates a catastrophic failure of Friendliness. "Terminal" both in the sense that the AI's goal system would be deteriorating, and that humanity might not survive if the AI were able to reach human-indifferent recursive self-improvement. See also paperclip AI, Riemann Hypothesis Catstrophe, thermostat AI, UnFriendly Singularity.

terraced scan:
Douglas Hofstadter's term for how humans usually approach information. Progressively more focused stages of scanning/analysis, based on what the previous scan turned up as interesting. We often use the terraced scan approach in searching for mates, business partners, books at a library, or web pages on Google.

thermostat AI:
Eliezer Yudkowsky's term for an AI with a very simple goal system, like that of a thermostat. Sometimes used to signify an AI with a very simple goal system that has entered recursive self-improvement, conducting improvements to its own hardware and intelligence, all in service of this simple goal - a very dangerous scenario from the viewpoint of humans. See also paperclip AI, Riemann Hypothesis Catastrophe, terminal failure, UnFriendly Singularity.

TMS:
See transcranial magnetic stimulation.

tool-level AI:
Very infantlike, subhuman AI, capable of only incomplete facets of general cognition. Tool-level AI is below infrahuman AI and near-human AI, but above most software. Whether or not "tool-level AI" even exists yet at all is open for debate. Non-speculative. See also infrahuman AI, near-human AI, prehuman AI.

transcranial magnetic stimulation (TMS):
Stimulation of the skull with powerful magnetic pulses, for the aim of stimulating underlying brain tissue in specific areas. TMS is able to treat a variety of psychological ailments noninvasively, as opposed to most other techniques. Aside from its use in treating ailments, TMS has been reported to enhance abilities in normal people, although the jury is still out on that claim. TMS can also be used to selectively analyze the function of small areas of the cerebral cortex through activation or suppression, as seen on this MIT web page: http://www.ai.mit.edu/projects/medical-vision/surgery/tms.html. See also neurohacking.

transhuman:
"A level of ability substantially in advance of that possessed by humans. A transhuman AI is one which has substantially exceeded human intelligence. A further distinction can be made between weak transhumanity, which involves intelligence differentials roughly of the same order as those that distinguish Einstein from the average human, and strong transhumanity, involving an intelligence difference similar to that distinguishing a human from a chimpanzee. See also the definitions of weak and strong superintelligence." [Yudkowsky01] Somewhat speculative. See also Singularity, superintelligence, Power.

transhuman intelligence hypothesis:
The idea that qualitative improvements in human intelligence are possible, given the right hardware. If ultrafast minds are able to produce qualitatively better decisions (and far more of them) simply by virtue of extra time, then the transhuman intelligence hypothesis would still be considered true (because then even an accelerated mind could have a profound impact on the current society.) The transhuman intelligence hypothesis would be disproved if we created brains millions of times faster and more complex than human brains, yet consistently achieved human-level results on standardized tests such as IQ tests. Somewhat speculative. See also Singularity, smartness, strong superintelligence, weak superintelligence.

transhumanist:
As defined by the World Transhumanist Association, "Transhumanists seek to expand technological opportunities for people to live longer and healthier lives and to enhance their intellectual, physical, and emotional capacities". Many Singularity analysts, advocates, and AGI researchers are transhumanists, but not all. Many transhumanists argue that transhumanism is the logical extension of secular humanism.

Transition Guide:
"For the last fifty thousand years, humanity has been essentially at the mercy of a hostile universe. Post-Singularity, things will be different. A Transition Guide is a Friendly seed AI that has achieved superintelligence and created nanotechnology, or whatever other tools are needed to (a) perceive whatever state of existence comes after the Singularity and (b) handle the transition. A Transition Guide is the bridge between modern-day humanity and the other side of a Friendly Singularity." [Yudkowsky01] Somewhat speculative. See also Friendly AI, Friendly Singularity, Sysop.

unity of will:
The point at which a Friendly AI in the process of being created starts to spontaneously generate similar goals to programmers attempting to approximate idealized altruism. The point at which people can start to stop worrying so much. The value of the concept of unity of will is that it can demonstrate to skeptics of the possibility of moral AI that an AI can be used to spark a Singularity equally as humanitarian as a human-sparked variant. Unity of will is explained in "Creating Friendly AI" here: http://www.singinst.org/CFAI/design/seed.html#unity, but it is recommended that the reader start from the beginning [Yudkowsky01].

UnFriendly Singularity:
A Singularity that harms humanity or prevents us from achieving our full potential, where a Singularity means a transhuman intelligence giving rise to a feedback loop of intelligence improving and creating more intelligence on telescoping timescales. Most UnFriendly Singularities are more likely to kill humanity outright than curtail it while simultaneously preserving its ability to survive. Happy minds are an incredibly specific configuration of matter; unless a given superintelligence understands what happy minds are and why they are desirable, we shouldn't expect them to be created spontaneously, or for existing happy minds to be preserved. Most physically possible goal systems for transhuman intelligences are probably too simple to contain the rich complexity underlying altruism, empathy, and volitionism, hence the need for Friendliness and Friendly AI. See also UnFriendly superintelligence.

UnFriendly superintelligence (UFSI):
A malevolent or human-indifferent being with a level of ability enormously beyond humans. If the superintelligence is unrivalled, the complete extinction of the human race, and perhaps all animal and plant life could take place. An UnFriendly superintelligence need not be deliberately malevolent to constitute a huge threat; a superintelligence viewing humans as building materials for its own personal value-structures would be just as dangerous. An UnFriendly superintelligence is definitely something we want to avoid, which might best be done with the assistance of a Friendly superintelligence.

veme:
You have genes, the units of selection for life, memes, the units of selection for human ideas, and then you have vemes, which would correspond to whatever units of information superintelligences shuffle between one another and within themselves. We can't say very much about vemes at this point except that, if they ever come into existence, they're likely to be shuffled around at much greater rates and in greater numbers than genes and memes are.

Vinge, Vernor:
"Hard" science-fiction writer with decades of stories under his belt, including "Marooned in Realtime" and "A Deepness in the Sky". Vernor Vinge originally coined the term "Singularity" and has spoken on it to scientific as well as science-fictional audiences. Vinge is also a mathematician at San Diego State University. Most Singularitarians respect Vinge, but don't look up to or idolize him much. Homepage: http://www-rohan.sdsu.edu/faculty/vinge.

volition:
Someone's desire to do what they want, to be self-piloted and to self-actualize, and to avoid nonconsensual death, suffering, ignorance, pain, and so on. Some would argue that maximizing our own volition is the primary driver beyond all human progress and aspirations. A Friendly Singularity would channel the power of smarter-than-human intelligence into increasing our own volitional capabilities.

volitionism:
The philosophy that the definition of "good" grounds primarily in volition. Under volitionism, goodness roughly means "helping as many people as possible get what they really want, while helping them avoid what they don't want, unless this conflicts heavily with what other people want or don't want". As in utilitarianism, volitionism need not be a strict dogma, but simply a heuristic to guide us on the path to rightness.

Voss, Peter:
AGI researcher, head of the A2I2 project. Website: www.optimal.org.

Yudkowsky, Eliezer S. :
Research fellow and founder of the Singularity Institute for Artificial Intelligence. The most prolific writer on the topic of the Singularity and AI Friendliness. A primary activist within the Singularity movement. Website: www.yudkowsky.net.

References:

Bostrom, Nick. (1998). "How Long Before Superintelligence?" International Journal of Futures Studies, 2. http://www.nickbostrom.com/superintelligence.html

Bostrom, Nick. (2002). "Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards." Journal of Evolution and Technology, 9. http://www.nickbostrom.com/existential/risks.html

Bostrom, Nick. (2003). "Ethical Issues in Advanced Artificial Intelligence." Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003, pp. 12-17. http://www.nickbostrom.com/ethics/ai.html

Drexler, K. E. Engines of Creation: The Coming Era of Nanotechnology. (Anchor Books: New York, 1986). http://www.foresight.org/EOC/index.html

Fodor, Jerry (1983). The modularity of mind. Cambridge , Mass. MIT Press.

Goertzel, B (2001). "The Webmind AI Engine – a True Digital Mind in the Making". Webpage. http://www.goertzel.org/papers/SingularityPath.htm

Hutter, Marcus. (2003). "A Gentle Introduction to the Universal Algorithmic Agent AIXI". http://www.idsia.ch/~marcus/ai/aixigentle.htm

Moravec, H. Robot: Mere Machine to Transcendent Mind. (Oxford University Press: New York, 1999).

O'Donohue, William; Yeater, Elizabeth A.; Fanetti, Matthew. (2003). "Rape Prevention with College Males: The Roles of Victim Empathy, Rape Myth Acceptance, and Outcome Expectancies." http://www.northwestmedia.com/research/rape-i.pdf

Sperber, Dan. (2002). "In Defense of Massive Modularity". In Dupoux, E. Language, Brain and Cognitive Development: Essays in Honor of Jacques Mehler. 2002, Cambridge, Mass. MIT Press. 47-57. http://www.dan.sperber.com/modularity.htm

Tversky, A., and D. Kahneman. (1986). Rational choice and the framing of decisions. Journal of Business 59 (4, pt. 2): 251-278.

Wright, John. (2001). The Golden Age.

Yudkowsky, Eliezer. (2001). Creating Friendly AI 1.0. Publication of the Singularity Institute: http://singinst.org/CFAI/

Yudkowsky, Eliezer. (2001). General Intelligence and Seed AI 2.3. Publication of the Singularity Institute: http://singinst.org/GISAI/

Yudkowsky, Eliezer. (2003). Levels of Organization in General Intelligence. Publication of the Singularity Institute: http://singinst.org/LOGI/

Yudkowsky, Eliezer. (1999). "Future Shock Levels." Webpage. http://yudkowsky.net/sing/shocklevels.html

Yudkowsky, Eliezer. (2002). "The AI Box Experiment." Webpage. http://sysopmind.com/essays/aibox.html