Accelerating Future

I was recently informed that my abstract was accepted for presentation at the Society for Philosophy and Technology conference in Denton, TX, this upcoming May 26 – 29. You may have heard of their journal, Techné. Register now for the exciting chance to see me onstage, talking AI and philosophy. If you would volunteer to film me, that would make me even more excited, and valuable to our most noble cause.

Here’s the abstract:

Anthropomorphism and Moral Realism in Advanced Artificial Intelligence
Michael Anissimov
Singularity Institute for Artificial Intelligence

Humanity has attributed human-like qualities to simple automatons since the time of the Greeks. This highlights our tendency to anthropomorphize (Yudkowsky 2008). Today, many computer users anthropomorphize software programs. Human psychology is extremely complex, and most of the simplest everyday tasks have yet to be replicated by a computer or robot (Pinker 1997). As robotics and Artificial Intelligence (AI) become a larger and more important part of civilization, we have to ensure that robots are capable of making complex, unsupervised decisions in ways we would broadly consider beneficial or common-sensical. Moral realism, the idea that moral statements can be true or false, may cause developers in AI and robotics to underestimate the effort required to meet this goal. Moral realism is a false, but widely held belief (Greene 2002). A common notion in discussions of advanced AI is that once an AI acquires sufficient intelligence, it will inherently know how to do the right thing morally. This assumption may derail attempts to develop human-friendly goal systems in AI by making such efforts seem unnecessary.

Although rogue AI is a staple of science fiction, many scientists and AI researchers take the risk seriously (Bostrom 2002; Rees 2003; Kurzweil 2005; Bostrom 2006; Omohundro 2008; Yudkowsky 2008). Arguments have been made that superintelligent AI — an intellect much smarter than the best human brains in practically every field — could be created as early as the 2030s (Bostrom 1998; Kurzweil 2005). Superintelligent AI could copy itself, potentially accelerate its thinking and action speeds to superhuman levels, and rapidly self-modify to increase its own intelligence and power further (Good 1965; Yudkowsky 2008). A strong argument can be made that superintelligent machines will eventually become a dominant force on Earth. An “intelligence explosion” could result from communities or individual artificial intelligences rapidly self-improving and acquiring resources.

Most AI rebellion in fiction is highly anthropomorphic — AIs feeling resentment towards their creators. More realistically, advanced AIs might pursue resources as instrumental objectives in pursuit of a wide range of possible goals, so effectively that humans could be deprived of space or matter we need to live (Omohundro 2008). In this manner, human extinction could come about through the indifference of more powerful beings rather than outright malevolence. A central question is, “how can we design a self-improving AI that remains friendly to humans even if it eventually becomes superintelligent and gains access to its own source code?” This challenge is addressed in a variety of works over the last decade (Yudkowsky 2001; Bostrom 2003; Hall 2007; Wallach 2008) but is still very much an open problem.

A technically detailed answer to the question, “how can we create a human-friendly superintelligence?” is an interdisciplinary task, bringing together philosophy, cognitive science, and computer science. Building a background requires analyzing human motivational structure, including human-universal behaviors (Brown 1991), and uncovering the hidden complexity of human desires and motivations (Pinker 1997) rather than viewing Homo sapiens as a blank slate onto which culture is imprinted (Pinker 2003). Building artificial intelligences by copying human motivational structures may be undesirable because human motivations given capabilities of superintelligence and open-ended self-modification could be dangerous. Such AIs might “wirehead” themselves by stimulating their own pleasure centers at the expense of constructive or beneficent activities in the external world. Experimental evidence of the consequences of direct stimulation of the human pleasure center is very limited, but we have anecdotal evidence in the form of drug addiction.

Since artificial intelligence will eventually exceed human capabilities, it is crucial that the challenge of creating a stable human-friendly motivational structure in AI is solved before the technology reaches a threshold level of sophistication. Even if advanced AI is not created for hundreds of years, many fruitful philosophical questions are raised by the possibility (Chalmers 2010).

References

Bostrom, N. (2002). “Existential Risks: Analyzing Human Extinction Scenarios”. Journal of Evolution and Technology, 9(1).

Bostrom, N. (2003). “Ethical Issues in Advanced Artificial Intelligence”. Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence.

Bostrom, N. (2006). “How long before superintelligence?”. Linguistic and Philosophical Investigations 5 (1): 11–30.

Brown, D. (1991). Human Universals. McGraw Hill.

Chalmers, D. (2010). “The Singularity: a Philosophical Analysis”. Presented at the Singularity Summit 2010 in New York.

Good, I. J. (1965). “Speculations Concerning the First Ultraintelligent Machine”, Advances in Computers, vol 6, Franz L. Alt and Morris Rubinoff, eds, pp 31-88, Academic Press.

Greene, J. (2002). The Terrible, Horrible, No Good, Very Bad Truth about Morality and What to Do About it. Doctoral Dissertation for the Department of Philosophy, Princeton University, June 2002.

Hall, J.S. (2007). Beyond AI: Creating the Conscience of the Machine. Amherst: Prometheus Books.

Omohundro, S. (2008). “The Basic AI Drives”. Proceedings of the First AGI Conference, Volume 171, Frontiers in Artificial Intelligence and Applications, edited by P. Wang, B. Goertzel, and S. Franklin, February 2008, IOS Press.

Pinker, S. (1997). How the Mind Works. Penguin Books.

Pinker, S. (2003). The Blank Slate: the Modern Denial of Human Nature. Penguin Books.

Rees, M. (2003). Our Final Hour: A Scientist’s Warning : how Terror, Error, and Environmental Disaster Threaten Humankind’s Future in this Century – on Earth and Beyond. Basic Books.

Wallach, W. & Allen, C. (2008). Moral Machines: Teaching Robots Right from Wrong. Oxford University Press.

Yudkowsky, E. (2001). Creating Friendly AI. Publication of the Singularity Institute for Artificial Intelligence.

Yudkowsky, E. (2008). “Artificial Intelligence as a positive and negative factor in global risk”. In N. Bostrom and M. Cirkovic (Eds.), Global Catastrophic Risks (pp. 308-343). Oxford University Press.

Tweet

25 Responses »

Here it is. Congrats to Nikola for getting the interview. I was fortunate enough to have a one-on-one conversation with Stephen Wolfram at Singularity Summit 2009 and I found him extremely clever.

9 Responses »

This is a very interesting, accessible-to-laypeople talk about a new technology (injected ice slurry) for avoiding cell death after trauma.

Sean McCabe’s introduction is also very funny in this one. Sean, who was MC at SS09 and SS10, has been brought on board SIAI to help us organize and run our annual Summit. This is good for me because it means we get to share work.

One Response »

“It’s made up of everything from young hippies, to old hippies, to academia.”

11 Responses »

The link is here. Here’s the description:

We currently have the following openings, all based at our headquarters in Sunnyvale, CA:

Director of Communications – This senior leadership role will be responsible for the vision and execution of our movement-building program. (There is a $2,500 referral bonus for this position.)

Director of Development – This individual will be responsible for the fundraising that fuels the organization. (There is a $2,500 referral bonus for this position.)

Director of Operations – This individual will help translate our world-changing vision into solid, predictable results. (There is a $2,500 referral bonus for this position.)

Administrative Assistant – This individual will be responsible for making sure our team is as efficient and productive as possible. Work will involve office management, travel arrangements, and much more. This is a part-time contract position.

Legal Assistant – This individual will be responsible for making sure our legal team is as efficient and productive as possible. Duties include assisting with drafting and writing of legal research papers, proposals, and general legal research. This is a paid, part-time position at our office in Sunnyvale.

If you apply, be sure to say you saw it here!

Tweet

One Response »

Talking to Phil, Stephen, and PJ on FastForward Radio last night, I made a point that I make often in person but I don’t think I’ve ever said on my blog.

The point is a reaction to accusations of doomsaying. People say, “you’re so negative, contemplating catastrophic scenarios and apocalypse!” My response is that rather than being indicative of me being pessimistic or depressed, it is actually evidence that I am a happy person. Because I have a high happiness set point, I am enabled to consider negative scenarios without suffering personal depression or momentary sadness. I am immune from the reactive flinching away that most people have when they consider nuclear war or robots destroying all humans. Well, not entirely immune, but certainly more immune than most, and acclimation is part of it.

Because of my high happiness set point, there are greater volumes of idea space that I can comfortably navigate. Try it. Can you consider nuclear war in an entirely objective way, thinking about scientific facts and evidence, rather than fixating on the emotional human impact? For me and some of my friends, nuclear war can be brought up at a casual conversation, without gloominess, simply because it’s interesting to work through the probabilities involved. We can be sad and humanistic/emotional about it too, but we have the option to be analytical as well. Others don’t have a choice. More choices is good in this situation.

People with an average or low happiness set-point are unfortunately handicapped. They can’t think about negative possibilities without feeling sad. Thus, that portion of the memetic state space is blocked off to them. Poor schmucks.

Ironically, their inability to rationally confront existential risks increases the probability that we will all experience a disaster. Unfortunate, because their actions will cause others to suffer.

A corollary of this effect is that when existential risks are brought up at all, it tends to be in a humorous context, because most people are too fragile to consider it in a non-humorous context.

Tweet

12 Responses »

My friend, associate, and past SIAI Visiting Fellow Tom McCabe was quoted in the cover story of this month’s Forbes:

Yale math major Thomas McCabe, 19, is applying for a Thiel grant. McCabe hopes to commercialize low-cost 3-D printers that now make a range of plastic goods on demand. “We are living among the ruins of a fallen civilization,” he says, sounding a lot like Thiel must have 24 years ago. “Take all of the basic infrastructure, our roads and bridges and so on that we built in the 1950s and ’60s. If we tried to build them now we couldn’t do it.” But with a grubstake from Thiel we might get a little closer.

Fun to see ideas that begin as quirky conversations among SIAI employees and visiting fellows find their way into cover stories on Forbes!

The second paragraph of the article indirectly references SENS, Seasteading Institute, Singularity Institute, and Halcyon Molecular. In the last few years, I have worked or consulted for all these orgs except the Seasteading Institute.

It would be easy to write off Thiel as a “wackaloon,” as one political blogger has called him. Indeed, Thiel is putting serious money behind companies and groups bent on extending life, colonizing on ocean platforms, commercializing space, promoting so-called friendly artificial intelligence and leapfrogging DNA sequencing, among other causes. Freedom, he has said, is incompatible with democracy. In one of his most provocative acts, he has offered hundreds of thousands of dollars to college kids if they drop out of school and start a business or pursue a breakthrough. “People think of the future as something other people do,” Thiel says backstage at a December philanthropic fundraiser in San Francisco. “But there’s something weirder about a society where people don’t think about the future.”

That’s the society we live in. “Contemplating the future” consists of wondering what you’re going to do next weekend.

Tweet

One Response »

I will be on FastForward Radio tonight with Phil Bowermaster, Stephen Gordon, and PJ Manney, to talk about the Singularity and the possibility of dangers in the future:

Phil Bowermaster and Stephen Gordon welcome futurists Michael Anissimov and PJ Manney to discuss the importance of bleak scenarios for the future, even in the face of criticism that one is being “too negative.” Maybe being an optimist isn’t enough — maybe the important thing is what KIND of optimist you are.

I’ll be as negative as I think the evidence warrants… believe me, I’m not being negative for the fun of it or to get attention. I’d rather that I could ignore futurism at my leisure and focus on other topics, like climbing mountains. The problem is that I see an explicit danger and the need to address that danger until it is gone, even if it takes a century.

If you join the radio chatroom you can submit questions.

One Response »

Here the chapter in English, here’s the Chinese version.

No Responses »

H/t Living to 150.

I highly recommend Aubrey’s book… it has 4.5 stars on Amazon. The book is a quick read and includes lots of details on SENS that don’t appear elsewhere. It also includes the story of how Aubrey came up with SENS in a hotel room in California. If anything, parts of it (near the beginning) are stuff you may have heard of, so skipping 100 or so pages can make it easier going. It has a total of 400 pages, which is pretty good. Most people aren’t familiar with the details it contains, but they’re very interesting biologically because Aubrey, and his collaborator Michael Rae, really have an unusual take on aging that sets Aubrey and a few others apart from everyone else. They hypothesize many research avenues which may contain value which aren’t really being pursued, or are only being pursued by the SENS Foundation.

No Responses »

I saw this at the Daily Mail, which everyone should know is a very unreliable source, but it’s still a little concerning:

They said that the super-volcano underneath the Wyoming park has been rising at a record rate since 2004 – its floor has gone up three inches per year for the last three years alone, the fastest rate since records began in 1923.

But hampered by a lack of data they have stopped short of an all-out warning and they are unable to put a date on when the next disaster might take place.

When the eruption finally happens it will dwarf the effect of Iceland’s Eyjafjallajökull volcano, which erupted in April last year, causing travel chaos around the world.

The University of Utah’s Bob Smith, an expert in Yellowstone’s volcanism told National Geographic: ‘It’s an extraordinary uplift, because it covers such a large area and the rates are so high.

‘At the beginning we were concerned it could be leading up to an eruption.’

The prior probability of a catastrophic eruption per year is about 0.00014%. Alexsei remarked in the comments that the probability is actually higher because it rises the more time elapses since the last eruption, and the eruptions have been fairly periodic for the last two million years. I want to do a more precise calculation, but say that it increased the probability by a factor of ten, that would be 0.0014% chance per year, or a 0.14% chance this century.

Tweet

9 Responses »

Watch live video from TechZulu on Justin.tv

Patri Friedman finishes up his fascinating talk on lifehacking, followed by Michael Vassar on “Networks, Hierarchies and the Vingean Singularity”.

No Responses »