Life, the Universe, and Everything

Suppose that a very reliable source told you that, somewhere in Turkmenistan, there was a world-destroying bomb set to go off in one month. The only way to disable the bomb is by cutting the red wire. Clearly, destroying the world is bad, and so the next day, you fly out to Turkmenistan, locate the bomb, and snip the red wire. The bomb then blows up, taking you and the world with it, as you needed to cut the blue wire before you cut the red wire.

This scenario results an obvious negative payoff from new information, both relative to doing nothing and relative to omniscience. The new information didn’t make anyone’s model of the world worse; a model of the world with a bomb and one wire is clearly superior to a model with no bomb and no wires, relative to an actual world with a bomb and two wires. It appears that the more inaccurate model leads to a better outcome. Can you rig a scenario where you know (in advance) that improving your world-model will result in a bad outcome, and that you should leave it unchanged?

In population ethics, a linear summing over utilities leads you to the Repugnant Conclusion: if you get 3^^^^3 people together, and have them all experience the smallest possible amount of joy, the expected positive utility will exceed that of our entire civilization. This is the negation of Pascal’s Mugging, although it has wider implications: if true, it means that we should fill the universe with cheaply reproducible happiness, even if we have to eliminate most of the happiness-related mind states that humans currently experience. Peter de Blanc has proven that, if you have an unbounded utility function, you will always wind up in situations like these, and so the problem must be with our estimates of our utility functions and not our ideas about happiness.

However, given the current state of the world, we should still consider the Repugnant Hypothesis: What if the sum of utility over most human lives is negative, rather than positive? In the real world, even if this is true, it shouldn’t matter that much; we still have hope. But it could have raised some awkward questions if we were rational enough to consider it in, eg., AD 1200. Should we no longer have children? Should we try and kill as many people as possible, except for those who are known to be happy? How does the utility created by destroying the planet balance against the potential future utility, if life in the future turned out to be better?

A quick thought experiment: Suppose that, in 1945, you had all of the high-ranking Nazi war criminals in a Star Trek-style holodeck. The obvious course of action is to shoot them, or torture them in various ways, but this won’t do anyone any good; your actions are never revealed to the public, and so you won’t deter future dictators or prevent another Holocaust. The only causal result of one additional unit of pain is one additional unit of pain. Yet, even though a lot of people accept the principle that pain is always bad and pleasure is always good, I suspect that many of them would be uncomfortable with letting the Nazis live out the remainder of their lives in the most pleasant way possible. What’s the right thing to do, in this situation?

Singularity Summit 2008: Opportunity, Risk, Leadership takes place October 25 at the intimate Montgomery Theater in San Jose, CA, the Singularity Institute for Artificial Intelligence announced today. Now in its third year, the Singularity Summit gathers the smartest people around to explore the biggest idea of our time: the Singularity.

Keynotes will include Ray Kurzweil, updating his predictions in The Singularity is Near, and Intel CTO Justin Rattner, who will examine the Singularity’s plausibility. At the Intel Developer Forum on August 21, 2008, he explained why he thinks the gap between humans and machines will close by 2050. “Rather than look back, we’re going to look forward 40 years,” said Rattner. “It’s in that future where many people think that machine intelligence will surpass human intelligence.”

“The acceleration of technological progress has been the central feature of this century,” said computer scientist Dr. Vernor Vinge in a seminal paper in 1993. “We are on the edge of change comparable to the rise of human life on Earth. The precise cause of this change is the imminent creation by technology of entities with greater than human intelligence.”

Singularity Summit 2008 will feature an impressive lineup:

* Dr. Ruzena Bajcsy, pioneering AI and robotics researcher
* Dr. Eric Baum, AI researcher, author of What is Thought?
* Marshall Brain, founder of HowStuffWorks.com, author of Robotic Nation
* Dr. Cynthia Breazeal, robotics professor at MIT, creator of Kismet
* Dr. Peter Diamandis, chair and CEO of X PRIZE Foundation
* Esther Dyson, entrepreneur, investor, philanthropist
* Dr. Pete Estep, chair and CSO of Innerspace Foundation
* Dr. Neil Gershenfeld, director of MIT Center for Bits and Atoms, author of Fab
* Dr. Ben Goertzel, CEO of Novamente, director of research at SIAI
* John Horgan, science journalist, author of The Undiscovered Mind
* Ray Kurzweil, CEO of Kurzweil Technologies, author of The Singularity is Near
* Dr. James Miller, author of forthcoming book on Singularity economics
* Dr. Marvin Minsky, one of AI’s founding fathers, author of The Emotion Machine
* Dr. Dharmendra Modha, cognitive computing lead at IBM Almaden Research Center
* Bob Pisani, news correspondent for financial news network CNBC
* Justin Rattner, VP and CTO of Intel Corporation
* Nova Spivack, CEO of Radar Networks, creator of Twine semantic-web application
* Peter Thiel, president of Clarium, managing partner of Founders Fund
* Dr. Vernor Vinge, author of original paper on the technological Singularity
* Eliezer Yudkowsky, research fellow at SIAI, author of Creating Friendly AI
* Glenn Zorpette, executive editor of IEEE Spectrum

Registration details are available at
http://www.singularitysummit.com/registration/.

About the Singularity Summit

Each year, the Singularity Summit attracts a unique audience to the Bay Area, with visionaries from business, science, technology, philanthropy, the arts, and more. Participants learn where humanity is headed, meet the people leading the way, and leave inspired to create a better world. “The Singularity Summit is the premier conference on the Singularity,” Kurzweil said. “As we get closer to the Singularity, each year’s conference is better than the last.”

The Summit was founded in 2006 by long-term philanthropy executive Tyler Emerson, inventor Ray Kurzweil, and investor Peter Thiel. Its purpose is to bring together and build a visionary community to further dialogue and action on complex, long-term issues that may transform the world. Its host organization is Singularity Institute for Artificial Intelligence, a 501(c)(3) nonprofit organization studying the benefits and risks of advanced artificial intelligence systems.

Singularity Summit 2008 partners include Clarium Capital, Cartmell Holdings, Twine, Powerset, United Therapeutics, KurzweilAI.net, IEEE Spectrum, DFJ, X PRIZE Foundation, Long Now Foundation, Foresight Nanotech Institute, Novamente, SciVestor, Robotics Trends, and MINE.

Contacts

Singularity Summit
Tyler Emerson, 650-353-6063
Curator
emerson@singularitysummit.com

My stock market modeling program has been quite successful over the past several days. Here’s the data from a simulation of 2002:

See here for a higher-resolution version.

A general rule of investing: Most ways to reliably lose money can be converted into ways to reliably gain money. In most financial transactions, wealth isn’t created or destroyed, it’s simply transferred from one person to another; the game is zero-sum. Hence, if money is predictably moving away from one person, it must be predictably moving towards another person (or group of people).

As a simple example, consider advance fee fraud (the “Nigerian” or “419″ scam). Most Internet users have realized that falling for these types of scams is an easy way to lose money- the scammers will just keep taking your cash, and the supposed payoff never comes. However, it is possible to make money off of 419 scams, as the money involved isn’t destroyed, it’s simply transferred. Some Internet “entrepreneurs” have already succeeded in doing so.

Extending this idea to the equities markets, note that if you can reliably predict when a stock, bond, etc. will lose value, you can usually make money off of it by selling short or betting against it. Hence, in a perfectly efficient market, as long as you have no non-public information, it’s just as impossible to lose money reliably as to gain money reliably, as none of the information you have will correlate with prices.

The principle behind most CAPTCHAs is the pattern recognition capability of the human brain; we can see patterns and separate them instinctively, even when contemporary software has a hard time doing so. A modern computer, for instance, would have a hard time reading this text, but humans can do so easily, by recognizing the pattern of horizontal lines and separating them from the letters.

As it turns out, this makes deleting written information quite difficult; even if you cross something out, scribble over it, and strike out every word, it’s usually still legible. The simplest solution to this is to write over the text with a random string of letters, preferably several times; since earlier letters are indistinguishable from later letters, this makes reading the original text impossible very quickly. A five-letter word overwritten three times, for instance, would have 5^4 = 625 possible values.

Suppose that you got a job as a Wall Street financier, worked eighty-hour weeks for ten years to make a lot of money, and finally retired with a huge pile of cash. You wanted to spend the money you’d worked so hard for, and so you went out and bought a large house in a nice neighborhood. You moved in, bought furniture, and lived there for a while, but due to a mistake in the electrical wiring, the entire house and everything in it burned to the ground. What do you do next?

If you haven’t studied rationality, you’ll immediately begin to make up reasons for why your house burning down was, not only a good thing, but necessary for you to live a happy life. You can now sue the electric company and make a ton of money. You can collect on your expensive insurance policy and make a ton of money. You can move somewhere else and meet new people. You can travel the world and see how the other half lives. It doesn’t matter what the specifics are- the human brain is amazingly good at inventing reasons for why bad things happen.

All else being equal, most people would prefer to live in a world where bad things never happen at all. Such a world is blatantly imaginary, so we like to imagine that the world is unbalanced, but fair; X amount of bad can happen, so long as 2X amount of good happens later to make up for it. Living in a world governed by the laws of physics- where X amount of bad can happen this year, and Y amount of good can happen next year, and both are largely random and unrelated unless you keep track of quarks- seems to scare people.

This type of rationalization seems to occur fairly frequently, even among people who have no explicit supernatural beliefs. The primary reason for this seems to be the trickiness of causality; when we say “A caused B”, we usually mean “~A -> ~B”, and there are almost always ten gazillion values for A that can be inserted to fit this requirement. I don’t know how common this is among, say, MIT graduates, but it’s certainly common among MIT rejects.

I haven’t found any explicit guide for how to dissolve this class of mistakes. For most irrationalities, you can make at least some progress by realizing that you’re being irrational, seeing a few examples where irrationality leads to bad outcomes, and thinking of new ways to study the world. But this type seems to be difficult to handle psychologically. To quote Eliezer:

“What would it be like to be a rational atheist in the fifteenth century, and know beyond all hope of rescue that everyone you loved would be annihilated, one after another as you watched, unless you yourself died first? (…) I wonder if there was ever an atheist who accepted the full horror, making no excuses, offering no consolations, who did not also hope for some future dawn. What must it be like to live in this world, seeing it just the way it is, and think that it will never change, never get any better?”

The Singularity Summit 2008 has been officially scheduled for October 25th, 2008, at the Montgomery Theater in San Jose. Most details have not yet been finalized; an official website for the Summit should be forthcoming within the next several weeks. Please contact Tyler Emerson for further details, or if you are interested in volunteering. A Google Group has been set up for Summit planning (currently private and invitation-only).

Endgame: Singularity is the title of a video game developed by EMH Software, where the player leads a fledgling artificial intelligence through a soft takeoff. The game isn’t, of course, a realistic model of the future; for one thing, a real upload would be able to spend hours thinking through problems, and a real AI wouldn’t even have the same cognitive architecture as a human player. It is, however, a reasonably successful attempt to popularize SL3 ideas; the game is now bundled with Debian, Ubuntu, and several other freeware distributions. So far as I know, the author isn’t involved in the transhumanist community, which is quite a shame.

A properly designed game could, if used well, greatly enhance transhumanist PR, and development is probably feasible. There are numerous, talented computer programmers in the WTA, and I’m confident that some of them will be happy to lend their assistance. Note that video games cannot be used to transmit technical knowledge; transmission of new concepts and overcoming future shock is probably the best we can hope for.