Accelerating Future

Cross-posted from SIAI blog:

Thanks to generous contributions by our donors, we are only $11,840 away from fulfilling our $100,000 goal for the 2010 Singularity Research Challenge. For every dollar you contribute to SIAI, another dollar is contributed by our matching donors, who have pledged to match all contributions made before February 28th up to $100,000. That means that this Sunday is your final chance to donate for maximum impact.

Funds from the challenge campaign will be used to support all SIAI activities: our core staff, the Singularity Summit, the Visiting Fellows program, and more. Donors can earmark their funds for specific grant proposals, many of which are targeted towards academic paper-writing, or just contribute to our general fund. The grants system makes it easier to bring new researchers into the fold on a part-time basis, widening the pool of thinkers producing quality work on Artificial Intelligence risks and other topics relevant to SIAI’s interests. It also provides transparency so our donor community can directly evaluate the impact of their contributions.

Human-level and smarter Artificial Intelligence will likely have huge impacts on humanity, but only a tiny number of researchers are working to understand how to ensure those impacts are good ones. The role of the Singularity Institute is to fill that void, bringing scholarship and science to bear on challenging questions. Instead of just letting the chips fall where they may, help the Singularity Institute increase the probability of a positive Singularity by contributing financially to our research effort. We depend completely on donors like you for all funding.

2010 marks the 10th year since SIAI’s founding. With your help, SIAI will still exist in 2015, 2020, 2025… however long it takes to get to a positive Singularity. Thank you for your support!

No Responses »

In this video, Dawkins acknowledges the possibilities of hard takeoff and open-ended recursive self-improvement in Artificial Intelligence.

24 Responses »

Kevin Warwick, though obviously is a Singularitarian, portrays the same adversarial stance against AI as other human chauvinists, such as James Hughes. I paraphrase it as: “If there’s an entity around that’s smarter and more powerful than me, then I’m going to equate that with me being subservient and freak the fuck out!”

My suggestion: calm down. Let’s do what we can to develop AIs that are nice people. There is no way we are going to outrace AI in the long run, so have to pursue this path, whether we like it or not. We are not going to eliminate all computers in the world, or keep power in the hands of humans forever. The question is not, “will the most powerful and capable entities in the world eventually be AIs?” (the answer is yes), the question is, “what the heck can we do to ensure our continued survival and prosperity once these entities inevitably become more capable than us?”

Sooner or later, positive experiences with AI programs or robots will cause these AI adversaries to understand that AIs could potentially become people too: worthy of our trust and love. The longer they keep up their adversarial attitude, the more time is wasted ignoring the challenge of engineering Friendly AIs. The year is 2010 and the clock is ticking.

15 Responses »

If you haven’t picked up this month’s GQ magazine, do it soon. There is a feature on the Singularity Summit and Singularity Institute. (I also hear there is a piece by Carl Zimmer on the Singularity in Playboy but I haven’t picked it up yet.) Seeing community names like Rick Schwall (an SIAI donor and supporter) in a national magazine sure is a trip. According to the National Magazine Awards, circulation is somewhere between 500,000 and 1,000,000 and is up in recent years.

Here is the Singularity portion (I removed the magazine cover due to copyright concerns and complaints from the comments section):

Really freaky, mmhmmm! Freaky like our ancestral past or Pandora freaky, I hope.

H/t to Gus K. for pointing out the article earlier this month.

7 Responses »

On Sunday, January 3rd, I did an interview on KUSP (Central Coast Public Radio) in Santa Cruz, California, a National Public Radio affiliate. I talked to Rick Kleffel for an hour about the Singularity, the Singularity Institute, what we do, anthropomorphism, Friendly AI, and the like. It was for his “Talk of the Bay” radio program. Here is the audio archive.

4 Responses »

There’s a guide to Singularity skepticism at Space Collective. It’s not bad — it goes over some of the main objections to the Singularity, mostly Kurzweil’s version thereof, but it touches on others. This page was particularly widely read this last week because it was linked from io9.

While the page is nice, I get the feeling that the author wrote about Singularity skepticism because the arguments and references just crystallized in his head fairly recently. For instance, he cites Charlie Stross’ really trite and crappy guide to the Singularity as accurate, but I think the reference might be partially tongue-in-cheek.

In general, I think the mood displayed by Cory Doctorow, io9 staff, and most of the Digg community is the same: “Oh, Singularity? That has to do with humans melding with machines, right? Boy, that gives me food for thought… *thinks while two seconds pass.* So how about that new TV show?” In this era of “infotainment”, any information not presented in an explicitly entertaining way is tuned out immediately. Since I know from years of observation that 95% of people that write comments on blogs and social websites about futurism think this way and never do any background reading, it’s worth realizing that their opinions are pretty much useless.

The writer of the Space Collective article actually goes above and beyond the usual effort to untangle Singularity skepticism, and as a result, I’d give the article a 6/10, especially for the richness of its links. The article routinely links longer pieces, a refreshing break from mostly thinking-free articles linked from sites like Cracked, Digg, and Reddit.

One Response »

Yesterday, Good posted the seventh and second-to-last installment of myself and Roko Mijic’s series on the Singularity, “Criticisms of the Singularity”. (My last contribution to the series, “The Benefits of a Successful Singularity”, was promoted to the front page of Digg.) For your benefit, the complete article is reproduced here.

Part seven in a GOOD miniseries on the singularity by Michael Anissimov and Roko Mijic. New posts every Monday from November 16 to January 23.

As was previously discussed in our series, the “singularity” means the creation of smarter-than-human intelligence, or “superintelligence,” a type of intelligence that is impressively more intelligent than humans. Possible methods for its creation include brain-computer interfaces and pure artificial intelligence, among others. Various scientists, futurists, and mathematicians that write about the singularity, such as Ray Kurzweil, Nick Bostrom, and Vernor Vinge, consider such an event plausible sometime between about 2025 and 2050. Among those who consider the singularity plausible, it is widely agreed that the event could alter the world, our civilization, and even our bodies and minds profoundly, through the technologies that superintelligence could create and deploy.

Because the singularity is such a new and speculative idea, and the subject of little academic study, there are people that take practically every imaginable position with respect to it. Some, unfamiliar and shocked by the idea, dismiss it outright or simply react with confusion. Others, such as philosopher Max More, dismiss some of the central propositions after more careful study. A substantial number embrace it openly and without too many qualifications, such as futurist Ray Kurzweil, who seems to expect a positive outcome with a very high probability. My organization, the Singularity Institute, and related thinkers such as philosopher Nick Bostrom, see a positive outcome as possible but not without very careful work towards ensuring that superintelligences retain human-friendly motivations as they grow in intelligence and power.

Criticisms of the singularity generally fall into two camps: feasibility critiques and desirability critiques. The most common feasibility critiques are what I call the Imago Dei objection and the Microsoft Windows objection. Imago Dei refers to Image of God, which is the doctrine that humans are created in God’s image. If humans are really created in the image of God, then we must be sacred beings, and the idea of artificially creating a superior being becomes dubious-sounding. If such a superior being could be possible, then wouldn’t God have created us that way to begin with? Unfortunately for this view, science, experimental psychology, and common sense have revealed that humans possess many intellectual shortcomings, and that some people have more of these shortcomings than others. Human intelligence isn’t perfect as it is; long-term improvements may become possible with new technologies.

The Microsoft Windows objection often surfaces when the topic of superintelligent artificial intelligence is brought up and goes something like this: “How can you be expecting superintelligent robots in this century when programmers can’t even create a decent operating system?” The simple answer is that too many cooks ruin a dish, and operating systems are plagued by a huge number of programmers without any coherent theory that they can really agree on. In other fields, such as optics, aerospace, and physics, scientists and engineers cooperate effectively on multi-million dollar projects because there are empirically supported theories that restrict many of the final product parameters. Artificial intelligence can reach the human level and beyond if it one day has such an organizing theory. At the present time, no such theory exists, though there are pieces that may fit into the puzzle.

Lastly, there are desirability critiques. I am very sympathetic to many of these. If we humans build a more intelligent species, might it replace us? It certainly could, and evolutionary and human history support this possibility strongly. Eventually creating superintelligence seems hard to avoid though. People want to be smarter, and to have smarter machines that do more work for us. Instead of trying to stave off the singularity forever, I think we ought to study it carefully and make purposeful moves in the right direction. If the first superintelligent beings can be constructed such that they retain their empathy for humanity, and wish to preserve that empathy in any future iterations of themselves, we could benefit massively. Poverty and even disease and aging could become things of the past. There is no cosmic force that compels more powerful beings to look down upon weaker beings—rather, this is an emotion that comes from being animals built by natural selection. In the context of much of natural selection it is evolutionarily advantageous to selectively oppress weaker beings, though some humans, such as vegans, have demonstrated that genuine altruism and compassion are possible.

In contrast to Darwinian beings, superintelligence could be engineered for empathy from the ground up. A singularity originating with enhanced human intelligences could select the most compassionate and selfless subjects for radical enhancement first. An advanced artificial intelligence could be built with a deep, stable sense of empathy and even lacking an observer-centered goal system. It would have no special desire to discard its empathy because it would lack the evolutionary programming that causes that desire to surface to begin with. The better you understand evolution and natural selection, the less likely you think it is for Darwinian dynamics to apply to superintelligence.

We should certainly hope that benevolent or human-friendly superintelligence is possible, or human extinction could be the result. Just look at what we’re already doing to the animal kingdom. Yet, by thinking about the issues in advance, we may figure out how to tip the odds in our favor. Human-posthuman synergy and cooperation could become possible.

Michael Anissimov is a futurist and evangelist for friendly artificial intelligence. He writes a Technorati Top 100 Science blog, Accelerating Future. Michael currently serves as Media Director for the Singularity Institute for Artificial Intelligence (SIAI) and is a co-organizer of the annual Singularity Summit.

4 Responses »

Over the past week and a half that the Singularity Research Challenge has been launched, we’ve received some nice support, including a post by Razib Khan at the Gene Expression blog and an explicit donation recommendation by Alan Darwst, author of Utilitarian-Essays.com and a well-regarded figure in the online utilitarian community. Here is a post by Alan on Felicia, the utilitarian community, that goes into why research charities like SIAI offer a very high return on philanthropic investment.

Don’t it just let be Alan and Razib — you too can make a blog post about the Singularity Research Challenge, right at this very moment!

No Responses »

In the context of our 2010 Singularity Research Challenge, Steven over at Black Belt Bayesian has a collection of “reasons to invest in reducing existential risk that you might not have considered before”.

No Responses »

Today is January 1st, 2010, the 10th anniversary of the online publishing of “The Singularitarian Principles” by Eliezer Yudkowsky. This document is a handy set of common sense advice for anyone who considers the possible creation of superintelligence a big deal in utilitarian terms (or otherwise). The work is divided into four “definitional principles”, which form of the central definition of the term “Singularitarian” (as it was defined at the time), and “descriptive principles”, which “aren’t strictly necessary to the definition, but which form de facto parts of the Singularitarian meme.”

The definitional principles are:

1. Singularity
2. Activism
3. Ultratechnology
4. Globalism

The descriptive principles are:

1. Apotheosis
2. Solidarity
3. Intelligence
4. Independence
5. Nonsuppression

The “Singularity” principle refers to believing in “some fundamental change in the rules” in the future. Looking back on this from the vantage point of 2010, I think the term “Singularity” as defined here (”defined many different ways”) is far too vague to be useful. Yudkowsky probably should have anticipated how diluted the meaning of the word would become and the need to define it more narrowly for it to be useful. Other than that, I think the rest of the document essentially makes sense, and even after ten years it is not obsolete by any means.

The activism principle says, “A Singularitarian is someone who believes that technologically creating a greater-than-human intelligence is desirable, and who works to that end.” Aside from #1, this is the most important principle in my eyes. Kurzweil’s definition of a Singularitarian is someone who “understands the Singularity” and has reflected on its consequences for their own life, but to my eyes this is excessively broad and meaningless. Am I an environmentalist just because I understand the environment and have reflected on its consequences for my life? No. Movements are defined by activism and change, not silent reflection alone. Kurzweil is making the definition of a Singularitarian vague so he can sell it to a wider audience, but I doubt that this definition is useful for actually inspiring humanity to have a positive impact on the Singularity, which it must do or perish.

The principle of “ultratechnology” is that “the “Singularity” is a natural, non-mystical, technologically triggered event”, and “What distinguishes the Singularitarians is that we want to bring about a natural event, working through ultratechnologies such as AI or nanotech, without relying on mystical means or morally valent effects.” What this basically says is that we are scientific materialists who refuse to believe in woo. Pretty basic. The few challengers that try to connect us with mysticism over the past decade have largely failed, in my opinion. A more useful avenue of approach has been to claim that the human brain is extremely complex, it must be copied exactly to produce intelligence, and Singularitarians are overconfident of the timescales on which this can be achieved. To help clarify the situation, we created the Uncertain Future modeling application. This application will allow us to make our assumptions explicit in debates, and force us to think about these assumptions carefully.

Globalism means that we want the benefits of a Singularity to extend to everybody. As far as I can tell, most Singularitarians of the Singularitarian Principles variety take this for granted. The benefits of the Singularity would be so large that it would take everyone to fully enjoy them. Programming an AI to benefit some small group rather than humanity in general would probably be a technical hassle in the long run anyway (though I could be wrong about that), and likely terribly unstable, not to mention idiotic and evil. Speaking for the Singularity Institute, we currently have strongly utilitarian and globalist bent. I worry more about our activism than our globalism.

For the descriptive principles, visit the original page, which I strongly recommend if you care to learn more about the Singularity and Singularitarians.

25 Responses »

Roko Mijic has written an article on the human benefits of the Singularity, “The Utopia Force” at Good.is. Here it is.

How intelligent machines could make being human unimaginably better.

Part six in a GOOD miniseries on the singularity by Michael Anissimov and Roko Mijic. New posts every Monday from November 16 to January 23.

Think of this note as if it were an invitation to a ball—a ball that will take place only if people show up. We call the lives we lead here “Utopia.”

– Nick Bostrom, Letter from Utopia.

Why should you care about the singularity when studies show that material possessions and technology beyond a certain point don’t actually make people any happier? Two weeks ago, I spoke about the possibility of giving a superintelligent AI the goal of doing whatever the human race would, after careful consideration, decide was best. This is known as the CEV algorithm. The outcome of this process would be very much unlike the technology, gadgets, and consumerism of today.

As Nick Bostrom has so eloquently reminded us, humanity’s biggest problems aren’t what we think they are: the most insidious and hard to notice, according to Bostrom, is that life is not nearly as good as it could be. This problem is really difficult for us to see; what could possibly be substantially better about our lives, even here in the developed world?

To start with, one has to realize that we’re not built for our own good. Evolution built us caring only about our ability to pass our genes on. We are easier to hurt than to pleasure, and we have been built with happiness set-points that are near impossible to significantly move away from without altering our biology. Studies have shown that giving a person $1,000,000 doesn’t actually make them happier in the long run, because of the hedonic treadmill effect: The human brain gets “used to” your circumstances, so that if your circumstances improve, your happiness goes up at first, but then returns to average. Our genes “calculate” that our bodies are worth keeping in good shape for 50 or so years, but after that, we are of little use, so our genes allow us to fall apart.

The society around us is also not built entirely for our benefit; it is a set of self-sustaining institutions that are, to a lesser or greater degree, influenced by the whims of a capricious electorate. Corporations can survive by hiring marketing departments to make us want things that we don‘t really need, and by hiring lobbying departments to make sure that the democratic process doesn‘t get in the way (see, for example, the tobacco industry). The challenges that work presents to us are often stifling and tedious; working in an office is not the natural human environment, which is why so many people ask for (and never get) a job that involves being outdoors.

Perhaps most importantly, the social dynamics that emerge from the interaction of many people who are each individually seeking status, power, and happiness often results in zero and negative-sum interactions. People are mean to each other, argue, fight, cheat, lie, and frequently make each others’ lives a misery—and this is ultimately a result of our evolved psychology, which was designed to deal with situations in which humans were forced by scarcity to kill each other in order to survive.

A world fashioned by the CEV algorithm would, at the very least, fix all of these fairly obvious flaws. Human psychology and biology could be altered to make us kinder, happier, healthier, and free from involuntary death or aging, and to remove the hedonic treadmill effect. New and better institutions could be developed from the ground up, and complex yet nourishing intellectual and physical challenges could be designed to replace what we today call “work.”

Iain Banks has described such a world in his science-fiction books about a future society called “The Culture”: enhanced humans live for thousands of years, and do exactly what they want with their time; they create art and science, they socialize, they enjoy a selection of customized virtual reality and real-world experiences and safe recreational drugs. They are all permanently young and attractive, with bodies and brains that have been altered in beneficial ways, they rarely argue with each other or have significant or prolonged negative interactions, and they have lots of sex.

If we consider all the possible ways that the universe could be arranged, and rank them in terms of how good they would be, Banks’ utopia certainly gets a very high rank. But it seems unlikely that it is the very best—or even close to the best. Banks’ utopia represents the limit of good experiences that we can currently think of and realize are good according to our complex values. Just how much better could it get?

In order to really make an accurate guess about the limits of goodness of the world, one must think about the problem indirectly or by analogy, because there are some states that are both so good and so complex that we cannot even imagine them yet. For example, what are the limits of goodness of subjective experience? What is the limit of the level and degree of mutual respect, friendship, passion or love that is possible?

At the risk of severely embarrassing myself forever over the internet, I’ll illustrate this with a personal example. Before I had ever kissed anyone, I didn’t actually know that the subjective experience of a passionate kiss was possible. I knew that kissing was possible, but not what it would feel like, or even that feelings that good were possible. Not only was I missing out, but I was unaware that I was missing out.

Humanity as a whole may have the same problem. We haven’t realized that a level of life that far surpasses what we currently experience might be possible. Imagine a human raised by animals who did not have the ability to speak, or the very real South American Indian tribe who are unable to learn, even when given food incentives, to count up to 10. Imagine a person who spent their entire life without experiencing romantic love, imagine a human who had neither hearing nor sight. These impoverished human beings are to us as we are to humans living in a post-positive singularity world: There are very probably intellectual, aesthetic, and emotional enhancements that would put their recipients as far beyond the citizens of The Culture as we are beyond a blind, solitary ignoramus.

A benevolent superintelligent AI would drastically and precisely alter the world, but do so in a direction that was dictated by your preferences. It would be like a new physical force that consistently pushed life towards our wisest utopian ideal. This ideal, or something very close to it, really is attainable. The laws of physics do not forbid it. It is attainable whether we feel that it is “unreasonable” that life could get that good, whether we shy away from it for fear of sounding religious, whether we want to close our eyes to the possibility because it scares us to believe that there is something greater out there, but we might let it slip through our fingers.

And indeed we might. As Carl Sagan puts it, “Our descendants, safely arrayed on many worlds throughout the solar system and beyond, will marvel at how vulnerable the repository of raw potential once was, how perilous our infancy”
.

Roko Mijic is a Cambridge University mathematics graduate, and has worked in ultra low-temperature engineering, pure mathematics, digital evolution and artificial intelligence. In his spare time he blogs about the future of the human race and the philosophical foundations of ethics and human values.

10 Responses »

I missed this one, from May: “The 5 Most Likely Ways Humans Will Become Obsolete”. See other articles on the Singularity that have reached the front page of Digg here. H/t to Roko for pointing me there.

2 Responses »