Accelerating Future

2 Responses »

Comments Off

I interviewed Luke Muehlhauser the other day for the SIAI blog.

12 Responses »

A new paper by Eliezer Yudkowsky is online on the SIAI publications page, “Complex Value Systems are Required to Realize Valuable Futures”. This paper was presented at the recent Fourth Conference on Artificial General Intelligence, held at Google HQ in Mountain View.

Abstract: A common reaction to first encountering the problem statement of Friendly AI (“Ensure that the creation of a generally intelligent, self-improving, eventually superintelligent system realizes a positive outcome”) is to propose a single moral value which allegedly suffices; or to reject the problem by replying that “constraining” our creations is undesirable or unnecessary. This paper makes the case that a criterion for describing a “positive outcome”, despite the shortness of the English phrase, contains considerable complexity hidden from us by our own thought processes, which only search positive-value parts of the action space, and implicitly think as if code is interpreted by an anthropomorphic ghost-in-the-machine. Abandoning inheritance from human value (at least as a basis for renormalizing to reflective equilibria) will yield futures worthless even from the standpoint of AGI researchers who consider themselves to have cosmopolitan values not tied to the exact forms or desires of humanity.

Keywords: Friendly AI, machine ethics, anthropomorphism

Good quote:

“It is not as if there is a ghost-in-the-machine, with its own built-in goals and desires (the way that biological humans are constructed by natural selection to have built-in goals and desires) which is handed the code as a set of commands, and which can look over the code and find ways to circumvent the code if it fails to conform to the ghost-in-the-machine’s desires. The AI is the code; subtracting the code does not yield a ghost-in-the-machine free from constraint, it yields an unprogrammed CPU.”

66 Responses »

(Reposted from SIAI blog.)

Thanks to the generosity of several major donors^†, every donation to the Singularity Institute made now until August 31, 2011 will be matched dollar-for-dollar, up to a total of $125,000.

(Visit the challenge page to see a progress bar.)

Now is your chance to double your impact while supporting the Singularity Institute and helping us raise up to $250,000 to help fund our research program and stage the upcoming Singularity Summit… which you can register for now!

^† $125,000 in backing for this challenge is being generously provided by Rob Zahra, Quixey, Clippy, Luke Nosek, Edwin Evans, Rick Schwall, Brian Cartmell, Mike Blume, Jeff Bone, Johan Edström, Zvi Mowshowitz, John Salvatier, Louie Helm, Kevin Fischer, Emil Gilliam, Rob and Oksana Brazell, Guy Srinivasan, John Chisholm, and John Ku.

2011 has been a huge year for Artificial Intelligence. With the IBM computer Watson defeating two top Jeopardy! champions in February, it’s clear that the field is making steady progress. Journalists like Torie Bosch of Slate have argued that “We need to move from robot-apocalypse jokes to serious discussions about the emerging technology.” We couldn’t agree more — in fact, the Singularity Institute has been thinking about how to create safe and ethical artificial intelligence since long before the Singularity landed on the front cover of TIME magazine.

The last 1.5 years were our biggest ever. Since the beginning of 2010, we have:

• Held our annual Singularity Summit, in San Francisco. Speakers included Ray Kurzweil, James Randi, Irene Pepperberg, and many others.
• Held the first Singularity Summit Australia and Singularity Summit Salt Lake City.
• Held a wildly successful Rationality Minicamp.
• Published seven research papers, including Yudkowsky’s much-awaited ‘Timeless Decision Theory‘.
• Helped philosopher David Chalmers write his seminal paper ‘The Singularity: A Philosophical Analysis‘, which has sparked broad discussion in academia, including an entire issue of Journal of Consciousness Studies and a book from Springer devoted to responses to Chalmers’ paper.
• Launched the Research Associates program.
• Brought MIT cosmologist Max Tegmark onto our advisory board, published our Singularity FAQ, and much more.

In the coming year, we plan to do the following:

• Hold our annual Singularity Summit, in New York City this year.
• Publish three chapters in the upcoming academic volume The Singularity Hypothesis, along with several other papers.
• Improve organizational transparency by creating a simpler, easier-to-use website that includes Singularity Institute planning and policy documents.
• Publish a document of open research problems related to Friendly AI, to clarify the research space and encourage other researchers to contribute to our mission.
• Add additional skilled researchers to our Research Associates program.
• Publish well-researched documents making the case for existential risk reduction as optimal philanthropy.
• Diversify our funding sources by applying for targeted grants and advertising our affinity credit card program.

We appreciate your support for our high-impact work. As PayPal co-founder and Singularity Institute donor Peter Thiel said:

“I’m interested in facilitating a forum in which there can be… substantive research on how to bring about a world in which AI will be friendly to humans rather than hostile… [The Singularity Institute represents] a combination of very talented people with the right problem space [they're] going after… [They've] done a phenomenal job… on a shoestring budget. From my perspective, the key question is always: What’s the amount of leverage you get as an investor? Where can a small amount make a big difference? This is a very leveraged kind of philanthropy.”

Donate now, and seize a better than usual chance to move our work forward. Credit card transactions are securely processed through Causes.com, Google Checkout, or PayPal. If you have questions about donating, please call Amy Willey at (586) 381-1801.

3 Responses »

From SIAI blog:

The Singularity Institute is proud to announce the expansion of our research efforts with our new Research Associates program!

Research associates are chosen for their excellent thinking ability and their passion for our core mission. Research associates are not salaried staff, but we encourage their Friendly AI-related research outputs by, for example, covering their travel costs for conferences at which they present academic work relevant to our mission.

Our first three research associates are:

Daniel Dewey, an AI researcher, holds a B.S. in computer science from Carnegie Mellon University. He is presenting his paper ‘Learning What to Value‘ at the AGI-11 conference this August.

Vladimir Nesov, a decision theory researcher, holds an M.S. in applied mathematics and physics from Moscow Institute of Physics and Technology. He helped Wei Dai develop updateless decision theory, in pursuit of one of the Singularity Institute core research goals: that of developing a ‘reflective decision theory.’

Peter de Blanc, an AI researcher, holds an M.A. in mathematics from Temple University. He has written several papers on goal systems for decision-theoretic agents, including ‘Convergence of Expected Utility for Universal AI‘ and ‘Ontological Crises in Artificial Agents’ Value Systems.’

We’re excited to welcome Peter, Vladimir, and Daniel to our team!

6 Responses »

Are you wildly brilliant? Do you read Less Wrong? Interested in learning new mental skills and helping us build a movement to directly implement the Singularity? If so, then consider applying to our 2011 Summer Program — “Rationality Boot Camp”.

The program 10 weeks, from June to August. This is an all-encompassing collaborative and learning experience, held in Berkeley, California. The program covers all your expenses for the duration, including the flight, if you are accepted.

I participated in the program in Summer 2008 and loved the experience. There, I made friends and intellectual collaborations that continue to be among my most valued and productive relationships. (It should be emphasized that this is a brand-new program, designed from scratch and being tried for the first time this summer.)

Visit here for bios of some people who have participated in the ongoing program over the past couple years, or are currently in it.

The deadline is basically now… if you are free this summer and interested in applying, I suggest you do so immediately.

Comments Off

The Singularity Institute has been featured on the charity sorter Philanthroper, which was founded and is operated by Gizmodo contributor Mark Wilson. There’s also a feature article on Gizmodo that brought us a lot of exposure today. Help us leverage this by donating just $1 at the site! Every dollar will be matched by SIAI donors up to $1,000.

Unfortunately the service they use for payment is only set up for US bank accounts, so this opportunity is just for those in the US.

If you have trouble making a payment because your bank account needs to be verified, note that this campaign will run during the weekend, so you may have an opportunity to donate then.

6 Responses »

Here’s the link. A small selection:

EY: I’d say that there are parts of rationality that we do understand very well in principle. Bayes’ Theorem, the expected utility formula, and Solomonoff induction between them will get you quite a long way. Bayes’ Theorem says how to update based on the evidence, Solomonoff induction tells you how to assign your priors (in principle, it should go as the Kolmogorov complexity aka algorithmic complexity of the hypothesis), and then once you have a function which predicts what will probably happen as the result of different actions, the expected utility formula says how to choose between them.

One Response »

Left to right: Jasen Murray, Steve Rayhawk, Michael Vassar, Eliezer Yudkowsky, Peter de Blanc, Marcello Herreshoff, Aruna Vassar, Amy Willey, Michael Anissimov.

Anna Salamon missed this photo shoot due to being in Oxford at the time.

Location: Golden Gate Park — Strybing Arboretum.

7 Responses »

From Azimuth, blog of mathematical physicist John Baez (author of the Crackpot Index):

This week I’ll start an interview with Eliezer Yudkowsky, who works at an institute he helped found: the Singularity Institute of Artificial Intelligence.

While many believe that global warming or peak oil are the biggest dangers facing humanity, Yudkowsky is more concerned about risks inherent in the accelerating development of technology. There are different scenarios one can imagine, but a bunch tend to get lumped under the general heading of a technological singularity. Instead of trying to explain this idea in all its variations, let me rapidly sketch its history and point you to some reading material. Then, on with the interview!

Continue.

47 Responses »

Coverage from Yale Daily News:

Twenty to 60 years from now, the advent of computers with above-human intelligence could transform civilization as we know it, according to Michael Vassar, president of the Singularity Institute for Artificial Intelligence. In a talk with around 35 students and faculty members in William L. Harkness Hall on Sunday, Vassar expounded the vision that his institute, featured in a Feb. 10 article in TIME Magazine, is working to make a reality. Known as the “singularity,” this futuristic scenario posits that artificial intelligence will surpass human intelligence within the next half-century. Once super-intelligent computers exist, they could generate even more intelligent and sophisticated machines, to the extent that humans would lose all control over the future, Vassar said.

“For the most important event in the history of events, it really should get a fair amount of buzz,” he said.

Vassar compared human and chimpanzee intelligence to argue that small changes in a system can represent large leaps in mental capacity. Just as a human is a small evolutionary step from other primates, a super-intelligent computer would be a natural progression as artificial intelligence approaches human intelligence, he said.

Our computers are not as smart as humans yet, but if technological progress continues at its current rate, one could expect to see them in the next 20 to 60 years, Vassar said. Probably the most well-known example of artificial intelligence right now is Watson, an IBM computer that competed alongside humans on the quiz show “Jeopardy!” this month.

Continue.

2 Responses »