Accelerating Future

Michael Anissimov: “Don’t Fear the Singularity, but Be Careful: Friendly AI Design” at Foresight 2010 Conference from Foresight Institute on Vimeo.

Here’s my talk from Foresight! If you read this blog, there won’t be much new to you. I probably should have summarized the talk at the beginning. Unfortunately I got cut off at around slide 40 out of 55 due to schedule problems, so I missed the opportunity to summarize some of SIAI’s recent work and ended up mainly talking about 1) generic progress in AI, 2) media coverage of AI and Singularity, 3) the intelligence explosion idea, 4) the AI advantage, and 5) the inherent unconnectedness of morality and intelligence (Hume). Ignore the title; I didn’t really get into Friendly AI design at all. It was more of an introduction to why Friendly AI may be required. (I’m not sure I would have even used the term “Friendly AI” if I were making up the talk title again, because it’s been argued by a number of people that the term sounds silly and unserious.)

If I could redo this talk (I plan to do so on video) I would focus a little more on ideas and less on AI advancements, and throw out all the quotes, just quickly summarizing them instead. I would also try even harder to avoid looking down at my laptop during the talk, and would have removed my nametag. I need to buy one of those remote clicker things. I realize I spent a fair amount of time summarizing other people’s AI research rather than ideas unique to me or SIAI, but at the time it seemed necessary because I assumed that few people in the audience would be familiar with the range of advances in AI over the last year alone. People have to understand that AI is making steady progress, otherwise why worry about more advanced AI? If I thought AI really were stuck in the mud, then I wouldn’t be as frantic about the need for safe AI.

Several people pointed out to me that the talk title also seems odd because I am all about getting people to “fear the Singularity” — or fear a negative Singularity where humanity gets steamrolled by indifferent superintelligence. My idea here was that we don’t have to fear the Singularity if we’re careful. I often get the impression that people’s minds just shut down when considering the prospect of an AI Singularity, even if they don’t object to the plausibility of human-level AI in principle, just because they see it as extremely alien in comparison to a human-sparked Singularity. Part of the idea I was going for was that an AI-sparked Singularity can be managed effectively, but as I mentioned, I didn’t even get around to talking about that.

Thinking about my comment on the superficial mundaneity of analyzing the genetic expression of baker’s yeast, I realize that it may not be considered that mundane to some scientists, but I’m not sure because I’m not a biologist that researches microbial genetic expression. I just figured that since yeast is a model organism, we already know a fair amount about its patterns of genetic expression and that the experiments were mainly for show.

You can follow along with the talk with my slides here.

5 Responses »

Contrary to common wisdom among intellectuals, television is not completely useless — it can help us learn about the EEA, that humorously elusive concept. Take some of the program with a grain of salt. After all, it is a dramaticization.

Fun fact: Homo sapiens spent about 70% of its evolutionary history in small bands in the plain/forest mixed environment of sub-Saharan Africa. Extensive water exposure was probably involved as well. The EEA is a complex thing.

The more you understand the EEA, the more non-surprising all types of human behavior are today.

Bonus topic: how did Jesus of Nazareth artfully play to certain human evolutionary desires while downplaying others? Is the Church that Jesus preached about fundamentally a social project or a personal one?

Extra bonus: If Australopithecus existed today, would we justify their enslavement by calling them subhuman, like we do to Sus scrofa and Bos primigenius today?

12 Responses »

In this video, Dawkins acknowledges the possibilities of hard takeoff and open-ended recursive self-improvement in Artificial Intelligence.

24 Responses »

g-speak overview 1828121108 from john underkoffler on Vimeo.

Here is the NYT article.

No Responses »

Foresight 2010 debate: Futarchy from Monica Anderson on Vimeo.

H/t Foresight Institute.

One Response »

One Response »

6 Responses »

H/t to Michael Graham Richard for the link.

No Responses »

No Responses »

H/t to Joshua Fox for the link.

I have been watching some of Tim’s videos over the last few months, but I definitely haven’t seen them all. This one is nice because it summarizes a poignant feeling of concern.

In this video, he builds a model of AGI development using construction paper and Post-It notes.

4 Responses »

Here is the commentary. Most of all, I enjoy reviews and comments by outsiders with no contact with our current community. Here are a few quotes and my comments:

It is video conference phone call split screen debate between this Yudkowsky guy who is the head scientist at the Singularity Institute, and Lanier who has been the genius hippy in red dread locks since his early pioneering work with Virtual Reality and artificial vision systems.

Before you click the link, let me frame the debate.

These two guys represent the two extremes of a subtle range of viewpoints on evolution, AI, and human consciousness.

An interesting and subtle range that deserves more popular and academic attention and will get it sooner or later because we are building technologies that produce divisive responses to the relevant philosophical issues.

Jaron’s main criticism of the hard AI camp in this debate is that their strong attachment to finding a way past death and their apriori beliefe in the posibility of resonably building self evolving intelegence together become so rhetorically invasive that they can no longer do objective investigation or engineering… that their beliefs and desires make them “religious”.

Well, Jaron would probably prefer if we didn’t do any objective investigation or engineering, but that’s not true. Remember, as cybernetic totalists, we are totally devoted to our goal. Totally awesome!

From my perspective, Jaron is a nothing more than a (very bright) priest who can’t stop doing science in the basement, and Yudkoswsky is nothing less than a scientist that can’t help wanting to build a God.

Hah! A superintelligence would be like a god. I can vaguely understand why people who don’t regard MNT as plausible would disagree with this, but I never understand why those who do believe that MNT is plausible would.

The fireworks in the video begin at 11:00! I actually agree with many of Jaron’s points in the abstract. I disagree with him when he says that we cannot represent some physical systems in totality or simulate them precisely.

10 Responses »

Anders Sandberg directs us to two new lectures by Oxford philosophers.

“Global Catastrophic Risks” by Nick Bostrom
“Human Enhancement: Bioliberation or Biothreat?” by Julian Savulescu

Scroll down a bit to see the controls if you don’t see them at first. The custom flash interface has some cool features, like simultaneously showing the slides and speaker. You can even click a button near the bottom to expand the slides or the speaker window.

In his talk, Savulescu mentions the cognitive enhancement value of iodine in salt. He says that about a billion IQ points are lost each year due to iodine deficiency. If you’re a pregnant woman and you don’t get iodine in your salt during pregnancy, your child loses about 10-15 IQ points. It would cost 2 cents per person per year to iodize salt. 4 billion people lack adequate iodine.

No Responses »