Michael Anissimov: “Don’t Fear the Singularity, but Be Careful: Friendly AI Design” at Foresight 2010 Conference from Foresight Institute on Vimeo.

Here’s my talk from Foresight! If you read this blog, there won’t be much new to you. I probably should have summarized the talk at the beginning. Unfortunately I got cut off at around slide 40 out of 55 due to schedule problems, so I missed the opportunity to summarize some of SIAI’s recent work and ended up mainly talking about 1) generic progress in AI, 2) media coverage of AI and Singularity, 3) the intelligence explosion idea, 4) the AI advantage, and 5) the inherent unconnectedness of morality and intelligence (Hume). Ignore the title; I didn’t really get into Friendly AI design at all. It was more of an introduction to why Friendly AI may be required. (I’m not sure I would have even used the term “Friendly AI” if I were making up the talk title again, because it’s been argued by a number of people that the term sounds silly and unserious.)

If I could redo this talk (I plan to do so on video) I would focus a little more on ideas and less on AI advancements, and throw out all the quotes, just quickly summarizing them instead. I would also try even harder to avoid looking down at my laptop during the talk, and would have removed my nametag. I need to buy one of those remote clicker things. I realize I spent a fair amount of time summarizing other people’s AI research rather than ideas unique to me or SIAI, but at the time it seemed necessary because I assumed that few people in the audience would be familiar with the range of advances in AI over the last year alone. People have to understand that AI is making steady progress, otherwise why worry about more advanced AI? If I thought AI really were stuck in the mud, then I wouldn’t be as frantic about the need for safe AI.

Several people pointed out to me that the talk title also seems odd because I am all about getting people to “fear the Singularity” — or fear a negative Singularity where humanity gets steamrolled by indifferent superintelligence. My idea here was that we don’t have to fear the Singularity if we’re careful. I often get the impression that people’s minds just shut down when considering the prospect of an AI Singularity, even if they don’t object to the plausibility of human-level AI in principle, just because they see it as extremely alien in comparison to a human-sparked Singularity. Part of the idea I was going for was that an AI-sparked Singularity can be managed effectively, but as I mentioned, I didn’t even get around to talking about that.

Thinking about my comment on the superficial mundaneity of analyzing the genetic expression of baker’s yeast, I realize that it may not be considered that mundane to some scientists, but I’m not sure because I’m not a biologist that researches microbial genetic expression. I just figured that since yeast is a model organism, we already know a fair amount about its patterns of genetic expression and that the experiments were mainly for show.

You can follow along with the talk with my slides here.