Media Director for SIAI Sunday, May 3 2009
me 7:16 pm
In a development that will surprise no one, I’ve been named Media Director of the Singularity Institute for Artificial Intelligence. I’m taking this as a half-time position for now, so I can keep up my independent writing and continue fundraising for the Lifeboat Foundation. My job is to communicate the activities of the Singularity Institute to the public, so if you have a question — ask me!
The Lifeboat Foundation’s plan for dealing with the risk of AGI is to support the Singularity Institute’s Friendly AI plan (something I’m starting to call “artificial morality” because it’s not a “term of art” that way). However, the Lifeboat Foundation is also focused on other risks that may be important. Though I consider the majority of the extinction risk probability mass to be concentrated around UFAI (unFriendly AI), other risks are important too, including synthetic biology, chemtech, MNT, and the ever-present “unknown” category. Hence my focus on all of these areas.
Basically, the unifying theme here is to prevent humanity from capping itself in the head before we get to The Future.
Thanks to Michael Vassar for hiring me, and to Eliezer Yudkowsky for being a continuing inspiration.
May 3rd, 2009 at 8:34 pm
As someone self-fed on sci-fi and self-raised as transhumanist since my early teens, I feel the disconnect to the general populace including family and friends only growing day by day. I cannot relate to their way of thinking because I see none of importance. Their goals and aspirations are H0 or H-, not H+. Depending on the level of your own estrangement from the general public’s interests (aka waste of time), you (and our kind in general) may not be the best communicators as far as reaching and influencing the general public is concerned.
But all you really need to influence are the ones who make the decisions and create the technologies. The regular people have little say or influence in things concerning themselves, today or in their future - you take what you’re given.
It may be that only a small percentage of humanity will ever be reached - or perhaps it will go exponentially mainstream in a matter of years or a decade or two. The internet has changed the dynamics of communication.
You might consider getting a second opinion from someone less steeped in H+ ideas and ideals.
Your job, ultimately, is to get SIAI and transhumanism to go viral.
May 4th, 2009 at 12:37 am
Congratulations, Michael! You’re exactly the right person to get SIAI’s message out.
May 4th, 2009 at 12:48 am
Congratulations!
Since you’re in the mood for taking questions, I’ll give it a shot. What theoretical and practical progress has SIAI made in the science/art of AGI and Friendly AI since publication of “Coherent Extrapolated Volition” 5 years ago? Thanks.
May 4th, 2009 at 12:50 am
I think the best way the best way to mitigate the risk of an unfriendly AI with greater than human intelligence is to upgrade ourselves first.
May 4th, 2009 at 10:46 am
Congratulations Michael on your new role. I like also your shift toward the term “artificial morality” but would suggest it’d be better described as “amplified” than “artificial.” Of course even that is not a very good label for a project which must eventually subsume our present moral faculties, to a greater extent than culture relative to instinct.
May 4th, 2009 at 2:00 pm
Congrats!
May 4th, 2009 at 2:36 pm
Congrats. So here’s a question. Has SingInst actually done any AI programming? Does it plan to in the near future?
May 4th, 2009 at 2:42 pm
Well, the 2nd phase of their research program is only 2 years away and it’s description is fairly unambiguous:
“If viable, design and engineer safe, beneficial AGI, utilizing the theories, tools, and technologies developed in Phase I.”
http://singinst.org/research/phases
May 4th, 2009 at 3:40 pm
Isaac, if you read my recent interview with Michael Vassar, you’ll see that the focus right now is on basic research. As Skaulardas points out, ideally the project would be started in two years, but there’d have to be adequate funding, with at least 6 or so genius-level researchers, and right now we have only 3. Eliezer has compared coding AI to climbing Mt. Everest — would it be worth it to start climbing it with just a tiny team and insufficient equipment, just to let people know you’re “doing something”, when it’s obvious you’ll fail and die? Probably not.
Heartland, that’s a large question, and I could answer it now, but I’ll consult SIAI researchers and see if I can make a project out of it.
May 4th, 2009 at 6:26 pm
I’m as well interested in the recent (past 3 years) work, noticing there has been no official publication since 2006/2007 (at least nothing that can be found on their - your
- website, or can you point me to something?)
May 4th, 2009 at 6:35 pm
1. What is the total in SIAI’s bank account right now?
2. What are your strategies for securing more funding?
3. Apart from room and board for the researchers, what funds or equipment are needed for doing mathematical research?
4. Yudkowsky has publicly stated he does not believe Ben Goertzel’s AI theory to be reasonable. If I send in a check earmarked “SIAI Research Fund” (which I have), is it going to Yudkowsky or Goertzel?
5. I too would like to know the current status of CEV, especially since the original document from 2004 refers to itself as “dreadfully obsolete.”
6. What is the status of the Doug Wolens Singularity documentary?
May 4th, 2009 at 6:52 pm
Do you have a roadmap for getting SIAI “on the radar”? I.e. e.g., in 5 years, there’ll be articles on SIAI in every major newspaper, Eliezer or someone else from SIAI or yourself appearing on every talk show answering dumb superstitionist questions honestly thus failing utterly and resulting in a PR catastrophe and deaththreats for SIAI and transhumanism in general, etc. usual… SIAI will be popular acronym for “Geeks Gone Wild and Off the Deep End”.
May 4th, 2009 at 7:16 pm
Congrats, Michael!
May 4th, 2009 at 8:35 pm
Congratulations.
May 4th, 2009 at 10:31 pm
I just read the SingInst research phases. Neither of them mentions Friendly AI at all. For Shame.
May 4th, 2009 at 11:38 pm
I agree with #4. We should upgrade ourselves first (if the politicians do this to themselves first, we’re gonna be in even more of a world of hurt).
– –
Information is key. A disembodied AI seems to me that it would be a locked-in mentality that would experience needles of info stabbing into it from nowhere. What the thingy needs is the ability to sense, feel, control or move some aspect of itself in the real world. Similar creatures can empathize, and the dissimilar not so much. Are you gonna cuddle up with your Roomba or a cute fuzzy pet?
Is the SIAI gonna be getting into androids? Discussing phenomenology with an android would be a kick–eh?
May 5th, 2009 at 3:40 am
Sounds like an awesome job. Congratulations and good luck!
May 5th, 2009 at 10:59 am
Congrats, Michael!
I believe you’re a perfect fit for the job.
Keep going!
May 5th, 2009 at 3:39 pm
What sort of protection does the SIAI have? What protection does it have concerning investors? What’s to stop some corporation or bank from wanting to toss in $50 million and incorporate the Institute and steal the research or worse yet use it for its own greedy ends?
May 5th, 2009 at 4:37 pm
congrats! I’m sure there’s great things ahead.
May 6th, 2009 at 2:55 am
Congratulations, Michael. You will, I’m sure, also want to help (to the extent you can) with the theoretical development. To this end, I suggest you get Eli and Ben and others to check-out Peter Danielson’s book *Artificial Morality* (Routledge, 1992). Yeah, I know, it’s 17 yrs old. But still worth perusing. See also the works of Christine Korsgaard and J. David Velleman on practical reason and reflection. And, as I’ve urged before, check-out James’s Otteson’s work on *Adam Smith’s Marketplace of Life*, his exploration of Smith’s *Theory of Moral Sentiments*…
You’re perfect for the new position. And Delta Infinity is—ultimately, anyway—right, strategically. The awareness of the importance of FAI should be made to, as he put it, “go viral”.
Ciao…
May 6th, 2009 at 12:44 pm
“What sort of protection does the SIAI have? What protection does it have concerning investors? What’s to stop some corporation or bank from wanting to toss in $50 million and incorporate the Institute and steal the research or worse yet use it for its own greedy ends?”
Yes, interesting to know. This is a point that strikes me. I don’t want google or microsoft to own a fully functionable AGI engine!
May 6th, 2009 at 4:01 pm
Delta Infinity,
Not really, too much exposure will guarantee backlash. I prefer an optimal level of exposure, which I’d say would be about 2-3 times the current exposure. Also, the value of exposure is limited. In the last three years, there have been Singularity write-ups in over 100 major papers worldwide, but what we really care about are locating serious donors and talented researchers. Media exposure and donors/researchers do not linearly scale. Sometimes, continuous exposure in a niche context (like this blog) can actually have a greater benefit than intermittent exposure in mass media.
RMK, answers to your questions:
About $300K.
Communicating our activities to the public, soliciting donations from the public, speaking 1-on-1 with people who are willing to make larger contributions, scaling up Summits, selling DVDs of past Summits, and career maturation of our supporter base.
Used laptops and cars would be useful, and can be accepted as tax-deductible donations. Also, funds to fly people to the US, e.g. from Russia, and pay some of their associated expenses.
Yudkowsky, but I think what he originally said (and this was over 5 years ago) was that more effort should be put in ahead of time to FAI research, rather than taking a “wait and see” approach. This is different than Yudkowsky criticizing the usefulness of Goertzel’s AI theory as a path to AGI. It’s worth noting that since then Goertzel has been devoting much more speaking and thinking time to Friendliness issues than he used to.
I’m currently following up on this. Basically, one thing to remember is that it takes TONS of time to write down these detailed ideas and Eliezer has already been spending a lot of time writing (rather than researching) in his 1.5 year series on Overcoming Bias.
Same as before, he still needs $45K so he can work on editing full-time. We only raised $625 last time. I will post a notice on this blog soon.
Isaac,
We don’t have to mention it. SIAI came up with the concept of Friendly AI, has produced over a thousand pages on the topic, gotten coverage for it from dozens of media outlets, and SIAI researchers have made Friendly AI their life goal. It’s not like there’s any doubt that SIAI is committed to Friendly AI.
The “protection” we have is our own commitment to a beneficial Singularity. If you look at SIAI’s history, 9 times out of 10 we are accused of being too obsessed with this goal. Too obsessed with the non-profit model where we have control instead of shareholders or investors, too obsessed with ethics at the expense of pragmatism, etc. If you look in the old SL4 archives, there are hundreds of pages of diatribes by Eliezer on this very issue. Our commitment to the integrity of the Singularity has been called “cult-ish” by some.
May 6th, 2009 at 11:02 pm
Good. Excellent on all accounts. Keep the cult-ish work ethic.
May 7th, 2009 at 4:39 am
“But I think what he originally said (and this was over 5 years ago) was that more effort should be put in ahead of time to FAI research, rather than taking a “wait and see” approach.”
Those are the respective positions I remember from their many back-and-forths on the SL4 list, but I was referring to this thread from last December (http://www.overcomingbias.com/2008/12/open-thread.html?cid=140964300#comment-140964300) where EY claims BG’s theory is unreasonable, BG has only a loose association to SIAI and he is not interested in continuing dialogs with him.
May 7th, 2009 at 5:02 am
Is it bad etiquette to post twice in a row? I thought of some more:
Between now and coding an FAI, is the SIAI expecting to spin off a dozen narrow AI applications that it can dangle under the noses of venture capitalists? Will being able to speak to the baby AI in Lojban, for example, require any advances that can be applied to natural language understanding? It seems like that would have lots of easy commercial applications.
How much does Goertzel earn in a year from BioMind and other commercial ventures? How about Yudkowsky from speaking engagments? You can’t solicit public donations if you’re not willing to disclose what your researchers are earning from their independent projects.
I would definitely like to buy DVDs of past summits. I’m afraid my donations can only be in the three-figure range for now, but even still, I find it hard psychologically to part with money without getting an immediate tangible benefit in my hand.
May 7th, 2009 at 8:49 am
Congratulations on the new gig!
May 8th, 2009 at 9:42 am
“artificial morality” what a great term, I hadn’t come across it before. No wikipedia entry for it yet! Is it yours, if not where did it emerge?
May 8th, 2009 at 10:17 am
I don’t know, I saw it in Wallach and Allen’s book on Moral Machines. It’s definitely not new and has been kicking around for a while, though not often used.