7Apr/0931
AGI Discussion
Discussion of AGI ethics at Convergence 2008. From left going clockwise, dunno, Anna Salamon, Peter Voss, Matt Bamberger, dunno, Steve Rayhawk, dunno, dunno, me, Brad Templeton.
Discussion of AGI ethics at Convergence 2008. From left going clockwise, dunno, Anna Salamon, Peter Voss, Matt Bamberger, dunno, Steve Rayhawk, dunno, dunno, me, Brad Templeton.
April 7th, 2009 - 14:41
looks like a pretty varied group.
April 8th, 2009 - 05:08
Now is the similarity to this photo purely coincidental?
April 8th, 2009 - 05:17
Or even this one
April 8th, 2009 - 13:21
@Roko
Cool, you really made me laugh. ^^
Hm, one question that still strikes me…
which of the bearded guys is Gimli? ^^
April 8th, 2009 - 23:02
Rayhawk is clearly gimli.
But anna s would have to be Arwen….
Michael annisimov is legolas – the long hair seals it. Eliezer is definately aragorn.
I’m pretty sure that hofstader is bilbo baggins – he bore the ring of AGI for many years and now he’s slightly mad…
Kurzweil is saruman, definately.
Gosh, this is fun!
April 8th, 2009 - 23:22
“Kurzweil is saruman, definitely.”
– no, Kurzweil is Denethor.
Sandberg is Tom Bombadil.
Dale Carrico is… The cave troll from Moria!
April 9th, 2009 - 07:46
LoTr-LOL. PreDict’ing that a ProtoAgi from a european AiStartUp will be able to identify the remaining “dunno”s UnTil2012 via FaceRecognition with a ConFidence of 99% ;)
April 9th, 2009 - 08:06
Peter Voss gets to be Saruman, I think.
April 9th, 2009 - 09:44
“An AGI is dangerous tool…”
April 10th, 2009 - 14:45
LOL…Why is Peter Voss Saruman?!
Also, Shouldn’t Sauron be Fukuyama?
April 10th, 2009 - 16:54
Shouldn’t Sauron be Fukuyama?
Hmmm… actually, no. Fukuyama has grown on me recently. He understands risks, even if he is bizarrely conservative.
Sauron should be … whoever first thought of the possibility of AGI. Perhaps I.J. Good? But he seems to be apathetic rather than actively evil. No, I think that maybe Sauron should be human greed and ambition, that thing that drove over 50% of the AGI’08 conference to decide that building AGI was important enough to justify wiping out the whole human race.
Voss is on a personal quest to build unfriendly AGI and kill us all, and our children, friends and families, though superfically he looks like a good guy. Hence Saruman.
April 10th, 2009 - 22:37
Since when was Voss on a personal quest to build unfriendly AGI and kill us all, and our children, friends and families?
April 11th, 2009 - 17:43
@Isaac J:
Perhaps I should go back into serious mode and rephrase: Voss wants to build AGI. He doesn’t care about friendliness. The result of him succeeding would be all the bad things I mentioned.
April 12th, 2009 - 03:51
“He doesn’t care about friendliness.”
Because he thinks friendliness isn’t an issue or
because a positive outcome is not important?
Could you explain?
April 12th, 2009 - 13:55
“Because he thinks friendliness isn’t an issue or
because a positive outcome is not important?
Could you explain?”
– from second hand info I have on this, I do not know.
April 13th, 2009 - 23:08
I just realized Ben Goertzel isnt there. Isn’t he one of the people not interested in FAI like Voss?
April 14th, 2009 - 01:14
Ben Goertzel thinks, FAI is not that important.
April 14th, 2009 - 02:31
Christ…is there is any programmer actually working on FAI in a lab?!
April 14th, 2009 - 03:56
I don’t think so. But there may be several of them, who are not afraid of afriendly AI, I guess.
April 14th, 2009 - 08:41
Christ…is there is any programmer actually working on FAI in a lab?!
I have had discussions with the main players. It is very hard to do FAI until you understand a lot about AGI. In particular, good knowledge representations which allow you to write assertions about human-level concepts such as people, values, moral obligations, etc are required. At the moment, the closest thing would be Cyc, which doesn’t really work very well.
April 14th, 2009 - 11:52
Er, this is certainly enlightening, I had no idea Goertzel and Voss didn’t care about FAI anymore!
Goertzel is, of course, now Director of Research at SIAI, where “Friendly AI” was invented as a term. It’s hard to spin that as not interested in Friendly AI.
Here at a2i2 we talk about safety issues a lot in our AI research. I think, as a local consensus, we differ from Yudkowsky’s opinions on what safety looks like, how to define it, and how it needs to be developed.
To be definitional, we aren’t “Friendly” AI developers, because that’s a term of art that EY hasn’t yet specified to the point where others can use it. We certainly aren’t attempting to implement Coherent Extrapolated Volition, his last public attempt.
We do think about the technical issues involved in creating a human-level or above AI that remains friendly to us.
April 14th, 2009 - 13:05
Justin, what is worrying is that Peter has repeatedly said to us that he believes in objective morality (Objectivism, you know), and seems confident that any sufficiently intelligent entity will become human-friendly, but that he will be prepared if it ends up not looking that way. This is sort of vague, and I wish that a2i2 would share with the public just a little bit more about their safety ideas for the longer term. Voss’ writings on his website responding to the SIAI guidelines on Friendly AI are helpful, but we’ve seen little since then.
April 14th, 2009 - 13:41
Justin if you google video Goertzel an interview should pop up in the results where he states that he doesn’t think Friendliness is achievable. And yeah…it is kinda ironic considering he’s in the SIAI with the bulk of the FAI proponents.
April 14th, 2009 - 14:54
Justin: We do think about the technical issues involved in creating a human-level or above AI that remains friendly to us.
– this is excellent news, and I didn’t know about this. My impression from various third hand sources was that a2i2 had no “friendliness” strategy whatsoever. It’s good that you guys are working on the problem.
April 14th, 2009 - 14:59
Justin if you google video Goertzel an interview should pop up in the results where he states that he doesn’t think Friendliness is achievable. And yeah…it is kinda ironic considering he’s in the SIAI with the bulk of the FAI proponents.
which video? There are three, and I don’t want to watch all of them…
April 16th, 2009 - 09:51
Is that the Anna Salamon who studied philosophy at UCSD?
I’m just curious because it seems like quite the odd coincidence If so, as I occasionally get her email.
My last name left as an exercise to the readers.
April 16th, 2009 - 10:59
“Friendliness”, as a term of art, is about rigorous, provable safety. Goertzel, as you may guess from his papers, has a messier view of AI. He’s stated a few times that he doesn’t think Eliezer’s goals are realistic. He still thinks that safety is extremely important, but has a different view of how it can be achieved.
MA, it’s true we don’t have any public long term safety policy. Peter wrote the response to SIAI’s guidelines because he felt it was worth responding to, I think, but that CEV was too speculative to comment on publicly. And we’ve seen little from SIAI since then too, in terms of recommendations or policy.
It might be worth investigating if some agreement on practices and guidelines could be found, but I’m not sure. The few explicit strong AI projects are in very wildly different places, organizationally, developmentally, and theoretically. It might be hard to find common practical guidelines that apply to them all, much less can be agreed upon as useful.
April 16th, 2009 - 11:52
Yes, that’s that Anna Salamon. Last summer I worked on an SIAI-sponsored research project with her, Steve Rayhawk, Tom McCabe, Rolf Nelson, and assorted others. We all lived at the same house in Palo Alto for six weeks, it was fun!
Justin, it’s obvious that the SIAI is semi-obsessive about safety and makes a huge deal about it all the time, for instance most recently in an extensive sequence on Overcoming Bias. Maybe you guys are equally concerned about safety in private, it’s just less obvious, because (as a private company working towards a profit) you are probably more focused on actually building the system rather than just talking about it. There’s a bias here, where I actually spend time with SIAI researchers but you’re the only one I talk to from a2i2 regularly, so it’s reassuring to hear that that you have safety discussions at a2i2.
April 20th, 2009 - 16:43
AGI/Singularity in our lifetime means that the humans actually making it happen are alive right now or should be born pretty soon. Who are they?
June 29th, 2010 - 13:28
hi,just want to say i enjoy u r site and i bookmark it for later on
May 22nd, 2012 - 20:34
Thanks in move up in the interest any news assumption to me. They unquestionably helped me discern something.