AGI Discussion Tuesday, Apr 7 2009
Discussion of AGI ethics at Convergence 2008. From left going clockwise, dunno, Anna Salamon, Peter Voss, Matt Bamberger, dunno, Steve Rayhawk, dunno, dunno, me, Brad Templeton.
Categories:
- aerospace (3)
- AI (200)
- animal rights (14)
- anthropics (7)
- art/music (2)
- articles (2)
- BCI (13)
- bioethics (10)
- biology (30)
- brain (14)
- chemistry (3)
- comics (5)
- cryonics (35)
- cybernetics (5)
- environmentalism (4)
- ethics (8)
- events (99)
- friendly ai (115)
- futurism (148)
- humor (18)
- images (61)
- intelligence (39)
- interviews (17)
- IQ (7)
- kurzweil (1)
- life extension (79)
- lifeboat (10)
- manufacturing (5)
- math (2)
- me (12)
- meta (62)
- nanotechnology (131)
- nuclear (20)
- philanthropy (13)
- philosophy (69)
- physics (12)
- policy (5)
- politics (12)
- random (78)
- rationality (13)
- religion (8)
- risks (145)
- robotics (50)
- science (32)
- SIAI (86)
- singularity (305)
- space (42)
- superintelligence (45)
- technology (124)
- transhumanism (249)
- treaties (2)
- uploading (8)
- videos (137)
- warfare (4)
- web 2.0 (12)
Monthly:
- January 2012 (2)
- December 2011 (8)
- November 2011 (8)
- October 2011 (5)
- September 2011 (12)
- August 2011 (12)
- July 2011 (21)
- June 2011 (18)
- May 2011 (5)
- April 2011 (11)
- March 2011 (17)
- February 2011 (21)
- January 2011 (44)
- December 2010 (44)
- November 2010 (32)
- October 2010 (21)
- September 2010 (40)
- August 2010 (40)
- July 2010 (32)
- June 2010 (39)
- May 2010 (44)
- April 2010 (25)
- March 2010 (14)
- February 2010 (40)
- January 2010 (39)
- December 2009 (47)
- November 2009 (84)
- October 2009 (71)
- September 2009 (65)
- August 2009 (52)
- July 2009 (33)
- June 2009 (70)
- May 2009 (48)
- April 2009 (40)
- March 2009 (52)
- February 2009 (20)
- January 2009 (20)
- December 2008 (1)
- November 2008 (2)
- October 2008 (4)
- September 2008 (15)
- August 2008 (14)
- July 2008 (1)
- June 2008 (16)
- May 2008 (58)
- April 2008 (31)
- March 2008 (45)
- February 2008 (27)
- January 2008 (3)
- December 2007 (5)
- November 2007 (7)
- October 2007 (29)
- September 2007 (18)
- August 2007 (11)
- July 2007 (24)
- June 2007 (20)
- May 2007 (23)
- April 2007 (27)
- March 2007 (29)
- February 2007 (11)
- January 2007 (26)
- December 2006 (22)
- November 2006 (44)
- October 2006 (27)
- September 2006 (28)
- August 2006 (39)
- July 2006 (9)
- June 2006 (6)
- May 2006 (9)
- April 2006 (6)
- March 2006 (9)
- February 2006 (3)
- January 2006 (16)
RSS Feeds:
Technorati:
Author:
looks like a pretty varied group.
Now is the similarity to this photo purely coincidental?
Or even this one
@Roko
Cool, you really made me laugh. ^^
Hm, one question that still strikes me…
which of the bearded guys is Gimli? ^^
Rayhawk is clearly gimli.
But anna s would have to be Arwen….
Michael annisimov is legolas – the long hair seals it. Eliezer is definately aragorn.
I’m pretty sure that hofstader is bilbo baggins – he bore the ring of AGI for many years and now he’s slightly mad…
Kurzweil is saruman, definately.
Gosh, this is fun!
“Kurzweil is saruman, definitely.”
– no, Kurzweil is Denethor.
Sandberg is Tom Bombadil.
Dale Carrico is… The cave troll from Moria!
LoTr-LOL. PreDict’ing that a ProtoAgi from a european AiStartUp will be able to identify the remaining “dunno”s UnTil2012 via FaceRecognition with a ConFidence of 99% ;)
Peter Voss gets to be Saruman, I think.
“An AGI is dangerous tool…”
LOL…Why is Peter Voss Saruman?!
Also, Shouldn’t Sauron be Fukuyama?
Shouldn’t Sauron be Fukuyama?
Hmmm… actually, no. Fukuyama has grown on me recently. He understands risks, even if he is bizarrely conservative.
Sauron should be … whoever first thought of the possibility of AGI. Perhaps I.J. Good? But he seems to be apathetic rather than actively evil. No, I think that maybe Sauron should be human greed and ambition, that thing that drove over 50% of the AGI’08 conference to decide that building AGI was important enough to justify wiping out the whole human race.
Voss is on a personal quest to build unfriendly AGI and kill us all, and our children, friends and families, though superfically he looks like a good guy. Hence Saruman.
Since when was Voss on a personal quest to build unfriendly AGI and kill us all, and our children, friends and families?
@Isaac J:
Perhaps I should go back into serious mode and rephrase: Voss wants to build AGI. He doesn’t care about friendliness. The result of him succeeding would be all the bad things I mentioned.
“He doesn’t care about friendliness.”
Because he thinks friendliness isn’t an issue or
because a positive outcome is not important?
Could you explain?
“Because he thinks friendliness isn’t an issue or
because a positive outcome is not important?
Could you explain?”
– from second hand info I have on this, I do not know.
I just realized Ben Goertzel isnt there. Isn’t he one of the people not interested in FAI like Voss?
Ben Goertzel thinks, FAI is not that important.
Christ…is there is any programmer actually working on FAI in a lab?!
I don’t think so. But there may be several of them, who are not afraid of afriendly AI, I guess.
Christ…is there is any programmer actually working on FAI in a lab?!
I have had discussions with the main players. It is very hard to do FAI until you understand a lot about AGI. In particular, good knowledge representations which allow you to write assertions about human-level concepts such as people, values, moral obligations, etc are required. At the moment, the closest thing would be Cyc, which doesn’t really work very well.
Er, this is certainly enlightening, I had no idea Goertzel and Voss didn’t care about FAI anymore!
Goertzel is, of course, now Director of Research at SIAI, where “Friendly AI” was invented as a term. It’s hard to spin that as not interested in Friendly AI.
Here at a2i2 we talk about safety issues a lot in our AI research. I think, as a local consensus, we differ from Yudkowsky’s opinions on what safety looks like, how to define it, and how it needs to be developed.
To be definitional, we aren’t “Friendly” AI developers, because that’s a term of art that EY hasn’t yet specified to the point where others can use it. We certainly aren’t attempting to implement Coherent Extrapolated Volition, his last public attempt.
We do think about the technical issues involved in creating a human-level or above AI that remains friendly to us.
Justin, what is worrying is that Peter has repeatedly said to us that he believes in objective morality (Objectivism, you know), and seems confident that any sufficiently intelligent entity will become human-friendly, but that he will be prepared if it ends up not looking that way. This is sort of vague, and I wish that a2i2 would share with the public just a little bit more about their safety ideas for the longer term. Voss’ writings on his website responding to the SIAI guidelines on Friendly AI are helpful, but we’ve seen little since then.
Justin if you google video Goertzel an interview should pop up in the results where he states that he doesn’t think Friendliness is achievable. And yeah…it is kinda ironic considering he’s in the SIAI with the bulk of the FAI proponents.
Justin: We do think about the technical issues involved in creating a human-level or above AI that remains friendly to us.
– this is excellent news, and I didn’t know about this. My impression from various third hand sources was that a2i2 had no “friendliness” strategy whatsoever. It’s good that you guys are working on the problem.
Justin if you google video Goertzel an interview should pop up in the results where he states that he doesn’t think Friendliness is achievable. And yeah…it is kinda ironic considering he’s in the SIAI with the bulk of the FAI proponents.
which video? There are three, and I don’t want to watch all of them…
Is that the Anna Salamon who studied philosophy at UCSD?
I’m just curious because it seems like quite the odd coincidence If so, as I occasionally get her email.
My last name left as an exercise to the readers.
“Friendliness”, as a term of art, is about rigorous, provable safety. Goertzel, as you may guess from his papers, has a messier view of AI. He’s stated a few times that he doesn’t think Eliezer’s goals are realistic. He still thinks that safety is extremely important, but has a different view of how it can be achieved.
MA, it’s true we don’t have any public long term safety policy. Peter wrote the response to SIAI’s guidelines because he felt it was worth responding to, I think, but that CEV was too speculative to comment on publicly. And we’ve seen little from SIAI since then too, in terms of recommendations or policy.
It might be worth investigating if some agreement on practices and guidelines could be found, but I’m not sure. The few explicit strong AI projects are in very wildly different places, organizationally, developmentally, and theoretically. It might be hard to find common practical guidelines that apply to them all, much less can be agreed upon as useful.
Yes, that’s that Anna Salamon. Last summer I worked on an SIAI-sponsored research project with her, Steve Rayhawk, Tom McCabe, Rolf Nelson, and assorted others. We all lived at the same house in Palo Alto for six weeks, it was fun!
Justin, it’s obvious that the SIAI is semi-obsessive about safety and makes a huge deal about it all the time, for instance most recently in an extensive sequence on Overcoming Bias. Maybe you guys are equally concerned about safety in private, it’s just less obvious, because (as a private company working towards a profit) you are probably more focused on actually building the system rather than just talking about it. There’s a bias here, where I actually spend time with SIAI researchers but you’re the only one I talk to from a2i2 regularly, so it’s reassuring to hear that that you have safety discussions at a2i2.
AGI/Singularity in our lifetime means that the humans actually making it happen are alive right now or should be born pretty soon. Who are they?
hi,just want to say i enjoy u r site and i bookmark it for later on