Accelerating Future Transhumanism, AI, nanotech, the Singularity, and extinction risk.

1Feb/1142

I’m Quoted on Friendly AI in the United Church Observer

This magazine circulates to 60,000 Canadian Christians.  The topic of the article is friendly AI, and many people already said that they thought this was one of the best mainstream media articles on the topic because it doesn't take a simplistic angle and actually probes the technical issues.

Here's the bit with me in it:

Nevertheless, technologists are busy fleshing out the idea of "friendly AI" in order to safeguard humanity. The theory goes like this: if AI computer code is steeped in pacifist values from the very beginning, super-intelligence won't rewrite itself into a destroyer of humans. "We need to specify every bit of code, at least until the AI starts writing its own code," says Michael Anissimov, media director for the Singularity Institute for Artificial Intelligence, a San Francisco think-tank dedicated to the advancement of beneficial technology. "This way, it'll have a moral goal system more similar to Gandhi than Hitler, for instance."

Many people who naively talk about AI and superintelligence act like superintelligence will certainly do X or Y (of course there are all sorts of intuitive camps, "they'll just leave us alone and go into space" is a popular sentiment) no matter what the initial conditions, implying that trying to set the initial conditions doesn't matter.

Would you rather have an AI with initial motivations closer to Gandhi or Hitler? If you have any preference, then you've just demonstrated concern for the Friendly AI problem. It's remarkable that I actually have a challenging time arguing on a daily basis that an AI with more in common with Gandhi would be better to build first than one with more in common with Hitler, but it's true.

Some people say, "but, whatever initial programming it has will be gone after many cycles of self-improvement". No, not necessarily, because the AI will be making its own programming changes. It will dictate its goal structure, not outside forces. More like a being creating itself than an evolution-made being with a goal system filled with strange attractors that flip back and forth depending on immediate context (humans).

Setting the initial conditions for AI properly is probably the most important task humanity faces, because AGI seems more likely to reach superintelligence first than human intelligence enhancement, despite the better science fiction movie potential and personal/tribal identification possibilities of the latter. John Smart presents a few good reasons why this is likely in his Limits to Biology essay.

Comments (42) Trackbacks (0)
  1. As I can’t readily recall an instance of initial motivations being important to the ultimate motivational disposition of a recursively self-improved AI, initial motivations are likely not important.

    ;-)

    • The reason initial motivations matter is because they direct how so-called “ultimate motivation[s]” will evolve.

      For instance, imagine the following initial motivation: do nothing except run a subroutine displaying the windows logo. How do you get from that to a new ultimate motivation, much less a destructive one?

      Although the example is simplistic, the point is general. For ultimate motivations to arise, they must develop from somewhere. Unless you think that ultimate motivations develop randomly, then initial motivations are the causal origin of any AI behavior, including the behavior of developing new motivations. You are possibly right, if you can show that all AI behavior will inevitably be random. But I doubt you can show that AI behavior will inevitably be random.

    • Panda, he was being sarcastic… (see the smiley face?) Your sarcasm-cluelessness reminds me of myself, though.

      I think people imagine motivations coming along inherently with agenthood, it’s not about randomness.

  2. Congrats on the citation Michael, and thanks for the spirit of open inquiry re: the Christian newspaper.

    I also thought it was a terrific article – the last line was especially profound.

  3. Mr. Anissimov, you say “It’s remarkable that I actually have a challenging time arguing on a daily basis that an AI with more in common with Gandhi would be better to build first than one with more in common with Hitler, but it’s true.”

    Why so challenging ?. I ask you what Stalin himself asked once – How many tank battalions did Gandhi build.

    You censored my comment from yesterday, the Ten Indian Commandments…

    If somebody had written bad about Lenin or Saddam, you would post it immediately, yes?. Seems you are a closet religious fanatic.

  4. Cool post keep up the good work, thanks for the information, I will pass your post on to my friends.

  5. It seems like #HashTraffic is just not doing work on Silicon Florist anymore?

  6. Very good blog! Do you have any tips and hints for aspiring writers? I’m planning to start my own blog soon but I’m a little lost on everything. Would you advise starting with a free platform like WordPress or go for a paid option? There are so many options out there that I’m totally overwhelmed .. Any ideas? Cheers!

  7. After checking out a handful of the blog posts on your website, I truly like your technique of writing a blog. I bookmarked it to my bookmark site list and will be checking back in the near future. Take a look at my website as well and tell me what you think.

  8. An impressive share! I have just forwarded this onto a friend who had been doing a little homework on this. And he actually bought me lunch simply because I stumbled upon it for him… lol. So allow me to reword this…. Thank YOU for the meal!! But yeah, thanks for spending the time to discuss this topic here on your internet site.

  9. The next time I read a blog, Hopefully it doesn’t disappoint me as much as this particular one. I mean, Yes, it was my choice to read, nonetheless I genuinely believed you would probably have something interesting to say. All I hear is a bunch of whining about something you could possibly fix if you were not too busy searching for attention.

  10. You need to take part in a contest for one of the finest blogs on the net. I most certainly will recommend this blog!

  11. Very good article! We are linking to this particularly great article on our website. Keep up the great writing.

  12. Good post. I learn something more challenging on different blogs everyday. It would always be stimulating to learn content from different writers and practice slightly one thing from their store. I?d choose to make use of some with the content material on my weblog whether or not you don?t minBlog Happy. Natually I?ll give you a hyperlink in your internet blog. Thanks for sharing.

  13. When I initially commented I appear to have clicked the -Notify me when new comments are added- checkbox and now each time a comment is added I get four emails with the exact same comment. Is there a means you are able to remove me from that service? Appreciate it!

  14. Everything is very open with a clear description of the issues. It was truly informative. Your site is extremely helpful. Thanks for sharing!

  15. I’m more than happy to uncover this site. I wanted to thank you for your time due to this wonderful read!! I definitely appreciated every little bit of it and i also have you book marked to see new information in your site.

  16. Hello! I could have sworn I’ve visited this website before but after going through some of the articles I realized it’s new to me. Regardless, I’m definitely delighted I found it and I’ll be book-marking it and checking back often!

  17. I wanted to thank you for this great read!! I absolutely enjoyed every bit of it. I have got you book-marked to look at new stuff you post…

  18. May I simply just say what a comfort to discover someone that really knows what they are talking about online. You definitely realize how to bring a problem to light and make it important. More people must look at this and understand this side of the story. I was surprised that you are not more popular given that you most certainly have the gift.

  19. Way cool! Some extremely valid points! I appreciate you writing this article plus the rest of the website is also really good.

  20. I’d recommend this because it’s actually readable. Unusual sad but true.

  21. I absolutely love your blog and find most of your post’s to be precisely what I’m looking for.Would you offer guest writers to write content to suit your needs?I wouldn’t mind composing a post or elaborating on most of the subjects you write about here. Again, awesome website!

  22. I’ve in recent times on track a blog, and the information you offer proceeding this website has helped me a lot. Thank you for all of your time & work.

  23. Aw, this was a really great post. In theory I’d like to write like this also taking time and real effort to make a good article… but what can I say… I procrastinate alot and never seem to get something done.

  24. Glad I found this site. Thank you for the article.

  25. Thank you very much for this great article. The best wishes for you

  26. I have been surfing on-line more than 3 hours as of late, yet I never discovered any interesting article like yours. It is pretty worth sufficient for me. In my view, if all web owners and bloggers made excellent content material as you did, the internet will be a lot more useful than ever before.

  27. Glad I found this site. Thanks for the article.

  28. That is very fascinating, You’re an excessively skilled blogger. I have joined your feed and sit up for in search of extra of your fantastic post. Additionally, I’ve shared your site in my social networks!

  29. Very good blog! Do you have any tips for aspiring writers? I’m hoping to start my own blog soon but I’m a little lost on everything. Would you advise starting with a free platform like WordPress or go for a paid option? There are so many choices out there that I’m completely overwhelmed .. Any recommendations? Thank you!

  30. very interesting details you have remarked, thankyou for posting.

  31. Regarding visas for international students, while I agree that in some instances visas are very difficult for some students in particular countries to get,and sometimes it seems to relate to particular immigration employees, I do not believe that getting visas is the difficult part. The area I think the sector needs to have a full conversation on is regarding those students having not performed or not having had the required attendance being able to stay in the country and attend another institution.I understand that the policy in theory means that a student must return home and then apply again but in reality this does not occur and there are many international students still in New Zealand, having either abandoned a course or been withdrawn through not attending or participating. These students are rorting the system and Immigration needs to make sure their database systems align with MOE ones in order to identify these students who should be either enrolled or returned home.

  32. There could be a double whammy coming up. A big Pheua Thai victory, and an International Court of Justice decision generating a tsunami of international opinion repudiating the military/nationalist/monarchist/fascist aggression against Cambodia. The n/m/f faction in the military could lose enough face for the Thaksin faction to make a comeback. If the latter were to finally morph into a strong pro-parliament faction, the bad times could be over. A triple whammy might not be necessary.

  33. So what then Matty? If Samak resigns what would PAD want next?

  34. Howdy very nice website!! Man .. Beautiful .. Wonderful .. I’ll bookmark your website and take the feeds also…I’m happy to find a lot of useful information right here within the publish, we need develop extra techniques on this regard, thank you for sharing. . . . . .

  35. Well articulated AjarnMichael!I find it deeply disturbing that many highly respectable, knowledgeable and talented commentors are so quiet (or so easily, conveniently overlook or even dismissive?) about the War on Drugs (part 2 is sadly on the drawing boards), Krue-sae and Takbai.

  36. What if I already have stuff on LAG group one?I already have ports 27, 28, 29, 30, 31, 32, 33 and 34 configured forLACP (L) and LAG (1)Once I add ports 45, 46, 47 and 48 to a new LAG (2) and LACP (L)Wont that kill my connection form the workstation I am working on?Id need to run that command on both switches?I did that earlier today via the web interface and I had to reboot the switches to get access back sense it only saved to the running config and not to the start up config.I’m adding both of my running configs from both switches below. For some reason its not showing the current LAG setup on the Server Cabinet switch.Cabniet Switch Runinng Configurationcab5448# show runninginterface ethernet g48description FiberLinkexitinterface range ethernet g(11-12)dot1x single-host-violation forwardexitinterface range ethernet g(11-12)dot1x multiple-hostsexitvoice vlan oui-table add 0001e3 Siemens_AG_phone________voice vlan oui-table add 00036b Cisco_phone_____________voice vlan oui-table add 00096e Avaya___________________voice vlan oui-table add 000fe2 H3C_Aolynk______________voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phonevoice vlan oui-table add 00d01e Pingtel_phone___________voice vlan oui-table add 00e075 Polycom/Veritel_phone___voice vlan oui-table add 00e0bb 3Com_phone______________interface range ethernet g(27-34)channel-group 1 mode autoexitiscsi target port 860 address 0.0.0.0iscsi target port 3260 address 0.0.0.0interface vlan 1ip address 192.168.2.202 255.255.255.0exitip default-gateway 192.168.2.200hostname cab5448username admin password f188ad00f8e6c0cdaaf59a0e4a3e899a level 15 encryptedsnmp-server location “Server Cabniet”snmp-server contact “David L Litz”snmp-server community insynch rw view DefaultSuperclock timezone -4asset-tag BFMS6M1Default settings:Service tag: BFMS6M1SW version 2.0.0.46 (date 14-Apr-2011 time 13:10:53)Gigabit Ethernet Ports=============================no shutdownspeed 1000duplex fullnegotiationflow-control offmdix autono back-pressureinterface vlan 1interface port-channel 1 8spanning-treespanning-tree mode STPqos basicqos trust coscab5448#Closet Switch Running Configurationclst5448# show runninginterface ethernet g48description FiberLinkexitinterface range ethernet g(45-48)flowcontrol onexitvoice vlan oui-table add 0001e3 Siemens_AG_phone________voice vlan oui-table add 00036b Cisco_phone_____________voice vlan oui-table add 00096e Avaya___________________voice vlan oui-table add 000fe2 H3C_Aolynk______________voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phonevoice vlan oui-table add 00d01e Pingtel_phone___________voice vlan oui-table add 00e075 Polycom/Veritel_phone___voice vlan oui-table add 00e0bb 3Com_phone______________iscsi target port 860 address 0.0.0.0iscsi target port 3260 address 0.0.0.0interface vlan 1ip address 192.168.2.201 255.255.255.0exitip default-gateway 192.168.2.200hostname clst5448logging console debugginglogging buffered debugginglogging file debuggingusername admin password f188ad00f8e6c0cdaaf59a0e4a3e899a level 15 encryptedsnmp-server location “Hallway Closet”snmp-server contact “David L Litz”snmp-server community insynch rw view DefaultSuperclock timezone -4asset-tag CFMS6M1Default settings:Service tag: CFMS6M1SW version 2.0.0.46 (date 14-Apr-2011 time 13:10:53)Gigabit Ethernet Ports=============================no shutdownspeed 1000duplex fullnegotiationflow-control offmdix autono back-pressureinterface vlan 1interface port-channel 1 8spanning-treespanning-tree mode STPqos basicqos trust cosclst5448#

  37. Indeed, it is quite smart. Similarly, it told me off for using Firefox. I wonder what it says if you actually are using IE6?


Leave a comment

Trackbacks are disabled.