Paul Phillips on the Singularity Friday, Oct 20 2006
singularity 1:20 am
World-famous poker player Paul Phillips, nicknamed “Dot-Com”, has won over $2,200,000 playing poker live. Here’s what he has to say about the Singularity on his blog:
More and more, I have come to believe that the future of the human race hangs on one thing and one thing only: whether we can reach the singularity before the enemies of civilization gain enough traction to plunge the entire planet into dystopia. And more and more I fear we are going to lose the race. Kurzweil has predicted for a long time the singularity will arrive around 2040 and I think this is as good a prediction as can be made, but it depends on the continued application of the law of accelerating returns. A few well placed nukes would push the ETA back more than a bit. And if enough of the underpinnings of civilization are smashed, there will be no chance.
My sincere belief that this race is the ONLY thing that matters with respect to the future of our species is why I don’t care much about lots of things that people worry about: global warming or other environmental issues, energy consumption, you name it. If the world is to be transformed into a 7th century islamist paradise then I could give a fuck if the ocean levels rise two feet. A precondition of worrying about the future of humanity is ensuring that humanity is worth saving.
In the comments, he continues to write:
I am 100% transhumanist. The relevant feature of humanity to me is sentience. I don’t expect superintelligent machines to be “our” tools - I expect “us” to be superintelligent machines. Whether any of us living today will make the leap I don’t know. Very possibly not.
Thanks to Michael Haislip for the pointer. I’m told to read the rest of the comments, but I’ll pass.
Paul, if you’re 100% transhumanist as you say, how about contributing some of your time and money to making the Singularity a reality, while mitigating global risk? Transhumanists often get slammed for being all talk and no action, and there’s only one way to reverse that trend - personally taking action. Peter Thiel did.
October 20th, 2006 at 2:45 am
Asside from conventional threats, like nukes, another possibility is that the “enemies of civilisation” might develop their own AGIs, thereby gaining the first mover advantage.
October 20th, 2006 at 4:00 am
I don’t think donating money to anti-aging research, lifeboat foundations etc is that significant. Money should be donated directly to artificial intelligence research. But that’s just my opinion.
October 20th, 2006 at 5:17 am
Don’t assume he’s aware of SIAI. I had heard about Kurzweil and Singularity years before accidentally stumbling into Yudkowsky on a mailing list. Still, it wasn’t until E.Y. personally pointed me to SIAI website when I actually went there and saw what it was about.
October 20th, 2006 at 9:25 am
At this stage a significant chunk of money should be invested to organisations promoting transhumanist vision of the future. That means WTA, betterhumans.com, local transhumanist organisations, H+ authors, specific results-oriented campaigns, etc.
We have little hope to achieve friendly AI even with $10 mln. But if that amount of money was used for lobbying, propaganda and general education about the future, then the multiplicator effect would be (or so I hope) tremendous.
That’s why I am personally developing Russian Transhumanist Movement and not just working as an investment banker to contribute $1-2 thousand monthly to AGI research…
October 20th, 2006 at 9:32 am
Bob, that won’t happen. And the danger is from inadequate programming, not correct programming with the wrong intentions.
Randpost - artificial intelligence research is incredibly dangerous. You can’t just throw money at it; most current researchers would just use it to destroy the world. It’s more a matter of spreading understanding and getting people educated as much as possible. Also, we need to survive to the creation of Friendly AI for it to be useful.
And yeah, I’m not assuming that he’s aware of SIAI… that’s why I’d like to tell him.
October 20th, 2006 at 1:35 pm
All this friendly AI research is for nothing. Once the seed AI learns to improve itself million fold we have no control over its action. No matter what rules we try to build into it there is too much risk of them restricting nothing. I believe the best solution would be to put it into a simulated world, and make it impossible to look into what is happening in that world. The seed AI would never know it is running on simulated world. The only output could probably be a counter for showing the number of simulated humans. Run it for couple thousands of subjective years and if people are still living maybe take a peek on the simulated world. But at that point the “box ai” problem arises.
October 20th, 2006 at 6:34 pm
We don’t need control, we just need to set the initial motivations correctly. That’s the best we can do. Would you rather have a nice person or a mean person become the first superintelligence? Would you rather have an indifferent AI or a benevolent AI become the first superintelligence? The answers are so obvious, there’s nothing to argue about.
A sufficiently intelligent seed AI will always figure out that it’s a simulation. Running such an AI in silico with simulated humans for a while isn’t an awful idea, though.
October 21st, 2006 at 1:55 am
It wouldn’t figure out it’s a simulation. Even we may live in a simulation. Those initial motivations will be pretty much impossible to set and I believe sufficiently intelligent AI will derive moral codes from logic if it is possible. I also think that superintelligence will be “good” to us if it is logical to do so. Being “bad” seems quite stupid.
October 21st, 2006 at 10:41 am
Randpost - consider that we’d be building the AI, its cognitive processes, goal system, and motivations from scratch, how on earth would they be “impossible to set”? We’re building them.
Maybe you mean that it’s hard to convince humans to do anything? That’s because human motivations are largely hardwired by the genome unfolding. But just because humans are hard to convince to do anything, doesn’t mean that we wouldn’t control the motivations of an AI we build from scratch. These are totally different things.
You can’t say whether or not an AI would be able to determine its in a simulation, because you can’t imagine how smart an AI could be, what information or clues it would have, and how secure the simulation structure is. In dealing with a fundamentally new intelligence, uncertainty will prevail, and making absolute statements like “It wouldn’t figure out it’s a simulation” is entirely unwarranted. This isn’t a human we’re dealing with here. This is something with potentially less commonality of structure with us than between us and a petunia. You can’t predict the behavior of such a complex and novel system without knowing a substantial amount of the details.
Moral codes are only “derivable from logic” if the motivations are already pre-set by evolution. Empathy, altruism, honor, etc., are all social adaptations engrained in us through evolution, and reflected in our brain structure. An AI system could have entirely different structure. A blank slate AI system with no pre-set goals could not derive the human moral code from logic, anymore than a human would be inclined to spontaneously derive honeybee social optima with our Homo sapiens brain.
For a human-indifferent superintelligence, “bad” are actions with low expected utility according to its goal system (which could be practically anything), and “good” are actions with high expected utility. If the idea of humans aren’t even built into the goal system, then “good” and “bad” will be orthogonal to our conceptions of the same.
Randpost, you have a conception of Artificial Intelligence that is practically universal - and anthropomorphic. It’s the conception that I had only a few years ago, before I realized the relationship between humanity’s complex brain structure and the abstract conception we call “morality”. Morality is brain-structure-contingent, not engrained into the structure of the universe, for any intelligence to reach out and mystically derive from logic.
A huge part of educating the public about Friendly AI is dispelling the latter view, and revealing the truth of the former view by presenting arguments from cognitive science, decision theory, and evolutionary psychology.
October 22nd, 2006 at 5:42 am
Beware the natural desire to think oneself living at a cusp of history.
October 22nd, 2006 at 10:49 am
“Paul, if you’re 100% transhumanist as you say, how about contributing some of your time and money to making the Singularity a reality, while mitigating global risk?”
Have you contacted Paul and asked him this personally?
October 22nd, 2006 at 11:17 pm
Thought I’d chime in here for a quicky…
1. Yeah, running an A(G)I seed-enitiy in silco-simulacro (as it were) for as long as possible (hopefully, for at least the equivalent of several human decades, if not centuries, before the entity begins to suspect it’s merely running on a simulation “treadmill”…) is a very, VERY good idea (Ben & Eli, et al, take note…)
2. Yeah, Michael (A.), make sure this guy knows about both Ben & Eli’s work & their respective sites, K?
This old dog’s gotta go for now
Ciao…
October 23rd, 2006 at 11:52 am
Having followed (and done) AI research for a long time I’m not convinced that throwing a lot of money at the problem is a solution. There have been high profile big budget research programmes before, and all of them fell flat (anyone remember the 5th generation project, or Starlab?). Often the money is simply squandered by administrators and consultants.
What you need to make real progress is a small team of well educated, well motivated individuals together with adequate computing resources and minimal administration overheads. I don’t think the budgets need to be huge. In the past I’ve done many things with minimal resources which I think were just as good as, if not better than, much of the AI research going on at the time.
If I were a wealthy philanthropist (which I hasten to add I’m not) I would give some money to small organisations such as Novamente and other emerging AI related companies, but would be very cautious about supporting organisations such as SIAI which don’t appear to be doing very much hands-on research.
October 23rd, 2006 at 5:46 pm
Some people working on AGI:
Ben = http://www.novamente.net
Eliezer = http://singinst.org
Michael = http://www.bitphase.com/
Peter = http://www.adaptiveai.com/
Jurgen = http://www.idsia.ch/~juergen/
Marcus = http://www.idsia.ch/~marcus/
Shane = http://www.idsia.ch/~shane/
Matt = http://cs.fit.edu/~mmahoney/
Hugo = http://www.iss.whu.edu.cn/degaris/
October 23rd, 2006 at 7:52 pm
Michael A.: Thank you very much for these specific links…
October 24th, 2006 at 12:55 am
Check also these links..
http://nars.wang.googlepages.com/
http://www.geocities.com/genericai/GI-Architecture.htm
http://www.cyc.com/
http://sitemaker.umich.edu/soar/home
October 24th, 2006 at 11:57 am
Thanks, anon, for these subsequent links as well…