ReadWriteWeb Mentions SIAI Wednesday, Jul 22 2009
SIAI and singularity 9:13 pm
The 12th most popular blog in the world according to Technorati, and with 250K subscribers, ReadWriteWeb, recently mentioned the Singularity Institute, in connection with the beliefs of our friend Peter Thiel. This time, unlike former times, the mentioned connection didn’t have sinister overtones.
Mike Treder at IEET recently hit Peter Thiel hard for his comments at Cato Unbound. SIAI is catching some flak from others in the transhumanist community from a perceived connection between our organization and political libertarianism. The fact of the matter is that any such connection is very loose or at least coincidental. Just because Eliezer Yudkowsky and Peter Thiel are libertarian doesn’t mean that the rest of our organization is. We have far more non-libertarian donors and supporters than libertarian ones.
SIAI is non-political, just like the Methuselah Foundation. SIAI gets donations from Thiel because of what we’re about, trying to engineer AI that helps humans rather than hurts them, just like the MF gets donations because they’re trying to extend life. Politics is not the central issue. Helping people is. Surviving the future is.
I am a liberal Democrat who lives in San Francisco who is Media Director for SIAI. Doesn’t that tell people something? Still, I may be “apocalyptic” about politics, because of three dead-simple beliefs:
1) Genuinely smarter-than-any-Homo-sapiens-you’ve-ever-met entities might be created in the next 20-40 years, through AI research, brain-computer-interfacing, or human biological enhancement.
2) Inter-species smartness matters, despite anthropocentric conceit. The earliest superintelligences will quickly become smarter than you in the same way that you’re smarter than chimps. Your ideas won’t really matter all that much then. Your entire existence will rest on the hope that superintelligences have empathy for you, otherwise you’ll die, not due to malevolence, but mere indifference.
3) When they are created, superintelligences will be much smarter than you and all your friends (by definition), and probably will be able to make so many new innovations in political theory that all of your old ideas will look like crap by comparison, no matter your political beliefs.
These beliefs are motivated by facts I know about cognitive science and technological progress, not political beliefs. If the ensuing implications seem anti-democratic, it’s not because I’m inherently anti-democratic, but because my beliefs about cogsci and tech progress have forced me to come to different conclusions in politics than traditional democrats. I know that Mike Treder and James Hughes are uncomfortable about that, but what I want to tell them is that my argument focuses on cogsci and tech progress, not politics.
If superintelligence can have better ideas about politics that make the world better for everyone, and following them would be “anti-democratic”, then I am anti-democratic. But wasn’t democracy always about keeping up-to-date about the latest political ideas and following them as I see fit? If I have only one vote, that’s it. Just because I’m pumping up the rationale behind my vote doesn’t mean that I’m about to try and override everyone else. Don’t jump to conclusions.
We’ve been talking about a hard-takeoff Singularity where a single seed AI or enhanced human becomes absolutely superior in a very short period of time due to their inherent abilities and the environment they’re operating in. The motivation part is easy to explain: no matter your goal system, the more power you have, the easier it is to implement. That includes democracy or whatever else you can name. If that’s the way that physical reality works (which is what we’re claiming), then that’s not my fault, and the belief isn’t politically motivated.
If you are a general intelligence that can copy yourself, enhance your own intelligence, thread your consciousness into thousands of subroutines, integrate computers into your brain in a couple seconds, and instantly share knowledge, you’ll be superior on this planet in days, and that’s my belief of the implications of those abilities. If you disagree, disagree with the prior sentence, don’t second-guess my political beliefs and say that my beliefs about cognitive science are mysteriously being influenced by my beliefs about politics. Politics isn’t so important to me that I let it contaminate every thought I have about science.
Every present-day political belief is just a program running on the operating system called Homo Sapiens OS. When that OS is upgraded, all the programs will be upgraded as well. No need for a computer metaphor — you can pick any metaphor you want. Any metaphor where the underlying operator gives rise to qualitatively different models and actions.
I have no beef with your political views. But if you’re right about the singularity hitting in two, four decades or thereabouts doesn’t that tend one to the thought that much of what we (you, I, those with whom we disagree) might struggle for in politics will be rendered irrelevant by post-human intelligence(s)? This does look like a path to quietism or libertarianism. To give a crude example (which I am not saying you would endorse): “hey, why bother fighting vested interests to reduce carbon emissions because super-intelligence(s) will sort it all out (or leave us to fry) anyway?”
And what if things turn out differently — if, say, humans co-evolve with the new intelligence (which is what I understand Jamais Cascio to be arguing in a piece in the July edition of The Atlantic)?
> wasn’t democracy always about keeping up-to-date about the latest political ideas and following them as I see fit
No Michael, not really. Democracy is about a society that governs itself through free discussion among relatively equal citizens. If individuals or the whole society decide to subordinate themselves to dictatorship, or a robot god, then they no longer have democracy, even if the decision itself was made democratically. That is why we do not allow people to sell themselves into slavery. That is why so many of us are uncomfortable with people who want to throw out democratic self-governance and embrace cybernetic dictatorship.
As to your assertion that it is irrelevant that Thiel, the 99% funder of SIAI, is a raving right-wing anarcho-capitalist, who supports Republicans for public office and sits on the Hoover institution board, and that your founder and head guru, Eli, is also a libertarian of some vague sort…well…good luck with that. As a liberal Democrat you may want to think more deeply about the elective affinity between libertarianism and Singularitarianism.
“The hottest places in hell are reserved for those who in times of great moral crises maintain their neutrality.” – Dante Alighieri
Professing neutrality when faced with the moral repugnance of views like Peter Thiel’s is a sure ticket to a warmer climate.
To the comments above that may all be the case, I would label myself as a liberal democrat and technoprogressive. But if it’s also the case that the fate of humanity rests on whether the first strong AI we build is Friendly or not, then obviously we NEED to make sure the first AI is Friendly. To say it is the highest priority is redundant. It may be unfortunate if the primary founder is a “raving right-wing anarcho-capitalist”, but unless you think it would better help the effort to actually turn down that money, I don’t see a better alternative. And if Thiel is 99% of the funding base, that’s because not enough other people are stepping up to do this!
I think the whole point of the article is that the politics are all well and good, but in the end we have to choose the route that looks most likely to effect a future we value, and that absolutely requires we survive to get there. If that comes out to honestly require a super powerful AI, that’s what we need to do, unless you value extinction over such a governmental system. (Personally I can imagine it serving to facilitate a (near) perfect democracy just as much as a cyber dictatorship, assuming we build it “well”). You can have the better political stance and argue against a human being, changing their mind, but you don’t get to change Nature’s mind. You play by Nature’s rules, or you die.
And with regard to Caspar Henderson, I agree we have to look at other alternatives. Even if we think one future is most likely, unless we’re supremely confident it will come about (which strikes me as overconfidence) we ought also to take actions that will be needed if that one scenario doesn’t come to pass. I think it would only be people who are either cognitively lazy or radically confident in this one scenario who would completely ignore all other worthwhile activities, such as vested interests in carbon emissions. But it does change how much energy you should be putting into certain things, like any existential risk knowledge would. Seeing a large number of ways we might not make it to the year 2050, I’m not so focused on climate change in 2100 as some of my friends.
Frank
Yes, if the only choice in the world was to take money from right-wingers to save humanity, or see humans extinguished, then its a no-brainer. Those aren’t the only choices in the world, and only it serves the interests of the wealthy to pretend they are.
I accept that AGI is a momentous undertaking, and that friendliness research is an important part of the overall intellectual endeavour, and that friendliness will be a part of ensuring cybersecurity from a-life and AI. But I think the so far completely ineffective exhortatory approach of SIAI to this project reflects the libertarain presumptions of its participants. They have been completely uninterested in building a dialogue with the existing cybersecurity community on how to prevent dangerous AI research and build security and resilience into global IT infrastructure in the event of hostile use of AI by humans, or autonomous AI. About as effective as the exhortatory approach to drug safety, or industrial safety, or disarmament, or any other real threat to humans.
J. Hughes
>Democracy is about a society that governs itself through free discussion among relatively equal citizens
Interesting. Where and when did this exist ?
Are citizens relatively equal in the United States ? If so why would someone whine about the extra influence that Thiel has because he has more money.
Does an ordinary citizen match Michael Bloomberg’s ability to fund his mayoral campaigns ?
How about the (worldwide) influence and benefits of being in Goldman Sachs or being a Goldman Sachs alumni ?
A US citizen gets the right to choose between the finalists from Coke (Republican party) and Pepsi (Democratic party).
A more open and lower threshold for getting something presented which might get made into law is the California proposition system. (Get 5% of the signatures from the total votes in the last election.) This system is generally not held up as a successful model.
Equal citizens to someone from the Bush, Kennedy or Clinton Clans ?
Fine someone does not like turning over decisions to any new AI. I would have to say that the first trial run of allowing too many decisions to be based on financial software and based on mathematical models did not go well. However, I think the point is to make better software/AI.
John Hughes and Mike Treder do not like the romantisizing of the 1920′s. I do not see any shining current great examples democracy. It appears to be which is less bad.
I also think the governing systems should be judged based on the quality of the results/plans/choices that are produced. Scientific method can be applied to analyze results and systems of governance. Using more powerful computers and software to enable unbiased examination/testing of more options and more results would be useful.
Funding of SIAI is broader than just Theil although he is the biggest donor listed
http://www.singinst.org/donornetwork
As for insufficient interaction between friendly AI and cybersecurity. I would think the cybersecurity side is part of the tango of both groups not interacting. Cybersecurity community is probably pretty focused on the issues and economic interests of the nearterm.
James, we have built and are continuing to build a dialogue with the mainstream cybersecurity and infotech communities. For instance, SIAI employees Anna Salamon and Steve Rayhawk and SIAI intern Carl Shulman recently presented papers at the European Conference on Computing and Philosophy. I have engaged in dialogue with Wendell Wallach, the author of Moral Machines, which looks at roboethics in systems of all kinds as well as general cybersecurity considerations.
James, you have to remember that our approach to friendliness pivots around our relatively high probability estimation that a single seed AI could potentially self-improve fast enough to become unrivaled in a short period of time. Though we can and do engage in dialogue with cybersecurity communities, it also makes great sense to pursue in-house research. Also, the point that Brian makes about cybersecurity community not being overly concerned about superintelligence makes sense.
Remarkable, I have said it once or twice, but finally the “somewhat arbitrary” yet “sometimes causally obligatory” link between ideological (versus pragmatic) libertarianism and transhumanism is held against the light.
Too bad that somewhere between 10-15 trillion US$ equivalent had to vaporize and several continents had to stagger closer to starvation in order to to wake up some people that some market assumptions have become dogma. But even now you can’t argue that on pain of being labeled dangerous, naive and/or a socialist.
Well, let’s hope we don’t need any Guillotines to straighten those people out.
James, first I admit that I’m flattered that one of my first postings on a transhumanist site drew attention from such a prominent figure! Also, I greatly appreciate your work with intelligent public policy that incorporates these rapidly changing technologies. I’ve been dissapointed with how little work in the policy sector seems to do so, and will soon be pursuing a Masters in International Security and Technology policy for that reason. Speaking of building connections however, I feel some of your comments can be more invidious than they need to be, especially regarding the Singularitarian community. In my opinion we’re (most) all trying to do our best to build a positive world for the majority of people, we would do well to help each other as much as possible, and we should drive a minimum number of wedges between groups with different ideas on how to do it or what exactly that world would look like. Not to say any wedge is a bad idea, I myself think some ideas or desired worlds are dangerously poor, but just that the wedges should be kept to a minimum.
I agree that a hortative approach only goes so far, and that building infrastructure to prevent dangerous AI research is critical. However my own technical beliefs make protecting against a true general AI, once it’s already constructed, seem like a potentially worthless endeavor, IF it can indeed do any significant amount of self improvement. In terms of donors it seems like the paramount issue to the discussion is whether great gobs of money from libertarian donors leads to great gobs of libertarianism (or worse) in a future world in which SIAI “succeeds”. I’m currently under the impression that the effect would be limited, but I admit I don’t know for sure.
@Frank
Have you ever known a non-profit to refuse money? Especially one that states it needs in excess of several millions of dollars to build AGI.
It’s so unfortunate that an anti-immigrant, anti-welfare, anti-feminist, racist and oxymoronically gay billionaire is the sole source of funding for the SingInst. I fear that Thiel is already bludgeoning Vassar and the other SingInst designers/leaders with his great wads of money until he gets his way. We might end up with an AGI named Lord Cato that builds the world into a gay-friendly white-male dominated libertarian socioeconomic experiment with no reset button.
FAI means no-programmer-sensitive FAI morality
(From 2002, but I don’t believe the policy has changed.)
@nickptar
Sure Eli and the other AGI midwives can subscribe to the idea of not placing personal morals and political ideology into their programming but at the end of the day Thiel is paying everyone’s paycheck. With only one source of funding he has a monopoly on making demands. If they don’t do what he says they don’t get paid.
I question the idea that greater intelligence leads to objectively better politics. The scientific method can’t tell us how society should be organized. That’s a different realm, a subjective judgment. We can expect superhuman AIs to craft more persuasive rhetoric and tighter theory. While their arguments might convince us, we shouldn’t blindly yield up our beliefs. Everyone, regardless of smarts, has their own unique perspective and goals. You don’t need credentials to express how you want to live. The debate stands completely open, free for all to enter.
The point is that Yudkowsky (from his statements on ‘Overcoming Bias’) still claims to be a moderate (small-l) libertarian ; he also claims to be at work on a ‘super ethical’ super-intelligence. But politics cannot be devoted from ethics, so his extreme (and almost certainly badly mistaken) political views must cast serious doubts on his ethical assertions. One has to wonder what else he’s wrong about.
As many readers surely know, I personally am convinced the SIAI folks are grossly mistaken about basic foundational issues, and are far far away from AGI, but of course I have to accept that I’m the one that could be deluded. Perhaps SIAI folks *are* as special as they say (they are undeniably very smart in the sense of raw IQ). But as Hanson pointed out on ‘Less Wrong’, it’s anthropically unlikely.
The way I see it, consciousness is vision, intelligence is power. Consciousness can tell you what to do, and in a vague sense, even how things are. But to actually do things and prove things, you need intelligence.
I represent vision without sufficient power, our erst-while super-geniuses represent power without sufficient vision. It’s a tragic situation, further compounded by the ironic fact that only someone with lots of vision can even recognize it.
I’ve tried to scream, shout yell, cajole, and frantically point in what I believe to be the right direction, but our super-geniuses take no notice of me and go right on marching in what I believe to be the wrong direction ;)
Given everything I’ve posted on ‘Overcoming Bias’ the only conclusion I can draw is that Bostrom, Hanson and Yudkowsky are not going to recognize that I’m right short of being handed utterly precise scientific equations on a plate proving that Bayesian Induction is merely a special case of categorization (analogy formation) and all the rest of my radical assertions. Either that, or my own ideas really are nonsense ;)
My point is, there are radically dissenting views about the nature of Intelligence/Singularity out there.
It may perhaps be a better strategy (rather than working in secret) for the SIAI to have some sort of public IT project going as well, which perhaps could be related to AGI without giving away critical information, yet still bring in some public support for the longer-term goals. I’m thinking something along the lines of the ‘Wolfram Alpha’ search engine or something similar. Set up a new start-up, become the next Google, get all the funding you need!
I myself have been working on just such a project… ;)
Just an addendum expanding on the analogy in my previous post, I said that:
Consciousness=Vision
Intelligence=Power
A super-intelligence without consciousness would be like a giant mummy… able to accelerate and run at tremendous speed in a given direction, but unable to *change* direction. And that’s why I think a non-sentient AGI cannot initiate Singularity… because it has a fixed precisely defined goal, no manner how clever the definition of that goal, it cannot self-improve, because it cannot *change* that goal, it cannot reach all possible locations in the ‘goal space’… it can only run one fixed direction.
Continuing my analogy, I see the high IQers as mummys… Olympic gold runners, able to accelerate and run much faster than me when they set off in a given direction… however they don’t know *which* direction to run in, for they are blind . I run much slower but I’m running in the right direction… because I possess *vision*. At the end of the day, ain’t no mummys ever gonna beat Geddes. ;)
@mjgeddes
So what exactly are your own ideas and what is the right direction? You referenced them but never stated them. If your answer is “go check on overcoming bias”, save your key strokes.
@kingraven
There is an abstract summary of my current ideas on the everything-list here:
http://groups.google.com/group/everything-list/browse_thread/thread/44c15d7c10a4b2ae#
This cannot be expected to make technical sense, as I told you, my strength is ‘vision’, not ‘power’. To convert these ideas into technical sense the ‘power’ knob of my brain would need to be greatly turned up. ;) However it should at least make ‘intuitive’ sense, hopefully pointing in the right general ‘direction’.
Professing neutrality when faced with the moral repugnance of views like Peter Thiel’s is a sure ticket to a warmer climate.
Sheesh, if I wanted to see people get condemned to a lake of fire for all eternity for honestly trying to work out their position on complex issues, I wouldn’t typically come to this site. Maybe I’d go back to the Southern Baptist church camp in Alabama that I attended as a teenager.
But, no, come to think of, that’s not fair. The Baptists were never that judgmental.
One area where transhumanists consistently disappointment me is politics. We can talk about accelerating change and singularities and human enhancement and the possibilities are endless, but when the subject comes to politics, everyone seems to revert to one of a very small number of philosophical templates, most of them created in the 19th century or earlier. And for some reason those are inviolate.
But that’s not to say that technology has played no role in the recent evolution of political discourse. The rise of the blogosphere and sites like Daily Kos and Free Republic have established a new “accelerated” rhetorical framework for politics which now seems to be more or less universally applied. The basic assumption behind the framework is that there is Our Group and then there is the Other. Any ideas from the Other are subjected to a three-step analysis and response:
1. Hysteria / overreaction
2. Vilification
3. Condemnation
(See Kingraven, above.)
This process has worked great for the political blogs in drawing in huge masses of eager readers, mostly the same people who think they’re up to date on current events because they watch The Colbert Report or listen to Rush Limbaugh.
Personally, I’d like to see a group such as IEET take a different approach. Maybe they could look for some kind of, oh I don’t know, Middle Way that transcends opposites? Or maybe that’s too ambitious. To use Brian’s analogy, maybe they could at least come up with a middle way that transcends Pepsi and Coke? Frankly, I would expect that sort of thing to be more in line with their world view than all this (both figurative and now literal) fire and brimstone talk.
Forgive my reductionism, but there will always be tension between those who believe that the good of the individual is primary and that the good of the group must be subordinated to it, and those who believe that the good of the group is primary and that the good of the individual must be subordinated to it. A working system (as opposed to a lofty set of ideological propositions) will inevitably consist of a series of trade-offs between those two. Technology has the potential to ease the impact of some of these trade-offs, and even replace them with new trade-offs, but the tension will never completely go away.
Even without Michael’s super-intelligences (which will show up sooner or later) the introduction of an open-source universal assembler enabled by nanotechnology and potent narrow AI could do significantly more to liberate the world’s poor than any trickle-down economic growth model or redistributionist scheme. When technology trumps political theory, I go with the technology. The vital question: would such technology be made available through some big government push or through private efforts?
Either. Both. Neither. Take your pick. Maybe if we find a way to talk with each other about these things like reasonable people we’ll come up with a completely new model that’s better than anything we’ve tried before.
The existence of a technology alone doesn’t solve any problems. Knowledge has to be employed to matter, and that involve political decisions. We’ve had the technical capacity to give everyone on the planet a comfortable material life for at least half a century. Yet our vast productive power doesn’t prevent people from starving daily. The present economic regime precludes this. Billions suffer want unnecessarily. Society must be properly organized for technology to produce the desired result.
That’s why I’m profoundly skeptical of claims that nanofactories and abundant energy will inevitably end poverty. They could and should, but human culture so far has responded poorly to plenty. When faced with effectively limitless reproduction of digital media, authorities clamped down. Political concerns, not technical ones, severely hinder access to artwork, music, literature, and programs. Sadly, I see scant reason to hope for better once nanotechnology brings digital abundance to the physical world.
Phil,
As M said, all our current political ideas are gonna seem pretty crappy in the future.
But for what its worth, I will tell you what I see with my now untrammeled transhuman vision ;)
“Future politics has much more precisely defined boundaries. I see three types of politics… socialism, capitalism and authoritarianism (as there is today) but unlike today, each has its own precisely defined domain, and the domains are kept sharply separated from each other. For instance, in the domains where socialism reins, its pure socialism and nothing else, and so with the other two types of politics. All tax is raised via a tax on land and natural resources. So natural resources are in the socialist domain and are totally democratically managed, no one owns land any more, they only rent it. But in the capitalist domain its a pure free market, no tax and no regulations. There is no income tax, all tax is generated via the aforementioned land tax. You get the idea. Sharp domains where each type of politics totally applies, no mixed economy. Horses for courses”
[...] forums and blogs should be very familiar with the kind of behavior Phil Bowermaster refers to in this comment left in response to a post on transhumanism at Accelerating Future: But that’s not to say that [...]
[...] Political Food Fight: Transhumanist/Singularitian Political Food Fight h+ Magazine and ReadWriteWeb Mentions SIAI Accelerating Future (hat tip reader David [...]
The current problem with Libertarianism is that it has so few supporters.
Even in the original piece that Thiel references, Patri Friedman quotes a source that estimates less than 16% self id with any libertarian beliefs.
So it’s either an elite that got there first (“there” being the intellectual high ground), or a minority that needs more supporters to be useful. Neither path is politically viable in the US right now, and I would think dubious anywhere else in the developed world.
And I agree with MA- either way it won’t matter to AI.
The bigger problem is the propagation of the “us vs. them” meme and whether we can create or allow to emerge AI that doesn’t self identify as “us vs other” with regards to humans. If he does, then we’re in Trouble.
If AI sees itself as a separate “us” or self from humans then it would either have to be benevolent, indifferent or hostile to humans.
Benevolence sounds good- but temporary. Indifference sounds dangerous – though perhaps initially not as much so as hostility. Best would be AI that does not self id as a separate “us.” Is that possible when we cannot transcend do it yet?
The “End of Politics” Delusion…
You have my express permission to kick the next person — especially someone advocating the embrace of radical forms of technological advancement — who tells you that they wish nothing more than to get rid of, move beyond, or otherwise……