Keith Elis on the Financial Future of AGI Friday, Sep 15 2006
friendly ai 12:36 am
Keith Elis’ statement from a discussion on the newly-created Singularity list:
Hi all,
There is a possibility that at some point in the future, government
agencies, wealthy foundations, and non-profits will significantly
increase expenditures as to AGI development. When the purse-strings
open, and the money flows, it will flow like tax dollars, bequests, and
donations do — toward politically tenable projects. Yudkowsky’s
Friendliness theory, whether you agree with it’s technical feasibility
or not, is very effectively positioning the Singularity Institute’s
future AGI projects to be Politically Friendly.In the summer of 2003, the US media reported on an attempt by DARPA to
put a futures market in place which would ostensibly be able to forecast
certain undesirable events such as terrorist attacks, assassinations and
the like. The idea was to find a way to elevate our awareness before a
threat materialized, and so DARPA was studying these prediction methods.
Now, while the idea itself has theoretical and practical merit, some
members of Congress sensed an easy victory against an injured opponent
and piled on. There was no debate; there was no thoughtful consideration
of the project’s chances for success; there was no collective desire to
learn more about the idea. The project was incinerated by fiery
political soundbites, and no opposing voice was willing to be
incinerated along with it. The incident caused Adm John Poindexter to
resign in disgrace and we will never hear of the US government playing
with prediction markets again in our lifetimes.There is a lesson here for everyone working on an AGI project that “just
needs funding to get there.” You must be politically tenable. Funding
your project must be justifiable in a soundbite. And it should take a
Ph.D. droning on and on for pages in technical jargon to present an
argument against you, your theory, your design, and your most likely
outcome. As an exercise, and remembering that you’re really, really
smart, and the rest of us aren’t, how do you debate against the
following statement?“We should ensure, in fact guarantee, that AGI doesn’t wipe out
humanity.”Do you not see that lining up against this statement for whatever
technical mumbo-jumbo reason is suicidal? But forget funding for a
moment. Think about what happens when Congress gets involved in
regulating this field, and guys in jackboots come knocking. Is it
smarter to have publicly stated ‘Friendly AI is bunk’ or to have said
‘It’s the only kind AI worth building’?Please, everyone working on a real AGI project that might hasten a
Singularity, you must learn this lesson from Yudkowsky. He’s not wrong.Keith
The point of this message is that explicitly including a well-thought-out Friendliness strategy in an AGI project is likely to garner more approval from the higher-ups. Not to mention minimizing the risk of the human race being replaced by indifferent machine intelligence.
Is it smarter to have publicly stated ‘Friendly AI is bunk’ or to have said ‘It’s the only kind AI worth building’?
Michael, what you’re reposting here is a complete misrepresention of my position. I have never said that building an AI that tries to wipe out humanity is a good idea. Just read the first few sentences of the post in my blog that this is referring to, I state the exact opposite.
What I have done is to attack the work on friendly AI as being inadequate to ensure safe AI, I have not attacked the motivation for doing this work.
Sadly, the response to my criticisms by some people have been to suggest that “guys in jackboots” will “come knocking” in order to get me. And you’re reposted that threat here.
I’ve read your post of course, and obviously it doesn’t say that you want to create an AI that wipes out humanity… but you do say:
One of the things we discussed was friendly AI, that is, the idea that you can build AIs in such a way that they will not do nasty things — like killing off the human race. I think the idea is an impossible dream
You think it’s impossible to build Friendly AI. That’s fine… Keith is simply disagreeing with you. He’s not attacking you in a nasty way, just disagreeing. His point is that either we should either build reliably Friendly AI, or no AI at all.
The above isn’t a threat at all, it’s a post written by Keith Elis about why Friendly AI is the only AI we’d ever want, besides being a wise marketing position for attracting funds.
I see the “guys in jackboots” thing as entirely metaphorical at this point, but potentially an issue in a decade or more. Yes, the government might not like the idea of researchers who don’t believe in Friendliness, regardless of any technical justifications. Keith is saying you’re WRONG about Friendly AI, but not threatening you in the least. No one cares enough about AGI at this point to send hit men after anyone, and surmising to that effect is just silly. In the future… we don’t know!
Keith has a great point – there may be regulation against AI in the future, whether we like it or not. Just because it makes you nervous doesn’t mean 1) it won’t happen anyway, or 2) we should turn a blind eye to the possibility today.
By the way, the list archives are publicly available to anyone who reads them, so it’s not like I’m making something available that wasn’t already available to anyone who wanted it.
There are two ways to address your criticisms: the technical way, which has been done on the list and in your comments, and the political way, which was being done on the list and here. All approaches to AI should be addressed in both contexts. I’m not going to suspend speculation on the political effects of denying the possiblity of Friendly AI just because it makes us squirmish. Of course, the political implications of this belief are on a totally different plane than the technical implications.
You weren’t even mentioned in this blog post until now… the only people who know what Keith is referring to are the people who were already on the list… even if 2,000 people read this post, why would they bother singling you out? They’d have to go google, and seriously, only like 1% of people ever bother to do that with stuff like this.
Suggesting that someone else may be a threat to Y is not the same as directly threatening Y oneself… this is an important distinction, one you kinda seem to be ignoring. When Robin Hanson’s prediction market ideas were shut down by the government, it’s not like his career ended or he was assassinated, either.
Shouldn’t this post be titled: “Keith Elis on the Public Political Future of AGI” ?
Not really, because the key idea is purses, and purse strings, not people in jackboots. Financial differentials are a much bigger deal to AGI than very-unlikely-to-pass bits of AI regulation.
To most people AGI, if they have even heard of it at all, is just some crackpot idea on the web amongst many others. But as time goes on its quite likely that it will make its way onto the political agenda. Politicans are only interested in two things: wealth and power. If AGI technology can help them to increase either of those factors then you can bet that money from the public purse will come raining down upon the researchers or companies involved.
Although I greatly disslike the prospect, I think regulations governing what types of AGI are considered permissable are inevitable, just as there are regulations for industrial robots or IT systems at the present time. There will be a period of time, perhaps a few decades in duration, where AGIs are mostly of the sub human level intelligence and can be controlled in a relatively straightforward manner using conventional IT infrastructure. However, I think such systems will eventually grow beyond the control and understanding of humans – like the weather.
I have heard the arguments about building “friendly” AGIs. However, I think it will be exceedingly difficult, maybe even impossible, to be able to formally prove that some system capable of general purpose learning will always make a human-friendly decision under all possible circumstances. This kind of proof is possible for simpler systems which do not learn, such as those which control space vehicles or airliners, but I remain to be convinced that the same methods could be applied to any non-trivial AGI.
Michael & Bob: We are, after all, talking ultimately about constructing an artificial/artifactual *conscience*, with full (“Kantian”, if you will) *autonomy*. This may indeed prove difficult, if not in-principle impossible. I’m more-or-less agnostic, but as to the latter “in-principle” impossibility, I must say that I’m mildly dubious. Being ultimately a substrate-neutralist (more-or-less), I’ve no objection to it on (“substance”) ontological grounds. BUT, we’re talking not only about a conscience, but also, of course, about cognitive-intellectual autonomy, learning capability, as well as *self-improving/self-augmentation* capability, on the part of a genuine (or “full-fledged”) AGI. Eli, Ben & others are pursuing this as if it (the AGI) will be completely “stand-alone”. And methodologically this is commendable (and perhaps even necessary, for both epistemological and engineering reasons), BUT I think we may actually be on a evolutionary-developmental path such that IA will tend to co-evolve along with A(G)I, and so we may see more-&-more mergence/coalescence between humant “meat minds” and IA/A(G)I systems. Indeed, this is already happening, and is poised to make significant further strides over the next 2-6 yrs or so. (At least two countries, U.S.A. & Australia, are vigorously pursuing, with some collaboration, but also somewhat competitively, this very stuff (“Firefox” tech, and, ultimately “Krell-tech” [so to speak], both in government/private consortiums…) ). So the path for the next 5-30 yrs or so, would seem to be significantly toward *convergence* of IA and A(G)I, hence my (“slashed”) merging of the two terms. This in NO way vitiates the Yudkowskian “research program”, however–not AT ALL. In fact, quite the opposite. I think–perhaps a bit paradoxically–that “methodological stand-alone-ism” (as it were) may ultimately be the ONLY *logical* approach one can take toward AGI. Why? Well, interestingly enough, we are, after all, talking about the ultimate tool (as Irving Good put it yrs ago, “…the last invention humankind need make…”), so we have to treat it and develop it as a stand-alone “device” or “entity”—*even though* it (this “device”) may well evolve/develop way, way beyond us. So the approach Eli is taking is spot-on. But we shouldn’t discount the possibility (if not, in my judgment, likelihood) that what we will *really* be dealing with is (meta)friendly IA/A(G)I protocols & processes. Which means (at least in part), as Pogo would have it, “We has met the enemy and they is us.”
I’m neither Cassandra nor Pollyana about this. I think it’s certainly possible to navigate these waters, as it were, in the yrs to come, but we shall have to be informed & vigilant. I’m certainly pleased that SingInst is poised to be well-viewed by the government weenies…
The shortest path to success will win the game. The shortest of all tried, that is.
Most likely, a dark horse will win the race, also. Maybe, a horse so dark, almost nobody is able to see it yet. To recognize it as a runner and a contender. What does not mean, of course, that every black dog is a dark horse. Most, are not.
Minerva’s Owl only flies in the evening, or very late in the afternoon.
So, try even harder to spot it now, if you wish to be correct for this one!
– Thomas
[...] Original post by Michael Anissimov and software by Elliott Back [...]
I think Michael fairly summarized what I was trying to get across.
My point wasn’t an especially subtle point, but some researchers seem still not to get it, even after painstaking explanation by Yudkowsky and others.
Go ahead and publicly state that Friendly AGI is impossible. In fact publicly state that you haven’t given up on showing how impossible it really is. As your reward, please accept the enclosed bar from experimentation in this research field for the rest of your life and for most of your later re-animation.
The only option available to AGI researchers is to embrace Friendliness. You better be working on Friendly AGI, or eventually, you will be shut down. I’ll see to it, if no one else does, as a man of the people, for the people. I will demonize you with jocular soundbites and nearly content-free press releases. I’ll choose the exact right time during an election year. In the end, at best, you’ll be institutionalized as little more than a mad scientist bent on destroying the world. At worst, you’ll be drawn and quartered.
AGI researchers are smarter than 99.9% of humanity. Of course. And with the irony that often attaches itself to such genius, AGI researchers are truly awful at discerning how average guys like me think. There’s 1 of you for every 999 of me. You’re smart, but you can’t out-talk the evening news. You can’t even come close.
Your nuanced position would take an hour to explain in mathematics, never mind English. Mine is clear, you’re going to kill us all, and you don’t care. Who wins that debate?
Friendly AGI is an impossible dream, and only heroes pursue impossible dreams, and only villains get in their way.
Are we getting it yet?
Keith
Freakmommas Jennifer…
General jennifer freakmommas despite freakmommas jennifer performances broadway seventeen. But muscled when Bunny Luv feere. …
Viagra….
Viagra….
Buy online cheap and fast tramadol….
Tramadol 93. Tramadol online. Buy tramadol. Tramadol withdraw. Tramadol….