Accelerating Future

In the secret, back-room Singularitarian mailing lists and discussion venues, we often ask: “More publicity good? Or more publicity bad? How much publicity is optimal?”

There’s no question that our cause (building safe seed AI) has more exposure now than ever. While it can be hard, if not impossible, to distinguish references to Singularity a la Kurzweil from Singularity a la I.J. Good, the two concepts are meshed together and people really do get exposure to both, even if they come away thinking that Singularity means “transhumanism” instead of “recursively self-improving superintelligence”. And the people who are really in the know can actually tell the difference. For instance, Kevin Kelly, founding editor of WIRED, recently wrote about our version of the Singularity at his blog, the Technium. When the Intel CTO mentioned the Singularity coming by 2060, he was talking about Kurzweil’s Singularity, so in my mind that doesn’t really count.

The goal is to get ourselves enough exposure to get the funding and talent we need to implement Friendly AI as quickly and safely as possible, and no more. Any additional exposure is a risk, because it increases the chance that someone with a ton of money says, “AGI, that sounds like a great idea! Good thing Isaac Asimov did all the groundwork on that friendliness issue for us, so we can just plow ahead on the intelligence part!” Then, after a successful brute force implementation, the AI develops self-replicating robotics, creates trillions of dummies that meet the definition of “human” based on its training set, and goes about spending the rest of eternity converting the universe into sock puppets and making certain to obey them. (Which is pretty easy, considering that the AI controls both the dummies and the system doing the obeying.)

The answer to the “more publicity?” question depends greatly on how hard one wagers AGI to be, or more appropriately, what your probability distribution over difficulty levels is. The people who wager that AGI is relatively “easy”, as in, requiring about a dozen brilliant programmer-theorists a la Fellowship of the Ring, along with a good ten or twenty million dollars, won’t want our cause to gain much more publicity or exposure. Those who wager AGI is extremely hard, as in requiring thousands of programmer-theorists and billions of dollars, would obviously want as much exposure as possible, as it would be necessary to reach the finish line. I fall somewhere in the middle.

On Overcoming Bias, Eliezer Yudkowsky recently observed how he thought many people in the field of AGI were simply ordinary. In my worldview, this is great. My personal experience with SIAI employees and interns indicates they are anything but ordinary. That means the “good guys” — those who make a huge deal about AI Friendliness and warn that we could all be exterminated if we mess up AGI programming — are doing better than the “bad guys” — those who just want to create AGI because it sounds like an interesting research project and are anticipating nothing more than obedient robots with IQs of 90.

But, in my view, the “good guys” still don’t have enough resources and talent, so we need more exposure. Not exposure to the general public, but targeted exposure to highly educated audiences. In a certain sense, the meme is self-filtering. Our version of the Singularity can’t be boiled down to soundbites easily. It helps to have detailed background knowledge about things like philosophy of mind, reductionism, rationality, the human tendency towards anthropocentrism, Homo economicus, evolutionary psychology, and more. Average members of the general public may stumble upon blogs like this and try to understand what I’m saying, but based on what I’ve seen, they’re likely to seize on some tiny incidental point I made and ignore the bigger picture, thereby stopping the spread of the meme in its tracks. Insofar as it makes reckless drives towards AGI less probable, that’s a good thing.

In the end, I don’t think that a million dollars a year and a dozen supergeniuses is enough. We need more resources, more talent, because the challenge of AGI is huge. It looks like the probability of success (by anyone) before 2015 is quite low, and the good guys have a significant theoretical head-start. I think we can afford (and in fact require) more exposure, until the necessary philanthropists and supergeniuses step forward. A major software project is not cheap, and taking the planning fallacy into account, things are going to take more work than we suspect. But once we reach that threshold — stop! Don’t keep plugging ahead for exposure like a mindless robot. That’s just what we’re trying to avoid, y’know?

And wait — you said there are smart bloggers out there that actually aren’t writing about this stuff?

19 Responses »

In inbox-land, the place where emails happen, I have received a piece of “electronic mail”. This e-mail comes from Peter Voss, Founder & CEO of A2I2, a company formed in December 2001 with the “express goal of developing and commercializing an effective general intelligence software engine”. I’ve been following the company since it came into existence, because hey, making claims about AGI is a big deal. Here’s the email:

“Dear friends of A2I2,

We are nearing the commercialization phase of our project.

In the past many of you have expressed a desire to be involved in some way, and a few of you have helped us in various ways (thank you).

At this stage we could use assistance in four areas:

1) Help us identify a business that could serve as a pilot site.
2) Help us find a high-powered CEO to help with our commercial division.
3) Help us brainstorm various business issues — i.e., provide seasoned business advice.
4) Help us test our technology — no technical skills required!

If you feel that you are willing and able to help us, then send a short email introducing yourself to mail at adaptiveai dot com.

Please note that we require you to sign an NDA (non-disclosure).

Peter Voss and Tas Dienes”

Well, here’s your chance to get involved in an AGI company. I have no idea what they’re cooking up, or what it will do, but you can find more information on the company at their site. Peter Voss has said previously that AGI might be possible within a very short time, just 5-10 years. I’m skeptical, but Voss is no kook, so it’s worth at least considering what he has to say.

3 Responses »

Matt Mahoney has written “A Model for Recursively Self-Improving Programs”.

One of the interesting challenges in self-improving AI is that current decision theory models lack features for helping decision makers change the way they make decisions, or incorporate models of themselves into decision-making.

One Response »

I am working on a project that requires references on the comparative difficulty of neuromorphic AI (human brain-simulating, a la Blue Brain) vs. non-neuromorphic AI (not a simulation of the brain). As the terms used to discuss these issues are not always standard, and are sometimes even made up on the spot, locating references via conventional search is not easy. Do any come to mind?

Papers or books preferred; blog or forum posts can’t really be referenced.

2 Responses »

Protip: clicking on the image takes you to the first three chapters of Jaynes’ book.

4 Responses »

I’m having great trouble finding any citeable work that argues that artificial intelligence is completely impossible. People throw kiwis at AI theory in its current state, or the philosophy of functionalism, but every single argument I can find stops short of outright denunciation.

For instance, Gerald Edelman, winner of the 1972 Nobel Prize in Medicine and coiner of the term “Neural Darwinism”, argues that “AI” is impossible, expelling much hot air on the subject, but then it turns out that he believes, “It seems reasonably feasible that, in the future, once neuroscientists learn much more about consciousness and its mechanism, why not imitate it?”, and remarks “We construct what we call brain-based devices, or BBDs, which I think will be increasingly useful in understanding how the brain works and modeling the brain. But it also may be the beginning of the design of truly intelligent machines.” So that’s not very anti-AI. Edelman was also quoted in John Horgan’s recent anti-Singularity piece in IEEE Spectrum, the “Consciousness Conundrum”, in support of the idea that AI is difficult. But if he thinks AI is so difficult, why is he spending time and money on brain-based devices, which are steps towards AI?

According to his Wikipedia article, Hurbert Dreyfuss, author of “What Computers Can’t Do: the Limits of Artificial Intelligence“, argues “that we cannot know (and never will) be able to understand our own behavior in the same way as we understand objects in, for example, physics or chemistry: that is, by considering ourselves as things whose behaviour can be predicted via ‘objective’, context free scientific laws.” But then the article also states, “he doesn’t believe that AI is fundamentally impossible; only that the current research program is fatally flawed. Instead he argues that to get a device (or devices) with human-like intelligence would require them to have a human-like being in the world, which would require them to have bodies more or less like ours, and social acculturation (i.e. a society) more or less like ours.”

Very confusing, but I’m not done yet. Next comes famous physicist and Hawking-collaborator Roger Penrose and his poorly thought out theories on consciousness. Penrose argues that quantum decoherence in neural macrotubules is essential to our intelligence and consciousness. This was decisively refuted by our friend Max Tegmark in 2000, who calculated that the timescale of neuron firing and excitations in microtubules is slower than the decoherence time by a factor of at least 10,000,000,000. Still, although Penrose fusses about the alleged non-algorithmic nature of intelligence throughout his books on the topic, according to a review by Robin Hanson, “Penrose grants that we may be able to artificially construct conscious intelligence, and “such objects could succeed in actually superseding human beings.” But he thinks “algorithmic computers are doomed to subservience.” Another thinker who objects to the mainline AI philosophy and approach but doesn’t actually believe that AI will never be possible if we aren’t creative enough.

There’s more stuff out there. Paul Churchland says, “Classical AI is unlikely to yield conscious machines; systems that mimic the brain might”. More of the same. Copy a certain type of big-headed ape exactly, and intelligence will pop out, but if you try anything else, you’ll fail. Even Searle, the king of AI criticism, acknowledges that “machines with internal causal powers equivalent to those of brains” could think. I’m not sure precisely what he means by this, but by bothering to say something besides humans, even Searle seems to believe that some form of Artificial Intelligence is possible.

Where are the people saying “AI will never happen” or “only human beings can think”? I can find hundreds of references made by laypeople on various forums, but they generally don’t present coherent arguments, they just throw out their opinions.

If no philosopher, cognitive scientist, or computer scientist is willing to claim in public that true AI is impossible, then isn’t this an important finding in and of itself? If it is, then I totally get the credit.

25 Responses »

One position I have difficulty wrapping my head around is the position held by those who believe that human-level AI is possible but that it would lack the capability to quickly change the world. The reasons for why AI would likely have that capability are frequently cited. To summarize just a few:

1) AI could quickly and easily be copied as many times as is computationally feasible.

2) Running on a flexible substrate, AI could “overclock” their cognitive functions, leading to enhanced intelligence and capability.

3) Though robotics today is still maturing, it will be more sophisticated by the time AI arrives, and with AI’s help, it isn’t unreasonable to assume that AIs will have direct and broad access to the physical world through robotic means.

4) AIs would be able to share thoughts almost instantly, meaning that skills learned by one AI could be transferred to all other AIs very quickly.

5) AIs would be able to quickly and automatically perform tasks considered by humans to be “extremely boring”, but still pragmatically useful.

6) AIs could routinely perform intellectually demanding tasks for just the cost of the computer it runs on, plus electricity.

So, brainstorming the reasons why human-level AI would exist but lack the capability to quickly change the world:

1) Human-level AI might possess human skills and intelligence but lack free will, making them incapable of modifying the world in any real sense.

2) Humans will deliberately prevent AI from doing so.

3) AIs would need to be embodied to do anything, and there currently isn’t enough room on the planet for that many embodied AIs or the infrastructure to support the resources they would consume.

4) I object to the idea of human-level AIs in general, thus when the prospect of such AIs changing the world is brought up, I object to its feasibility, while concealing that I reject the premise outright.

5) Humans are equivalent to the most intelligent entity possible, therefore AIs will never be smarter than humans, and will lack any huge impact. (Sometimes this is phrased as saying that humans and AIs are both Turing complete and will thus have the same capabilities.)

6) AIs will just exist on the virtual layer, and being virtual beings, will always have highly limited access to the physical layer.

Any others I’m missing? If there are any actual papers with people presenting points in this vein, that would be ideal.

22 Responses »