Accelerating Future

DARPA gave it to them:

“RICHARDSON, Texas, Oct. 2 — Zyvex Labs today announced the award of a $9.7M program funded by DARPA (Defense Advanced Research Projects Agency) and Texas’ ETF (Emerging Technology Fund). The goal of this effort is to develop a new manufacturing technique that enables “Tip-Based Nanofabrication” to accelerate the transition of nanotechnology from the laboratory to commercial products. Starting with the construction of ‘one-at-a-time’ atomically precise, ‘quantum dot’ nanotech-based products in volume at practical production rates and costs. Harnessing this capability will position the United States and Texas with the fundamental technology to develop next-generation quantum dot applications for military and commercial applications such as advanced communications, metrology, and quantum computers. The spin-off nanomanufacturing capabilities from that early application will result in revolutionary nanotech products in follow-on development.”

Continue.

In my opinion, current advocates of molecular nanotechnology (MNT) aren’t doing enough to address the risks. Christine Peterson of the Foresight Institute advocates an open source physical security model, which is helpful, but should be accompanied by more specific recommendations to form a seed around which further ideas can accrete. The Center for Responsible Nanotechnology has laid out the technological specs of MNT and called for more discussion, but has provided little in the way of concrete recommendations. Ray Kurzweil seems to just think that everything will pretty much automatically turn out fine.

Nanofactories (manufacturing units based on MNT) will need to have extensive, unhackable built-in safeguards in order to be safe. If they can be hacked and these hacked nanofactories cannot be recovered, that could be very bad (significantly worse that terrorists getting weapons-grade uranium). That’s a phrase I’d like MNT advocates to repeat publicly: “terrorists or tyrants getting their hands on unlocked nanofactories would be far worse than weapons-grade uranium”. Unlocked (or poorly regulated) nanofactories would be able to build devices that enrich uranium many times more effectively than current centrifuge technology. That’s somewhat of a problem, unless we plan to gather up all the uranium on the planet and keep it locked up in vaults.

If unhackable nanofactories cannot be built, then to push ahead on the technology would be irresponsible. Mainstream “experts” will be saying this in 5-15 years, but I’m saying it now.

One Response »

It’s important to realize the obvious: that every human problem, every malady, every concern, every evil, is at root simply a suboptimal arrangement of atoms and molecules. If this sounds quasi-spiritual, it’s because it is — for millennia, pre-scientific humans have attributed all ills to various agents — the gods, magicians, and other humans. This is because these ills demand an explanation, and we didn’t have a plausible one, so we made it up. Now, at least in the abstract, we have a concrete, very likely correct answer: suboptimal atomic arrangements.

This realization is neither trivial nor too broad to be useless. If your problems are caused by the gods (that some people sadly still believe in…), then to solve them, you either need to give up, on engage in rituals (prayer, sacrifice, etc.) that have an empirical impact of precisely zero. The ultimate promise is that the gods or God will come at the end of time to make everything better. Unfortunately (?) for us, that will never happen.

The alternative is to slash all spirits from your worldview and model the world as a game board where all the pieces are humans. This too isn’t quite correct, as many who avoid the error of deification of Nature fall right into the trap of the fundamental attribution error, where everything that goes right or wrong becomes some human’s fault or credit. The attribution error is absolutely omnipresent in politics, because invoking it also invokes human political emotions that a leader can easily use to manipulate everyone who has never heard of the error. Since this is practically everyone, it’s politically rational to exploit it to its fullest, and a self-reinforcing feedback loop of error is created. Excuse me, but there are a lot of relevant forces in this world besides deliberate human choice. The shared biases of all human beings come to mind, as do biological realities such as the existence of malaria, and economic realities such as centralized manufacturing.

One sidenote on the notion that “all ills are caused by suboptimal atomic arrangements”. People will have different definitions of what is suboptimal, that is patently obvious. That doesn’t change the fact the subjective personal ills are caused by suboptimal atomic rearrangements, or that there’s a huge space in the center of the Venn diagram of shared humans goals that is specified by certain specific atomic arrangements. Simply because we can’t specify all these arrangements doesn’t mean they aren’t there.

Despite my recognition of a physicalist basis to all problems, I do not advocate a universal convergence towards One True Atomic Pattern or other such absolutist nonsense. I simply wish us to recognize that all shared human problems can be ultimately diagnosed and remedied using the scientific method plus remedial effort: use tests to determine the suboptimal atomic arrangements, then devise engineering solutions to rearrange current arrangements into a more optimal state. This holds true for mental phenomena as well as phenomena in the external world — my brain is “the external world” for others and it is entirely physical. Those who advocate an aphysical basis for consciousness are making the same mystical mistakes that our ancestors have yawn-inducingly made for thousands of years. I am special even if my consciousness has a purely physical basis.

3 Responses »

It really does exist!

3 Responses »

In the secret, back-room Singularitarian mailing lists and discussion venues, we often ask: “More publicity good? Or more publicity bad? How much publicity is optimal?”

There’s no question that our cause (building safe seed AI) has more exposure now than ever. While it can be hard, if not impossible, to distinguish references to Singularity a la Kurzweil from Singularity a la I.J. Good, the two concepts are meshed together and people really do get exposure to both, even if they come away thinking that Singularity means “transhumanism” instead of “recursively self-improving superintelligence”. And the people who are really in the know can actually tell the difference. For instance, Kevin Kelly, founding editor of WIRED, recently wrote about our version of the Singularity at his blog, the Technium. When the Intel CTO mentioned the Singularity coming by 2060, he was talking about Kurzweil’s Singularity, so in my mind that doesn’t really count.

The goal is to get ourselves enough exposure to get the funding and talent we need to implement Friendly AI as quickly and safely as possible, and no more. Any additional exposure is a risk, because it increases the chance that someone with a ton of money says, “AGI, that sounds like a great idea! Good thing Isaac Asimov did all the groundwork on that friendliness issue for us, so we can just plow ahead on the intelligence part!” Then, after a successful brute force implementation, the AI develops self-replicating robotics, creates trillions of dummies that meet the definition of “human” based on its training set, and goes about spending the rest of eternity converting the universe into sock puppets and making certain to obey them. (Which is pretty easy, considering that the AI controls both the dummies and the system doing the obeying.)

The answer to the “more publicity?” question depends greatly on how hard one wagers AGI to be, or more appropriately, what your probability distribution over difficulty levels is. The people who wager that AGI is relatively “easy”, as in, requiring about a dozen brilliant programmer-theorists a la Fellowship of the Ring, along with a good ten or twenty million dollars, won’t want our cause to gain much more publicity or exposure. Those who wager AGI is extremely hard, as in requiring thousands of programmer-theorists and billions of dollars, would obviously want as much exposure as possible, as it would be necessary to reach the finish line. I fall somewhere in the middle.

On Overcoming Bias, Eliezer Yudkowsky recently observed how he thought many people in the field of AGI were simply ordinary. In my worldview, this is great. My personal experience with SIAI employees and interns indicates they are anything but ordinary. That means the “good guys” — those who make a huge deal about AI Friendliness and warn that we could all be exterminated if we mess up AGI programming — are doing better than the “bad guys” — those who just want to create AGI because it sounds like an interesting research project and are anticipating nothing more than obedient robots with IQs of 90.

But, in my view, the “good guys” still don’t have enough resources and talent, so we need more exposure. Not exposure to the general public, but targeted exposure to highly educated audiences. In a certain sense, the meme is self-filtering. Our version of the Singularity can’t be boiled down to soundbites easily. It helps to have detailed background knowledge about things like philosophy of mind, reductionism, rationality, the human tendency towards anthropocentrism, Homo economicus, evolutionary psychology, and more. Average members of the general public may stumble upon blogs like this and try to understand what I’m saying, but based on what I’ve seen, they’re likely to seize on some tiny incidental point I made and ignore the bigger picture, thereby stopping the spread of the meme in its tracks. Insofar as it makes reckless drives towards AGI less probable, that’s a good thing.

In the end, I don’t think that a million dollars a year and a dozen supergeniuses is enough. We need more resources, more talent, because the challenge of AGI is huge. It looks like the probability of success (by anyone) before 2015 is quite low, and the good guys have a significant theoretical head-start. I think we can afford (and in fact require) more exposure, until the necessary philanthropists and supergeniuses step forward. A major software project is not cheap, and taking the planning fallacy into account, things are going to take more work than we suspect. But once we reach that threshold — stop! Don’t keep plugging ahead for exposure like a mindless robot. That’s just what we’re trying to avoid, y’know?

And wait — you said there are smart bloggers out there that actually aren’t writing about this stuff?

19 Responses »

The IEET sometimes replicates blog posts of mine in their articles section. They do this because I said they could. I like it when they do that, because I think what I say is important and should be heard by more people. Otherwise I wouldn’t say it.

You can help me and other IEET fellows by caring about the IEET and what it does. There’s cool stuff there. For instance, IEET executive director James Hughes, who many of you may have heard of, recently did an interview with the Boston Globe on extinction risks. It helps when he and IEET fellows do things like that, because it lowers the chance that we all die. And not dying is awesome.

2 Responses »

Say that the world’s foremost microbiology expert announced that he or she was working on a synthetic virus with the ability to wipe out all human life. Or that a space-capable country, say, the United States, decided that its new mission is to locate the largest possible near-Earth asteroid and alter its trajectory so it impacts the Earth. Or that the world’s foremost expert on machine learning decided that he is tired of humanity and wants to create an AGI that shreds us all to pieces, to put us out of our own misery. (Yes, the technology to make these risks real may not exist now, but imagine 10 or more years down the line — the near future.)

I’d want to be able to say, “So what? Go ahead. You’ll never succeed, we have safeguards against that in place, and they’ve been extensively tested.”

But I can’t, and we’re always in danger, and will continue to be until safeguards are in place.

A key difference between me and many others who think about extinction risk is that I wouldn’t be unpleasantly surprised at a Doomsday Announcement by a leading scientist — surprised as in I’d feel sad about the announcement, but I would have accounted for the possibility in advance. I’d say, “yes, it’s sad you have to be this way, but I knew all along that we needed safeguards in place that would account for this eventuality”.

Of course, the available resources at my disposal are far less than is necessary to put such safeguards in place. However, it could potentially be done for lower cost than one might think. By fueling our watchguard systems with superintelligence rather than human intelligence, they might actually work, rather than failing when the guy watching the cameras goes to take a leak (”human error”). Recursively self-improving superintelligence might be accessible merely through a well-funded seed AI or human intelligence enhancement effort. A few million or even a few thousand dollars can go a long way here.

Say the world’s leading microbiologist had a team of 100 geniuses ready to join him/her in his mission to wipe out us pesky humans, and they already started on the project yesterday. What are you going to do about it? Say “we need to optimistic in life, we can’t always worry about risks, lol”? The problem with that outlook is that it causes you to die.

12 Responses »

In inbox-land, the place where emails happen, I have received a piece of “electronic mail”. This e-mail comes from Peter Voss, Founder & CEO of A2I2, a company formed in December 2001 with the “express goal of developing and commercializing an effective general intelligence software engine”. I’ve been following the company since it came into existence, because hey, making claims about AGI is a big deal. Here’s the email:

“Dear friends of A2I2,

We are nearing the commercialization phase of our project.

In the past many of you have expressed a desire to be involved in some way, and a few of you have helped us in various ways (thank you).

At this stage we could use assistance in four areas:

1) Help us identify a business that could serve as a pilot site.
2) Help us find a high-powered CEO to help with our commercial division.
3) Help us brainstorm various business issues — i.e., provide seasoned business advice.
4) Help us test our technology — no technical skills required!

If you feel that you are willing and able to help us, then send a short email introducing yourself to mail at adaptiveai dot com.

Please note that we require you to sign an NDA (non-disclosure).

Peter Voss and Tas Dienes”

Well, here’s your chance to get involved in an AGI company. I have no idea what they’re cooking up, or what it will do, but you can find more information on the company at their site. Peter Voss has said previously that AGI might be possible within a very short time, just 5-10 years. I’m skeptical, but Voss is no kook, so it’s worth at least considering what he has to say.

3 Responses »