Specialized vs. General Molecular Assemblers and the Risk of AGI

J. Storrs Hall at the Foresight Institute has responded to my recent post about the challenges of self-replication. Specifically, the line where I refer to the Foresight Institute and the Center for Responsible Nanotechnology:

What is remarkable are those that seem to argue, like Ray Kurzweil, the Foresight Institute, and the Center for Responsible Nanotechnology, that humanity is inherently capable of managing universal self-replicating constructors without a near-certain likelihood of disaster.

Dr. Hall responds:

From this he jumps with very few intervening arguments (”there are terrorists out there”) to a conclusion that we need a benevolent world dictatorship (”singleton”), which might need to be a superhuman self-improving AI. This seems a wildly illogical leap, but surprisingly appears to be almost an article of faith in certain parts of the singularitarian community and Washington, DC. Let us examine the usually unstated assumptions behind it:

A singleton need not be a benevolent world dictatorship — just a “world order in which there is a single decision-making agency at …

Read More

David Gelles’ Article on Singularity University

David Gelles recently wrote up an article on Singularity University. Let’s skip to the good parts.

Rather, a new, pseudo-academic institution called Singularity ­University is going to solve our grand challenges: poverty, hunger, energy scarcity and climate change. Among others. Through a combination of techno-optimism, wide-eyed idealism and belief in the perfectibility of human beings, these well-connected geeks are creating an institution meant to legitimise their most extreme thinking.

Then, we see an out-of-focus image with Bruce and Susan to the right.

Then, Gelles talks to Diamandis, who dishes out some superlativity:

A few days before visiting Ames, I caught up with Diamandis at San Francisco’s Fairmont Hotel. Diamandis was attending the Cleantech Forum, a gathering of entrepreneurs and venture capitalists hoping to cash in on green technology. Diamandis also sees a market here – who doesn’t? – and hopes SU can contribute. Yet he may be a bit more extreme than his fellow forum-goers: technology won’t just solve our energy needs, Diamandis argues, but all the world’s problems. “People think there …

Read More

Following the Flu and Catastrophic Risk in General

The WHO has elevated the pandemic threat level to the second-highest rating. According to CNN this “indicates it fears a pandemic is imminent”.

Also according to CNN, later on Obama will make remarks that the swine flu is “very serious” and the “entire government is taking the utmost precautions”. This is in contrast to recent remarks that said we should have “concern but not alarm”.

I guess one should never have “alarm”, if alarm is defined as irrational emotion, but if “utmost precautions” in the entire government for a “very serious” situation isn’t alarm, what is?

Given the rate at which natural pandemics have historically emerged from the world (about once every 50 years), how could that rate increase when genetic engineering of novel microbes for industrial production purposes becomes commonplace, like Craig Venter wants it to be? Once every 20 years? Every ten? New microbes will be developed, but our immune systems will stay the same. (Unless we develop artificial white blood cells (microbivores) and antibodies, …

Read More

Challenge of Self-Replication Reprise

Just because I ran into it in a random Google search and I like it, here I am reposting some content from a post I made exactly five months ago, “The Challenge of Self-Replication”:

What is remarkable are those that seem to argue, like Ray Kurzweil, the Foresight Institute, and the Center for Responsible Nanotechnology, that humanity is inherently capable of managing universal self-replicating constructors without a near-certain likelihood of disaster. Currently Mumbai is under attack by unidentified terrorists — they are sacrificing their lives to kill, what, 125 people? I can envision a scenario in 2020 or 2025 that is far more destructive and results in the deaths of not hundreds, but millions or even billions of people. There are toxins with an LD50 of one nanogram per kilogram of body weight. A casualty count exceeding World War II could theoretically be achieved with just a single kilogram of toxin and several tonnes of delivery mechanisms. We know that complex robotics can exist on the microscopic scale — microwhip scorpions, …

Read More

SIAI call for Skilled Volunteers and Potential Interns

Over at Less Wrong, Anna Salamon is putting out a call for skilled volunteers and potential interns for SIAI-funded summer projects.

Here’s the introductory paragraph:

Want to increase the odds that humanity correctly navigates whatever risks and promises artificial intelligence may bring? Interested in spending this summer in the SF Bay Area, working on projects and picking up background with similar others, with some possibility of staying on thereafter? Want to work with, and learn with, some of the best thinkers you’ll ever meet? – more specifically, some of the best at synthesizing evidence across a wide range of disciplines, and using it to make incremental progress on problems that are both damn slippery and damn important?

Having worked with this group last summer, I can say: Anna is not kidding! This group is extremely intelligent and well-read, and we discussed a wide range of concepts and issues, including those that had little to nothing to do with SIAI’s artificial intelligence focus. In this group, stellar standardized test scores (like perfect SATs …

Read More

Singularity Institute Overview for Journalists

I have written a Singularity Institute Overview for Journalists, for those of you who are interested. This compiles data from SIAI’s website, Wikipedia, and other sources to give you a general picture of the organization.

This overview is the latest addition to my /pages folder, which also has a page that summarizes major SIAI news appearances and a Technological Singularity Overview for Journalists. Pages on transhumanism and other topics are forthcoming.

Read More

Interview: Novamente’s Parrots And Advanced AI Progression

An interview with Ben Goertzel on the subject of Novamente’s virtual pets is up on Gamasutra, a game developer site. The interviewer is the very same Jeriaska that is behind Future Current. Here’s the intro:

As AI developers were convening in San Francisco for GDC, another artificial intelligence conference was wrapping up in Arlington, Virginia, a short walk from the Pentagon. AGI-09, the second conference on artificial general intelligence, brings together researchers attempting to create learning, reasoning agents with broad, humanlike intelligence.

Organized by Dr. Ben Goertzel, chief science officer of Novamente LLC, the AGI conference series is a motivated effort to steer research back in the direction of the original intents of AI, namely to make a thinking machine.

Goertzel’s plan is to inch up the cognitive ladder by incrementally developing more cleverly adaptive pets in virtual worlds and massively multiplayer online games.

This discussion with the AGI designer focuses on the prospects of introducing general intelligence to non-playable game characters.

The topics addressed include contemporary examples of game AI and what …

Read More

Obama Mentions Artificial Intelligence in Speech to National Academy of Sciences

As part of his stated commitment to boost national funding in research and development to 3% of US GDP, Obama mentioned both AI and advanced prosthetics as research goals:

WASHINGTON (AP) — President Barack Obama on Monday promised a major investment in research and development for scientific innovation, saying the United States has fallen behind others.

“I believe it is not in our character, American character, to follow — but to lead. And it is time for us to lead once again. I am here today to set this goal: we will devote more than 3 percent of our GDP to research and development,” Obama said in a speech at the annual meeting of the National Academy of Sciences.

“We will not just meet but we will exceed the level achieved at the height of the space race,” he said.

Obama said the investments he is proposing would lead to breakthroughs, such as solar cells as cheap as paint and green buildings that produce all the energy they …

Read More

PhysOrg: Robots are narrowing the gap with humans

Singularity-relevant press release on PhysOrg, primarily about the “Robobusiness” conference in Boston:

Robots are gaining on us humans. Thanks to exponential increases in computer power — which is roughly doubling every two years — robots are getting smarter, more capable, more like flesh-and-blood people.

Matching human skills and intelligence, however, is an enormously difficult — perhaps impossible — challenge.

Nevertheless, robots guided by their own computer “brains” now can pick up and peel bananas, land jumbo jets, steer cars through city traffic, search human DNA for cancer genes, play soccer or the violin, find earthquake victims or explore craters on Mars.

At a “Robobusiness” conference in Boston last week, companies demonstrated a robot firefighter, gardener, receptionist, tour guide and security guard.

You name it, a high-tech wizard somewhere is trying to make a robot do it.

A Japanese housekeeping robot can move chairs, sweep the floor, load a tray of dirty dishes in a dishwasher and put dirty clothes in a washing machine.

Intel, the worldwide computer-chip maker, headquartered in Santa Clara, Calif., …

Read More

Singularity 101 with Vernor Vinge

This was in the first issue of H+ magazine, but now it’s featured at the website:

Singularity 101 with Vernor Vinge

My stance on Vinge’s position on the Singularity is that he’s too loose with the concept, and seems to slightly welcome people redefining it as agriculture or something else that has no relationship to the concept of smarter-than-human intelligence.

I actually agree with Vernor when he says he’d be surprised if the Singularity doesn’t happen by 2030. It could very well happen after then, but still, I’d be surprised.

He also seems to take a morally detached view of the Singularity — like, “The Singularity is something that could affect everyone on the planet for the profoundly better or worse, but I prefer to view it as an abstract intellectual concept rather than something that will actually affect people.”

Read More

Contained Biodisaster for Risk Analysis

Chris Phoenix says he found a dangerous area of biotech research, hopefully it is no big deal.

The other day a conversation with John Hunt reminded me of an idea we’ve visited before: that the only way that the world at large will take the biotech risk seriously is if an exceptionally virulent engineered pathogen is released in a controlled space, like a level 4 containment facility, with the intention of killing a specific test species.

This idea sounds sort of bad because it involves killing animals (which is usually a bad idea, as Joshua Greene would say: “boo to killing!”), but perhaps mice would do. If people cared about mice, we wouldn’t have allowed the existence of a billion cats.

More generally, the point here is not the specific idea, but just to come up with ideas, because time is elapsing and we aren’t getting any wiser or more compassionate, just more powerful. Synthetic biology is receiving a tremendous amount of scientific attention and research money with zero oversight. Please: regulate. I have …

Read More

Peter Thiel and Patri Friedman on Their Way Forward

Peter Thiel talks about stuff at Cato Unbound magazine.

I remain committed to the faith of my teenage years: to authentic human freedom as a precondition for the highest good. I stand against confiscatory taxes, totalitarian collectives, and the ideology of the inevitability of the death of every individual. For all these reasons, I still call myself “libertarian.”

Like most immortalist libertarians, Peter wants to connect together immortalism with libertarianism, boosting libertarian transhumanism. In transhumanism, the battle between socialists and libertarians is one of endless “excitement” to old-timers and confusion to journalists trying to report on the movement.

Let me comment that most restrictions on freedom come from finite resources. The development of simple self-replicating factories, fed perhaps by acetylene, water, and the Sun, ought to render irrelevant most of the awfully boring debates between libertarian and social democratic transhumanists.

See my interview with Robert Freitas for more on this angle. Dale Carrico calls this superlative vision the …

Read More