From our earliest days as an intelligent species, it has always been more difficult to create than to destroy. From fire to fission, forces of great constructive potential have invariably been used as weapons against innocent people with tragic results. The cumulative losses to individuals, nations – and indeed, the whole human family – can never be fully understood.

Despite a pervasive – and in many ways false – sense of security that came with the end of the Cold War, we are far from being past the threat of technologically facilitated global ruin. The rise of trans-national terrorism may not on the surface seem nearly as dangerous as a full-scale atomic conflict. But the bold acts of hatred performed by those who place no value on their own lives remind us daily of the fact that, among billions, there will always be a few who would destroy civilization itself if they had the capacity to do so.

The day is approaching when this awful power will be all too abundant. Technological progress, that relentless engine that has refined our tools of creation and destruction, is not slowing down. It is accelerating. Technologies that seemed like fantasy a few years ago are now discussed as old news and common knowledge. Scientists recently built a lethal polio virus from scratch by assembling a custom strand of DNA. Nanotechnology – the engineering of materials and machines at the molecular level – is already churning out fibers and coatings incorporated into commercial products, and concept components for devices smaller than human cells are created daily.

Times of tremendous potential are upon us. Where we had only decades ago acquired the ability to observe the most fundamental processes of nature, we are now becoming masters of them. The most intractable diseases and disabilities cannot long stand against the perfect scrutiny and manipulation of genetic engineers. The endless drought of economic scarcity that lingers in so much of the world has no chance of resisting the impending flood of material prosperity unleashed by self-reproducing nanofactories that produce goods of unprecedented quality at negligible cost.

But this flood of prosperity cannot help but flow with a dangerous swiftness equal to the technological progress which propels it. Even ignoring the usual sources of murderous discontent, this radical shift in the quantity and quality of life will probably be sufficient to cause dangerous political upheaval. And, as has always happened with knowledge, the arts of genetic engineering and nanotechnology will inevitably see perversions into killing applications. But this time the danger will be far greater than the threat of nuclear catastrophe – an event entirely survivable by many who might nevertheless wish they hadn’t. A custom-designed plague might be virulent enough to kill everyone, and a swarm of self-replicating nanomachines could swallow the biosphere whole.

Calls to relinquish technologies that could lead to such ends are unrealistic, as these are inextricably linked to positive applications – which greatly outnumber the negative ones. And any attempt to suppress technological progress through means of legislation and enforcement will only mean that when these technologies do inevitably mature, they will be in the hands of those who operate outside the law. Government bodies and committees certainly deserve respect for their ability to mediate disputes and create safety guidelines, but these have never proven capable of ensuring that a given technology is never once used for destructive purposes. And with advanced genetic engineering and nanotechnology, one single misuse may be all it takes to write the epitaph for the human race. We simply cannot rely on traditional organizations and regulations to guide us safely through these turbulent rapids of progress. The current is too swift and the hazards too numerous. And we know from history that somewhere, somehow, there is always a mistake. A human mistake.

Human mistakes are inevitable for the obvious reason that we are in possession of mere human intelligence. We also carry in our genes a myriad of irrational tendencies that do not serve us well, having been so far removed from the ancestral environments where they were useful. We often cherish our primitive instincts and delight in our child-like awe at mysteries “beyond human comprehension,” but these are fertile ground for the kinds of critical failures that could send civilization crashing into the lodestones of oblivion. Our need, then, is for faculties beyond human reasoning, and for minds free of evolutionary liabilities. Whether collectively or in the minds of a select few, we need greater-than-human intelligence to skillfully shoot the rapids of progress and chart the seas of universal prosperity.

Fortunately, the means to achieve greater-than-human intelligence (a milestone called the Singularity by many futurists) are found within the very currents of technology pushing us to this critical juncture. Genetic engineering is one possible answer, but given the relatively long time it takes for a human baby to mature into an adult, this approach would probably not be timely enough even if there were no ethical questions to consider. Augmenting human intelligence by connecting brains directly to powerful computers is another option, but this may not do anything to reduce the likelihood of rash, biological mistakes being made, and may actually amplify their damage. At present, the only conceivable way to promptly give rise to greater-than-human intelligence free of the most significant human failings is through the creation of Artificial General Intelligence (AGI). (The “General” is sometimes added by researchers to distinguish it from the narrowly specialized programs that are often claimed, for marketing reasons, to possess Artificial Intelligence (AI).) Computer technology has matured to the point where most AI researchers feel that an AGI could exist on today’s equipment, given the right design.

But of course the “right” design has not yet been developed, and will not be without determined effort. Any sufficiently intelligent AGI will be able to assist in the design of its own successors, making subsequent leaps in intelligence easier. But the initial design must do more than “merely” think in ways that match or exceed human capability. It must empathize with and care about the problems of its human comrades – a trait called “Friendliness” by some researchers. An AGI lacking this compassion would be as dangerous as any other technological nightmare. And since computer technology is so rapidly increasing in power and decreasing in cost, a time will come when rogue nations or sociopaths could create an unsafe AI – with consequences as potentially catastrophic as the misuse of nanotechnology or genetic engineering. Unless, that is, we already have greater intelligence on our side helping to discover ways to prevent such disasters.

The fate of humanity thus hinges on this question: When will we create greater-than-human intelligence that cares about our problems? There is every reason to act now. Without greater intelligence we are doomed to make human mistakes regarding forces so powerful that there may be no second chances. But, with the assistance of Friendly AI, we will have an extraordinary new capacity to not only safeguard our continued existence, but to meet every other challenge we currently face – or may face.

There is no greater or more responsible use for discretionary resources today than the advancement of this effort. Whether it be a few pennies, a few million dollars, or years of volunteer service, investments in Friendly AI will go further to improve the human condition than donations to any other charity or research project. After all, there are few causes that would not benefit from an infusion of Friendly superintelligence. But, more importantly, if we do not safely navigate the rapids of progress we will not be around to worry about disease, poverty or global warming. This is one swift ride that we are all along for, whether we like it or not, and it is up to each of us to help make sure the human family can survive the journey and come out on top.

(Read the latest short, Singularity-relevant story by Mitch at SIAI’s new blog.)