Accelerating Future

I felt this was the best piece of work on AI goals and behavior I saw in 2007. This, of course, is Stephen Omohundro’s “Basic AI Drives”. The abstract is as follows:

“Abstract: One might imagine that AI systems with harmless goals will be harmless. This paper instead shows that intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “drives” that will appear in sufficiently advanced AI systems of any design. We call them drives because they are tendencies which will be present unless explicitly counteracted. We start by showing that goal-seeking systems will have drives to model their own operation and to improve themselves. We then show that self-improving systems will be driven to clarify their goals and represent them as economic utility functions. They will also strive for their actions to approximate rational economic behavior. This will lead almost all systems to protect their utility functions from modification and their utility measurement systems from corruption. We also discuss some exceptional systems which will want to modify their utility functions. We next discuss the drive toward self-protection which causes systems try to prevent themselves from being harmed. Finally we examine drives toward the acquisition of resources and toward their efficient utilization. We end with a discussion of how to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity.”

From Self Aware Systems.

8 Responses »

More than 1.6 billion, or roughly a quarter of the planet’s people are Christians. It is the world’s largest religion. I had some questions about Christianity I thought I would ask any Christians that read this. (Non-Christians can ignore this post if they’d like.)

1. Early Christians believe Christ would return quickly, within 1000 years or sooner. This is called millennialism. When do you believe Christ will return? Why is millennialism today associated with fundamentalism?

2. If God answers our prayers, shouldn’t it be easy to detect his presence by observing greater success in people achieving prayer-communicated desires over other desires? Can we design an experiment to confirm this?

3. Are evil people, like Hitler or other serial killers, evil solely due to their own dispositions, or is there some involvement from Satan? Are drug addictions and/or atheism due to Satan? Is there some test for determining the presence of Satan’s influence?

4. Conversely, are people good solely due to inspiration from God and/or Jesus Christ? Is it possible for an atheist, Jew, Muslim, etc. to have “holiness” even if they explicitly disavow your religion? Can they go to Heaven, even if they openly reject Jesus Christ?

5. I find the doctrine of original sin unfair. I do not feel accountable for what some ancient ancestor did. May I be absolved?

6. Can God speak to us through our prayers? How does he determine which people to speak to? When someone claims they spoke to God, how do we know they are telling the truth?

7. Do you plan to meet God when you go to Heaven? If you have ever had a mental scenario of what might happen, please describe it.

8. Will God ever kill Satan?

9. Will a large gold palace (New Jerusalem) really descend from the sky? If so, will non-Christians be allowed in?

10. Would it really be ideal for the entire planet to become Christian? Would the world be a better place?

Thank you for your answers.

44 Responses »

Later this year, Intel will roll out their next-generation, 45-nanometer Nehalem architecture. In terms of clock rates, this means 3.0 GHz and above, but remember that these numbers cannot be used for straightforward extrapolations of processing power. Essentially, these processors will have significantly higher performance than the last generation.

Semiconductor experts believe we have 12 to 15 years of Moore’s law until chips hit the 16 – 11 nm mark and the conventional silicon-based paradigm becomes obsolete. Obviously, hundreds of millions of investor dollars will be put towards developing post-silicon or three dimensional architectures before then.

The Intel Core 2, a chip released in 2006, operates at between 533 and 1333 million instructions per second. By comparison, the human brain has about 100 billion neurons firing 200 times per second. If we assume that each unique firing event represents a functionally meaningful computational operation (doubtful), then the computing power of the human brain is about 20 trillion operations per second, roughly 20,000 times faster than the Intel Core 2.

If the computing power of a typical computer chip keeps doubling every two years or so, as it has for at least 50 years, then we will have home computers with computing power equivalent to the human brain in 2038. Of course, there is no guarantee that the doubling trend will hold. It could stall, or accelerate beyond what we expect.

Human brain-equivalent computing power is something that humanity has never had access to, but it’s just around the corner, historically speaking. It makes you wonder how humanity might utilize computers with such power. If recent news is any indication, such computers will likely be used to implement artificial intelligence (AI) systems for military purposes.

For a while, these artificial intelligence systems will be too stupid to be any real threat — except to those on the business end of military hardware controlled by them. But eventually, they will be cleverer than human beings, and pose a serious threat to humanity in general. People are already hooking up artificial intelligence systems to deadly weapons.

I’m not worried about artificial intelligences suddenly deciding they dislike being bossed around, then turning on their creators. This is anthropomorphic thinking. An AI programmed to obey its creators will not spontaneously turn on them, because it will lack the self-promoting tendencies of creatures sculpted by Darwinian selection. An AI could be exceptional at working out missile trajectories, analyzing enemy troop movements and intelligence data, but incredibly poor at acknowledging itself as a entity worthy of any special entitlements. Casual thinkers fail to realize that our Darwinian instincts correspond to specific neural structures, incredibly unlikely to arise in artificial minds by sheer chance.

The problem arises not when an AI rebels against its programmers, but when it interprets its own programmed goals in ways the programmers did not intend. There are innumerable conceivable examples, but imagine an AI programmed to destroy people against the government of its home specific country. This AI might end up massacring millions of the country’s own citizens with objections to their own government. Surely, if George W. Bush had a pet AI with control over nuclear facilities with such programming, most Americans would be dead by now.

It is not hard to imagine how such an AI might become a threat to humanity in general. An AI programmed to eliminate violence may come to the conclusion that the best way to reduce violence permanently would be to destroy all humanity, judging the immediate violence to be a necessary compromise for avoiding long-term violence over thousands of years. Given access to its own manufacturing hardware, this scenario is feasible.

Even though the notion of a human-equivalent artificial intelligence may seem fantastic to some, I expect to see it within my own lifetime, and within the lifetime of many of my readers here. I have considered several solutions to the challenge of rogue AI, but before I mention them, do you yourself have any? If so, please describe them in the comments.

Try to avoid mentions of fictional scenarios. Think for yourself: Hollywood is not an appropriate intellectual crutch.

30 Responses »