Accelerating Future

Two Papers You Should Read Wednesday, Aug 13 2008

AI Michael Anissimov 11:21 pm

Some of you may have seen these papers already, as I mention them frequently, but they’ll important enough that I like to re-mention them regularly. They’re “Artificial Intelligence as a Positive and Negative Factor in Global Risk” by Eliezer Yudkowsky and “The Basic AI Drives” by Steve Omohundro. The papers are 42 and 11 pages, respectively. There’s no abstract for the first paper, but here’s the abstract for the second:

“One might imagine that AI systems with harmless goals will be harmless. This paper instead shows that intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways. We identify a number of “drives” that will appear in sufficiently advanced AI systems of any design. We call them drives because they are tendencies which will be present unless explicitly counteracted. We start by showing that goal-seeking systems will have drives to model their own operation and to improve themselves. We then show that self-improving systems will be driven to clarify their goals and represent them as economic utility functions. They will also strive for their actions to approximate rational economic behavior. This will lead almost all systems to protect their utility functions from modification and their utility measurement systems from corruption. We also discuss some exceptional systems which will want to modify their utility functions. We next discuss the drive toward self-protection which causes systems try to prevent themselves from being harmed. Finally we examine drives toward the acquisition of resources and toward their efficient utilization. We end with a discussion of how to incorporate these insights in designing intelligent technology which will lead to a positive future for humanity.”

Feel free to post your reactions here.

4 Responses to “Two Papers You Should Read”

Ryan Says:
August 13th, 2008 at 11:50 pm

You know what would be helpful; a paper that explicitly defines and explains what is “A.I.”; because it seems that there are several definitions being used at the time, sometimes interchangeably.

Some define “A.I.” as a self-aware machine. Capable of having an individual identity and self-will; others define it merely as a sufficiently advanced algorithm based machine. I personally define it, not by how it works, but by what it is able to do or mimic.

Perhaps the idea of ‘intelligence’ at this point is not clear enough; some more time needs to be given to properly define it. After all, what shape A.I. will take in the future is still largely unknown.

Still, a paper that attempts to categorize and define what is now a semi-loose concept might prove very helpful (perhaps one already exists. And if not, maybe you could write it.)

In any case, I found both those papers very interesting, and will pass them on to whoever is interested.

(I particularly find it interesting that an intelligent program will act in ways that the designers of said program did not predict. It dispels the notion that the digital world is completely in our control; it is very much based on rules and logic, which stem directly from reality.)

Matt Grimes Says:
August 14th, 2008 at 8:00 am

Ryan is correct that the definitions are a true stumbling block. I wrote my Masters report on the deployment of Sensor Web technology with Intelligent Software Agents and what was immediately apparent when I began researching AI was the definition dilemma. The concepts are not very standardized and we have a long long ways to go.

I’ve extracted a few relevant paragraphs from my paper:

Sensor Web Technology
The term “sensor web” stands for a network of sensing devices which communicate and exchange information intelligently and autonomously. Sensor webs have to fulfill criteria such as interoperability (combination of different types of sensors), intelligence (autonomous decisions), and scalability (extensibility).

The long-term futuristic vision of sensor web applications can be derived by the following statement by Business Week (Gross, 1999):

“In the next century, planet earth will don an electronic skin. It will use the Internet as a scaffold to support and transmit its sensations. This skin is already being stitched together. It consists of millions of embedded electronic measuring devices: thermostats, pressure gauges, pollution detectors, cameras, microphones, glucose sensors, EKGs, electroencephalographs. These will probe and monitor cities and endangered species, the atmosphere, our ships, highways and fleets of trucks, our conversations, our bodies–even our dreams.”

From this visionary perspective we can imagine how autonomous sensor networks merging various sensed phenomena might influence our everyday’s lives. As this scenario plays out we find ourselves facing a problem of our own devising – how are we to communicate, coordinate, process, and react to this voluminous amount of sensory data? Clearly, as the number of sensors increases to thousands, millions and beyond, we must develop an architecture that provides standard methods to building, maintaining, and understanding sensor networks.

Agents provide a new paradigm for developing software applications that involves delegation of tasks and responsibility to computer systems. Before we expound upon this let’s lay a little groundwork. We’ll start with a couple of fundamental questions…What exactly is an agent and what makes it intelligent?

Carl Hewitt recently remarked that the question what is an agent? is embarrassing for the agent-based computing community in just the same way that the question what is intelligence? is embarrassing for the mainstream AI community. The problem is that although the term is widely used, by many people working in closely related areas, it defies attempts to produce a single universally accepted definition.

Certainly one of the stickier points is the concept of intelligence. Petrie points out that different people assign different meanings to intelligence and states that instead of talking about intelligence we should use autonomy as a way of separating agents from other software. This is one of the reasons that he finds the definition of Franklin and Graesser appealing:
“An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future.” [30]

Sandip provides another perspective – “An intelligent agent is an autonomous, proactive, utility-maximizing entity that has some capabilities and, given user preferences and goals, can autonomously plan and execute actions to achieve those goals effectively and efficiently.” [31]

This paper acknowledges that the science of intelligent agents is still in its infancy; incomplete in its definition, and unfortunately has no standards akin to the OGC effort towards SWE. There is a beginning effort underway called the Foundation of Intelligent Agents (FIPA). FIPA, the standards organization for agents and multi-agent systems was officially accepted by the IEEE as its eleventh standards committee on 8 June 2005. Progress seems to be quite slow.

Nevertheless, we will consider the commonalities as a starting point and focus on the generally agreed upon characteristics that an agent should possess, and make some expectations of what an intelligent agent should be capable of in our context of the Sensor Web.

Kaj Sotala Says:
August 15th, 2008 at 1:43 pm

There was discussion about the AI Drives paper on the AGI list, with some criticism from Richard Loosemore - see this message, for instance.

samantha Says:
August 23rd, 2008 at 10:10 pm

What I find implausible in Steve’s paper is an underlying assumption I consider quite suspect. The assumption is that an AI with the unalterable goal to be the best at X at the heart of all its motivations, is going to somehow attain a broad enough knowledge of all non-X specific things to understand which, if mastered sufficiently, will make it better at X. If we assume for the sake of argument that it does somehow attain this knowledge and seeks such mastery then how could it do so without comparing X to all other possible goals closely related to mastering these other domains? I find it very unbelievable that the AI would deviate enough from X to notice these other fields much less to master them and that if it did it would still be oblivious to all but being better at X tossing aside all other perspectives, goals and values it encountered to master these other abilities.

Two Papers You Should Read Wednesday, Aug 13 2008

4 Responses to “Two Papers You Should Read”

Leave a Reply

Categories:

Search:

Monthly:

RSS Feeds:

Technorati:

Email Michael:

Del.ic.ious:

Support: