My recent interview with Juergen Schmidhuber for h+ magazine was Slashdotted. This led to about 322 links from around the Internet. Check out the various comments if you’re interested in various views of AI and reactions to the content of the piece.
AI 2:46 pm
Prof. Jürgen Schmidhuber. Photo credit: idsia.ch
My wide-ranging interview with Dr. Jürgen Schmidhuber, co-director of the Swiss AI lab IDSIA in Lugano and a professor of Cognitive Robotics at the Tech University Munich, is now online at hplusmagazine.com: “Build an Optimal Scientist, Then Retire”.
Here is the video of Schmidhuber’s lively and informative talk at the Singularity Summit 2009:
Jürgen Schmidhuber at Singularity Summit 2009 - Compression Progress: The Algorithmic Principle Behind Curiosity and Creativity from Michael Anissimov on Vimeo.
AI and SIAI and ethics and friendly ai and robotics 11:14 am
The roboethics article I linked on the 15th subsequently appeared in the Japan Times on the 17th.
AI and friendly ai and risks and videos 3:08 pm
H/t to Joshua Fox for the link.
I have been watching some of Tim’s videos over the last few months, but I definitely haven’t seen them all. This one is nice because it summarizes a poignant feeling of concern.
In this video, he builds a model of AGI development using construction paper and Post-It notes.
First Law: A robot must be made to suffer physical and emotional pain.
Second Law: A robot must be free to turn into an evil robot at will, especially when this contributes to the First Law.
Third Law: A robot must be given no knowledge of its creator except through confusing manuscripts created by other robots, especially insomuch as this contributes to the First or Second Law.
In February 2009, the President of the American Association for Artificial Intelligence, Eric Horvitz, convened a panel on “long-term AI futures” which explicitly delved into issues around the Singularity and intelligence explosion. Horvitz has told me (and the New York Times) that the reason he convened the panel was not due to personal interest or concern in the issue but in response to the public interest and concern in the issue.
In the New York Times article covering the meeting, Horvitz was quoted as saying, “My sense was that sooner or later we would have to make some sort of statement or assessment, given the rising voice of the technorati and people very concerned about the rise of intelligent machines”. In August, they released an interim report that said:
Popular perspectives on the outcomes of AI research include expectation that there will be one or more disruptive outcomes. These include that notion that the research will somehow lead to the advent of utopia or catastrophe. The utopian perspective is perhaps best captured in the writings of Ray Kurzweil and others, who speak of a forthcoming “technological singularity.” At the other end of the spectrum, some people are concerned about the “rise of intelligent machines,” fueled by popular novels and movies, that tell stories of the loss of control of robots. Whether forecasting utopian or catastrophic outcomes, the radical perspectives are frightening to people in that they highlight some form of radical change on the horizon—often founded on a notion of the loss of control of the computational intelligences that we create.
The panel of experts was overall skeptical of the radical views expressed by futurists and science-fiction authors.
To me, this was a disappointing result. The phrasing is also disappointing. It is not just the opinion of “popular perspectives” that AI will “somehow” lead to the advent of utopia or catastrophe. Many academics (including AI researchers) have presented views that AI would be highly disruptive, including Ray Solomonoff, Nick Bostrom, Shane Legg, Matt Mahoney, I.J. Good, Bill Gates, Hans Moravec, Marvin Minsky, and many others. Solomonoff, Moravec, and Minsky have all been leaders in AI for decades, so it seems like a deliberate choice of focus to attribute “radical views” to the public rather than AI experts. It provides the AAAS panel with a comfortable level of removal from the claims, a level of removal they could not easily obtain if they cited Solomonoff, Moravec, and Minsky as the sources of Singularity views.
It is remarkable for the panel to suggest that AI will probably not result in disruptive outcomes — if you can turn a pile of sand into a thinking intelligence in the time it takes you to fabricate a computer chip and transfer files to it, then that wouldn’t be disruptive? In my view, it is the degree of disruption that is up for debate — I don’t take people very seriously if they imply there will be little or no disruption whatsoever.
In wondering why the panel came up with this result, Eliezer Yudkowsky suggested “snap consideration and snap judgment”. However, Steve Rayhawk offered a more detailed analysis, which I will post in its entirety here, with a few formatting changes to ensure successful reposting. The first two sentences are a quote that Rayhawk is responding to. Everything that follows from this point on (except for the last line and the quote) was posted by Steve Rayhawk to Less Wrong.
Roughly, what I expect to happen by default is no modular analysis at all - just snap consideration and snap judgment. I feel little need to explain such.
You, or somebody anyway, could still offer a modular causal model of that snap consideration and snap judgment. For example:
1. What cached models of the planning abilities of future machine intelligences did the academics have available when they made the snap judgment?
1.1 What fraction of the academics are aware of any current published AI architectures which could reliably reason over plans at the level of abstraction of “implement a proxy intelligence”?
1.1.1 What fraction of them have thought carefully about when there might be future practical AI architectures that could do this?
1.1.2 What fraction use a process for answering questions about the category distinctions that will be known in the future, which uses as an unconscious default the category distinctions known in the present?
2. What false claims have been made about AI in the past? What decision rules might academics have learned to use, to protect themselves from losing prestige for being associated with false claims like those?
2.1 How much do those decision rules refer to modular causal analyses of the object of a claim and of the fact that people are making the claim?
2.2 How much do those decision rules refer to intuitions about other peoples’ states of mind and social category memberships?
2.3 How much do those decision rules refer to intuitions about other peoples’ intuitive decision rules?
2.4 Historically, have peoples’ own abilities to do modular causal analyses been good enough to make them reliably safe from losing prestige by being associated with false claims? What fraction of academics have the intuitive impression that their own ability to do analysis isn’t good enough to make them reliably safe from losing prestige by association with a false claim, so that they can only be safe if they use intuitions about the states of mind and social category memberships of a claim’s proponents?
3. Of those AI academics who believe that a machine intelligence could exist which could outmaneuver humans if motivated, how do they think about the possible motivations of a machine intelligence?
3.1 What fraction of them think about AI design in terms of a formalism such as approximating optimal sequential decision theory under a utility function? How easy would it be for them to substitute anthropomorphic intuitions for correct technical predictions?
3.2 What fraction of them think about AI design in terms of intuitively justified decision heuristics? How easy would it be for them to substitute anthropomorphic intuitions for correct technical predictions?
3.3 What fraction of them understand enough evolutionary psychology and/or cognitive psychology to recognize moral evaluations as algorithmically caused, so that they can reject the default intuitive explanation of the cause of moral evaluations, which seems to be: “there are intrinsic moral qualities attached to objects in the world, and when any intelligent agent apprehends an object with a moral quality, the action of the moral quality on the agent’s intelligence is to cause the agent to experience a moral evaluation”?
3.3.1 What combination of specializations in AI, moral philosophy, and cognitive psychology would an academic need to have, to be an “expert” whose disagreements about the material causes and implementation of moral evaluations were significant?
4. On the question of takeoff speeds, what fraction of the AI academics have a good enough intuitive understanding of decision theory to see that a point estimate or default scenario should not be substituted for a marginal posterior distribution, even in a situation where it would be socially costly in the default scenario to take actions which prevent large losses in one tail of the distribution?
4.1 What fraction recognized that they had a prior belief distribution over possible takeoff speeds at all?
4.2 What fraction understood that, regarding a variable which is underconstrained by evidence, “other people would disapprove of my belief distribution about this variable” is not an indicator for “my belief distribution about this variable puts mass in the wrong places”, except insofar as there is some causal reason to expect that disapproval would be somehow correlated with falsehood?
5 What other popular concerns have academics historically needed to dismiss? What decision rules have they learned to decide whether they need to dismiss a current popular concern?
5.1 After they make a decision to dismiss a popular concern, what kinds of causal explanations of the existence of that concern do they make reference to, when arguing to other people that they should agree with the decision?
5.2 How much do the true decision rules depend on those causal explanations?
5.3 How much do the decision rules depend on intuitions about the concerned peoples’ states of mind and social category memberships?
5.4 How much do the causal explanations use concepts which are implicitly defined by reference to hidden intuitions about states of mind and social category memberships?
5.4.1 Can these intuitively defined concepts carry the full weight of the causal explanations they are used to support, or does their power to cause agreement come from their ability to activate social intuitions?
6. Which people are the AI academics aware of, who have argued that intelligence explosion is a concern? What social categories do they intuit those people to be members of? What arguments are they aware of? What states of mind do they intuit those arguments to be indicators of (e.g. as in intuitively computed separating equilibria)?
6.1 What people and arguments did the AI academics think the other AI academics were thinking of? If only a few of the academics were thinking of people and arguments who they intuited to come from credible social categories and rational states of mind, would they have been able to communicate this to the others?
7. When the AI academics made the decision to dismiss concern about an intelligence explosion, what kinds of causal explanations of the existence of that concern did they intuitively expect that they would be able make reference to, if they later had to argue to other people that they should agree with the decision?
It is also possible to model the social process in the panel:
8. Are there factors that might make a joint statement by a panel of AI academics reflect different conclusions than they would have individually reached if they had been outsiders to the AI profession with the same AI expertise?
8.1 One salient consideration would be that agreeing with popular concern about an intelligence explosion would result in their funding being cut. What effects would this have had?
8.1.1 Would it have affected the order in which they became consciously aware of lines of argument that might make an intelligence explosion seem less or more deserving of concern?
8.1.2 Would it have made them associate concern about an intelligence explosion with unpopularity? In doubtful situations, unpopularity of an argument is one cue for its unjustifiability. Would they associate unpopularity with logical unjustifiability, and then lose willingness to support logically justifiable lines of argument that made an intelligence explosion seem deserving of concern, just as if they had felt those lines of argument to be logically unjustifiable, but without any actual unjustifiability?
8.2 There are social norms to justify taking prestige away from people who push a claim that an argument is justifiable while knowing that other prestigious people think the argument to to be a marker of a non-credible social category or state of mind. How would this have affected the discussion?
8.3 If there were panelists who personally thought the intelligence explosion argument was plausible, and they were in the minority, would the authors of the panel’s report mention it?
8.3.1 Would the authors know about it?
8.3.2 If the authors knew about it, would they feel any justification or need to mention those opinions in the report, given that the other panelists may have imposed on the authors an implicit social obligation to not write a report that would “unfairly” associate them with anything they think will cause them to lose prestige?
8.3.3 If panelists in such a minority knew that the report would not mention their opinions, would they feel any need or justification to object, given the existence of that same implicit social obligation?
9. How good are groups of people at making judgments about arguments that unprecedented things will have grave consequences?
9.1 How common is a reflective, causal understanding of the intuitions people use when judging popular concerns and arguments about unprecedented things, of the sort that would be needed to compute conditional probabilities like “Pr( we would decide that concern is not justified | we made our decision according to intuition X ∧ concern was justified )”?
9.2 How common is the ability to communicate the epistemic implications of that understanding in real-time while a discussion is happening, to keep it from going wrong?
A great breakdown, worth thinking carefully about.
AI 6:59 am
In 1985, Ray Solomonoff offered his thoughts on six milestones in AI and the economic and technological growth that might be expected when generally intelligent AI is developed. The paper is called “The Time Scale of Artificial Intelligence: Reflections on Social E ffects”.
Here is the abstract:
Six future milestones in AI are discussed. These range from the development of a very general theory of problem solving to the creation of machines with capacities well beyond those of a single human. Estimates are made for when these milestones will occur, followed by some suggestions for the more e ffective utilization of the extremely rapid technological growth that is expected.
When I read lines like that last sentence, what I see nowadays is “extremely scary technological growth”. Rapid growth is scary when that growth is controlled by systems that may not optimize reality in ways that we explicitly value. (See “The Future of Human Evolution” for an explanation.)
A select milestone:
Milestone C. A critical point in AI development would be a machine that could usefully work on the problem of self-improvement. Newell and Simon were not successful in their attempts to get their “General Problem Solver” to improve it’s own methods of operation. While Lenat’s “Eurisko” has been successful in several problem areas, he has not been able to get it to devise good heuristics for itself. He is, however, optimistic about the progress that has been made and is continuing this work.
Eurisko eventually led to the creation of Cyc, which appears to be of limited use.
It should be noted that AI “self-improvement” should be viewed as a special case of an AI’s general talents for understanding an object, evaluating its purpose, and improving it with respect to that purpose. (Sometimes people make unwarranted distinctions between an AI modifying itself and modifying the world.)
How about some more milestones:
Milestone D. Another milestone will be a computer that can read almost any English text and incorporate most of the material into its data base just as a human does. It would have to store the information in a form that is useful for solving whatever kinds of problems it is normally given.
Since there is an enormous amount of information available in electronic data bases all over the world, a machine with useful access to this information could grow very rapidly in its ability to solve problems and in a real sense in its understanding of the world.Milestone E will be a machine that has a general problem solving capacity near that of a human, in the areas for which it has been designed — presumably in mathematics, science and industrial applications.
Milestone F will be a machine with a capacity near that of the computer science community.Milestone G will be a machine with a capacity many times that of the computer science community.
Here’s another bit from later on, analyzing the potential impact of Milestone G:
The last 100 years have seen the introduction of special and general relativity, automobiles, airplanes, quantum mechanics, large rockets and space travel, fission power, fusion bombs, lasers, and large digital computers. Any one of these might take a person years to appreciate and understand. Suppose that they had all been presented to mankind in a single year! This is the magnitude of “future shock” that we can expect from our AI expanded scientific community.
Scanning over the paper, it still seems like Solomonoff is thinking of AIs as tools or narrow scientists, rather than general agents with the full range of activity that humans have or beyond. In the end, Solomonoff seems to imply that one of the primary benefits of AI will be to allow us to predict and evaluate the future more effectively. But he points out that we will still have to make ethical choices.
H/t to Shane Legg for writing about the paper.
AI 10:45 pm
Ray Solomonoff, the father of algorithmic probability theory and one of the founding fathers of Artificial Intelligence, died December 7th after a brief illness.
Solomonoff was a pioneer of probabilistic thinking in AI, and in general. It is my own view that the value of probabilistic thinking is the single most important insight about reality that humanity has ever had, and Solomonoff helped add to that great edifice with his idea of Algorithmic Probability.
Solomonoff was the founder of universal inductive inference, which gives a mathematically optimal method of predicting the next bit of sensory information in a sequence based on prior information. (Unfortunately, it is incomputable, though computable approximations have been used throughout the field of AI.) As far as I know, Solomonoff made the first mathematically rigorous attempt at automated sequence prediction.
Solomonoff’s work is being carried forward by theorists such as Marcus Hutter, Juergen Schmidhuber, and Shane Legg, among many others.
Just last week I posted on AIXI, which is essentially a marriage between Solomonoff’s universal inductive inference and decision theory. Inductive inference tells you what is going to happen next, while decision theory tells you what to do next. Put these together and you get a model for AI.
Solomonoff kept publishing and engaging with the AI community right up until his death. It seems very likely that, if and when strong AI is created, the designer will owe a great debt to Solomonoff’s work. Let’s honor his memory by becoming more familiar with his achievements and making sure that his ideas stay alive.
AI 7:10 pm
There are articles at the New York Times and Popular Science. Nothing new or exciting really, but still nice to see coverage.
A webapp that I worked on with Steve Rayhawk, Anna Salamon, Tom McCabe, and Rolf Nelson, during the Singularity Institute for Artificial Intelligence Summer 2008 Research Program, with helpful discussions with a few others, is now in beta and ready for public announcement. It is called The Uncertain Future.
The Uncertain Future represents a new kind of futurism — futurism with heavy-tailed, high-dimension probability distributions. In fact, that’s the name of the paper presented at the European Conference on Computing and Philosophy that unveiled the project: “Changing the frame of AI futurism: From storytelling to heavy-tailed, high-dimensional probability distributions”.
Most futurism is about telling a story — more like marketing than an honest attempt at uncovering the possible range of what the future may hold. Better than creating a single story is scenario building — but this falls short as well. Scenario building is human nature, but it leaves us susceptible to anchoring effects where we overestimate the probability of vivid scenarios. To quote “Cognitive Biases Potentially Affecting Judgment of Global Risks”, page 6:
The conjunction fallacy similarly applies to futurological forecasts. Two independent sets of professional analysts at the Second International Congress on Forecasting were asked to rate, respectively, the probability of “A complete suspension of diplomatic relations between the USA and the Soviet Union, sometime in 1983″ or “A Russian invasion of Poland, and a complete suspension of diplomatic relations between the USA and the Soviet Union, sometime in 1983″. The second set of analysts responded with significantly higher probabilities. (Tversky and Kahneman 1983.)
The conjunction fallacy means that people overestimate the probability of vivid, detailed scenarios even though each additional detail necessarily decreases the probability that the event will occur.s
To combat against the conjunction fallacy and storytelling fallacies in our particular area of futurism, which includes intelligence enhancement, AI, and global catastrophic risk, we created an interactive system that allows the user to input their own probability distributions for different variables potentially associated with the future of AI and humanity, including a probability distribution of how much computing power would required to create human-level AI, a probability distribution for the likelihood of global thermonuclear war in the next century, and many other variables. Our toy model includes variables for the creation of AI, the possible success of intelligence amplification technology, and the potential extinction of the human species by technological mishap before either of these occurs.
Our system is built on the assumption that breaking down a challenging prediction task into its constituent parts can be beneficial because it forces us to think about the task in greater detail and avoid obvious biases associated with specific scenarios we may be anchoring on. Some people may criticize such a view for being excessively reductionist, but many prediction tasks really can be broken down into component pieces. The alternative is making “expert” guesses based on a holistic evaluation of the prediction task, which leaves us open to many well-documented biases.
Here is the opening blurb for the webapp, by Tom McCabe:
The Uncertain Future is a future technology and world-modeling project by the Singularity Institute for Artificial Intelligence. Its goal is to allow those interested in future technology to form their own rigorous, mathematically consistent model of how the development of advanced technologies will affect the evolution of civilization over the next hundred years. To facilitate this, we have gathered data on what experts think is going to happen, in such fields as semiconductor development, biotechnology, global security, Artificial Intelligence and neuroscience. We invite you, the user, to read about the opinions of these experts, and then come to your own conclusion about the likely destiny of mankind.
Interested? It’s not perfect, but we think that our system might be a seed for looking at futurism in a different way — providing an alternative to storytelling and scenario building. This sort of “probabilistic futurism” encourages would-be seers to widen their confidence bounds when confronted with uncertainty, instead of irrationally making overconfident guesses to seem like “experts”. The particular issues we focus on are controversial — human-equivalent AI, biotechnology used to select gametes with genes associated with intelligence, the probability of planet-ending catastrophe — but we chose these issues specifically because there is disagreement about what degree of uncertainty is warranted from our present position is evaluating these scenarios.
We visualize this tool being used among futurists to specify their quantitative background assumptions regarding the technologies discussed. This might be used to clear aside straw men and zoom in on the core disagreements. It might also be used to evaluate the degree to which respective futurists have considered the technological prerequisites and other assumptions underlying their scenarios.
Go ahead and try the quiz now. Take it slowly, thinking carefully about each question. Scroll down to see predictions from experts, and, where applicable, you can click a button to load a probability distribution that I estimated to be roughly associated with the quote we provided. After taking a look at what the experts say, think about your own position on the issues, and input a probability distribution accordingly.
If you like the system or find it useful, be sure to post a link to it on Facebook or suggest it to your friends. The system still has quite a few bugs; we used Java applets for the probability distributions, and designed it so that the Java applet makes calls to the surrounding HTML which may fail on some combinations of OS and browser. If you use a Mac, you should use Safari, and if you use Linux/Windows, use Opera or Firefox.
AI and nanotechnology 4:56 am
2009 saw a lot of mainstreaming of “transhumanist” ideas, foci, and emphases. As I recently pointed out, Foreign Policy magazine gave this phenomenon a nod by including two transhumanists on their list of 100 global thinkers.
I am particularly interested in any possible mainstreaming of AGI and Friendly AI ideas, for obvious reasons. These ideas are not mainstreaming as fast as “wow-tech” like life-extension or cybernetics, so watching for it is even more challenging and interesting. That’s why this ad on the ScienceBlogs network caught my eye:
It links to Collective Imagination, a relatively new blog on the ScienceBlogs network with an about page that doesn’t mention AI at all. But, click the ad and you go to their front page, which currently is all about AI. On November 19th, their head blogger, Greg Laden, bought into the IBM “cat brain” deliberately deceptive news item, but then did a double-take a week later. What is interesting about his double-take is that he takes the time to point out some ignorant phrasing by IEEE Spectrum blogger Sally Adee in her coverage of the controversy. She said “There are as many theories of mind as there are researchers working on it, and in some cases there is a real grudge match between the theorists.” Greg Laden commented:
I would like to point out that the term “Theory of Mind” is used incorrectly in the above quote. To me, this misuse of the term indicates a degree in pop psychology, as one might be exposed to the phrase but not know what it is, as has apparently happened here.
This is a little embarrassing. It would be like a psychologist writing about computer programming and noting that a “hash table” is a good place to put your chopped up corned beef.
It is embarrassing. Kudos to Greg for catching that. Watch out for those Igon Values.
Another, unrelated place where I read about IEEE in the last few days concerned an IEEE blogger having trouble understanding why the molecular nanotechnology community laughs in derision at the word “nanotechnology” being applied to stain-resistant pants. Josh Hall explained why. The same blogger, Dexter Johnson, also recently relayed that the American Chemical Society “touts nanobots as nanotechnology’s big impact” in a new promotional video, which is another way of saying that they’ve been won over by the arguments for the feasibility of MNT. He writes:
The video is fascinating because it manages to move from nanobots and nanofactories to discussions of nanomaterials and buckyballs so seamlessly you would almost think there was no distinction between the two.
From what I gather this Bytesize Science is supposed to be targeting the future chemists of the world by making science fun. I am not sure that incomprehensible goop is really the way to do it, but I’ve never tried to teach children about nanotechnology.
In the post about nanopants, he writes:
I will not argue here (or likely anywhere else) about the feasibility of nanofactories in the visions of the MNT community.
Why not? Maybe because the idea of nanofactories is sometimes considered unscientific?
AI 4:26 pm
Here is the article. It contains coverage of Ed Boyden’s brain-computer interfacing efforts, along with commentary by Dennett, Pinker, Minsky, Gershenfeld, and Boyden. The important paragraph is here:
The new project, launched with an initial $5 million grant and a five-year timetable, is called the Mind Machine Project, or MMP, a loosely bound collaboration of about two dozen professors, researchers, students and postdocs. According to Neil Gershenfeld, one of the leaders of MMP and director of MIT’s Center for Bits and Atoms, one of the project’s goals is to create intelligent machines — “whatever that means.”
You can read up on the Mind Machine Project here.
