The Singularity Institute’s Humorous (Yet Serious) Call for Researchers Sunday, Mar 1 2009
You may have read a lot of job listings, but this one probably tops them all in terms of how demanding it is. This is the job listing for becoming a Research Fellow at the Singularity Institute for Artificial Intelligence. It is written by “Research Fellow Eliezer Yudkowsky, your potential coworker”:
“Suppose a Bayesian decision agent, a classical expected utility maximizer, had the ability to modify her own source code – including the part of herself that chooses how to modify source code. When you plug this dilemma into classical Bayesian decision theory, it barfs on an infinite recursion. You can use classical decision theory to choose between actions, and choose between source code that chooses between actions, but you can’t actually close the loop; classical decision systems can’t quine themselves.
This is one of the many fundamental open problems required to build a recursively self-improving Artificial Intelligence with a stable motivational system. Now, if you’re the person we’re looking for, you can probably look at the above problem and think of a clever ad-hoc solution off the top of your head. So you need to be adaptable, and a fast unlearner, because cleverness is one of many habits of thought you’ll need to unlearn. We’re not looking for an ad-hoc solution. This isn’t about pumping out another paper or finding a quick hack that gets the job done. Too much weight is going to rest on this. Anything we don’t understand has to be solved, not cleverly swept under a rug. I’m not looking for someone who can invent powerful tools, like neural networks or evolutionary programming. I’m looking for someone who can help create new basic foundations. Pretend you’re working in a historical epoch before anyone realized that math could describe the business of “gathering evidence” or “betting on games of chance”, and ask yourself how you’d go about inventing Bayesian probability theory or Bayesian decision theory. The task is to illuminate the underlying structure of cognitive processes that are currently murky and ill-defined. Note that this is a matter of applied math, not math that is beautiful solely for the sake of being beautiful – the math has to describe an AI.
So what does it take to get that job done? Well, for starters, sheer raw fluid intelligence, plain old-fashioned Spearman’s g. You’ll need to know things that aren’t in textbooks and apply skills that aren’t taught in classes. You’ll have to pick things up rapidly, from a few hints, without them being hammered into you. I attended the inaugural symposium of the Redwood Center for Theoretical Neuroscience, and they asked a panel of prestigious experimental neuroscientists what kind of experience they’d most like to see in a hiree. And one said “Neuroscience”, and one said “Electrical engineering”, and then one said, “I’d rather hire a physicist, because they can learn anything,” and the rest all nodded. That’s the indispensable quality we’re looking for, whether it appears in a physicist or not.”
Continue.
Do you have what it takes to take a serious shot at AGI? If so, consider responding to this job listing, as intimidating as it may be.
This job listing is three years old. As I understood, the position was filled by Marcello Herreshoff, who worked with Eliezer for about a year.
This job listing is still wide open, according to SIAI President Michael Vassar. Several research fellows will likely need to be recruited in the next few years.
At least part of this text was written after that, Manuel. The story about affine transforms is a conversation between Eliezer and Marcello.
Although the opening problem about recursivity dates back to 2002 or so.
I bet they’d attract more people if they said, “If you’re smart enough to be useful, and have the requisite skills, we’ll make you financially independent immediately. You’ll get 1 million/year for the rest of your life. How does that sound?” I mean, come on, look at sports, entertainment, supermodels, all get 10-100+M year. If I was that smart, I’d get rich first and then start working on the project.
A wasted comment. SIAI doesn’t have a million/year. SIAI wants to attract people who are devoted to the cause and don’t care about the money. Reading the article, this is about as obvious as the fact that gravity pulls things down. Your aptitude in reading comprehension is zero.
There may be parties who *do* have that kind of money – like the DoD, or governments in general – who will throw it at anybody smart enough without blinking, even regardless of lack of credentials. For all we know, a Manhattan Project may be underway already. The point seems to be that without the money, SIAI run the risk of getting sidelined, and will fail to attract the serious life-long commitment that the project requires (existing researchers might even abandon ship if an undisclosed uber-lucrative gig calls), as it lacks even the most basic rewards that hugely above average intelligence, aptitude, experience and skill bestows upon you in the real world. If you’re a crucial asset for the future of humanity, surely you’re at least worth living like a sports star. I wonder what the nuclear and rocket scientists of past mega projects were paid.
Are they seriously searching for a general solution to the problem? If so, you’d be an idiot to take them up on it. A successful solution would ultimately be worth billions. Why pre-sell it for a marginal wage?
Also, there are several assumptions in this thing that quite possibly are wrong. One of the things humans do is create an imaginary universe that obeys rules of cause and effect as best we know them. We tentatively make competing decisions, run the clock forward and compare the expected results. We understand that one future scenario will probably not be unambiguously better than the other, so we need to create a kind of weighted ranking of the outcome. We ultimately make a decision and when the result isn’t exactly what our model predicts, we modify some of the generalizations that constitute our rules of causality. We do this by attempting to understand why the prediction and result were different. The next time we need to make a decision, the new revised generalization will be put to the test. By repeating this process many, many times we reach the point where our predictions are pretty good and our decision making processes become more about whether we are properly weighing the various aspects of the outcome. For example, that new job will be more work, but the money will be worth it. We take the job and, wow, I don’t have time to enjoy my new found affluence. So I modify my weighting to accomodate what I have learned.
When one deeply contemplates the processes by which humans make decisions, bayesian decision making is not really very applicable. The way people do this is really more like set theory. In other words, 1) You are a woman, so I expect you to behave this way. 2) You are an accountant, so I expect you to behave this way. 3) You are young and single, so I expect you to behave this way. I merge the behaviors of the sets to which you belong and from this I believe I can predict your behavior. We are constantly trying to explain the behavior of those around us (human, animal and inanimate) by modifying the rules of set membership and by putting different weights on membership in each set. In other words, I may decide that the behavior of the young lady above is more influenced by her membership in the set of accountants than her membership in the set of young, single people.
Now, there is a very real question as to whether we want artificial intelligence to function like human intelligence. We want it to function well, but perhaps differently. This is an objection I’ve always had to the Touring Test. It assumes that the only way to be usefully intelligent is how humans do it.
Anyway, enough. I imagine that this job would be very frustrating. One, because I don’t think the state of the art is there yet. And two, the Institute seems rather crystallized with regard to what the solution must look like. And I think in the fullness of time, we’ll find out that it will look differently.
Michael Ferguson
Also, FYI, the probability that a person with an IQ over 160 (15 point deviation)belongs to any identifiable elite approaches zero. That is a statistical statement. I’ve known a few who have managed it for a time. I suppose there will be doubt. Google Grady Towers, The Empty Promise for a review of elites. Average IQ ~126, Sigma 6.5. Run the numbers.
Regards,
Michael Ferguson