Ben Goertzel on the Psychology of AGI Programmers Monday, Mar 31 2008
AI 1:56 pm
On the AGI mailing list, Ben Goertzel, CEO of Novamente, got into a discussion with a businessman who claimed that AGI researchers would be more likely to work towards artificial general intelligence if there were more financial gain involved, and that current AI researchers are only in the business for financial gain, as they’re only human. Ben’s response sheds light on the way that AGI researchers actually think:
“Singularitarian AGI researchers, even if operating largely or partly in the business domain (like myself), value the creation of AGI far more than the obtaining of material profits.
I am very interested in deriving $$ from incremental steps on the path to powerful AGI, because I think this is one of the better methods available for funding AGI R&D work.
But deriving $$ from human-level AGI really is not a big motivator of mine. To me, once human-level AGI is obtained, we have something of dramatically more interest than accumulation of any amount of wealth.
Yes, I assume that if I succeed in creating a human-level AGI, then huge amounts of $$ for research will come my way, along with enough personal $$ to liberate me from needing to manage software development contracts or mop my own floor. That will be very nice. But that’s just not the point.
I’m envisioning a population of cockroaches constantly fighting over crumbs of food on the floor. Then a few of the cockroaches — let’s call them the Cockroach Robot Club — decide to spend their lives focused on creating a superhuman robot which will incidentally allow cockroaches to upload into superhuman form with superhuman intelligence. And the other cockroaches insist that the Cockroach Robot Club’s motivation in doing this must be a desire to get more crumbs of food. After all, just **IMAGINE** how many crumbs of food you’ll be able to get with that superhuman robot on your side!!! Buckets full of crumbs!!! ”
Ben mocks the idea that creating AGI is about money. For him, creating AGI is about finding a pathway to making himself more intelligent, and appreciating the world in entirely novel ways.
Read more about Ben’s company, Novamente, at Novamente.net. The site links to articles that mention Novamente at BBC.com and ComputerWorld.com.

March 31st, 2008 at 3:01 pm
If only he could hold out against powerful market forces forever… even if he always wants to.
March 31st, 2008 at 4:52 pm
I’m not sure he’s holding out against market forces here. Valuing money for its own sake seems like a failure of your goal system.
Sure, we want things like nice food, Ferraris, not having to mop our own floor, etc, and money can get all those things. Just wanting money seems like lusting after hammers when you were originally trying to build a house.
I don’t know anyone like that; everyone seems to want money, yes, but they want it *for something*. (Of course, this may speak to my limited experience, but it seems rare that instrumental values trump terminal values.)