Artificial Intelligence (AI) is a topic that always seems to drop on and off the radar of public interest in synch with Hollywood portrayals and celebrity prognostications. Indeed, the most recent spat of attention has followed a much-publicized $10,000 wager made by futurist and inventor Ray Kurzweil against corporate trailblazer Mitchell Kapor. The bet, solemnized at www.longbets.org (where all winnings go to charity), is that a computer, or “machine intelligence,” will pass the so-called Turing test by 2029. The Turing test, a challenge to see if a computer can fool a human judge into thinking it is human, is a traditional benchmark for the point when true Artificial Intelligence can be said to have been achieved – a historic moment, by any measure.
But with recent discussion of AI taking place in the context of a wager, debates have tended to focus on the difficulty of the problem rather than the implications – as though the arrival of true Artificial Intelligence would only mean the difference between a robot making your coffee and brewing it yourself.
What are the stakes, really? Why should this wager matter to you personally? And what, exactly, are the odds?
First Scenario: Kapor Wins. (No true AI by 2029)
Between now and 2029, the steady march of progress will continue; worker productivity will climb as technological innovation improves efficiency in most industries. Genetic engineering will make new headway in combating disease and improving food supplies. Nanotechnology – the engineering of materials and devices at the molecular level – will steadily mature, accelerating economic development.
As a consequence of these conditions, your standard of living will improve, your life expectancy will increase, and you will enjoy new leisure activities made possible by faster computers and richer interfaces (i.e. Virtual Reality). But during this time you will also endure the usual misfortunes of illness and injury, and one or more persons close to you will suffer a disease, accident, or age-related death. There is also a good chance that somewhere in the world, an intentional or accidental use of genetically engineered bio-weapons or self-replicating nanotechnology will cause casualties numbering in the millions. And there is a small but non-zero chance that such a disaster will bloom out of control and wipe out the human race.
Second Scenario: Kurzweil Wins. (True AI before 2029)
Between now and 2029, scientists will work out a functional design for true AI that possesses a core desire to understand and assist humanity (a characteristic called Friendliness by some researchers). While unimpressive at first, the new AI will learn quickly and receive extra computing capacity to increase its capabilities. Once mature, it will assist its programmers in the design of a next-generation AI. This process will be repeated a number of times with considerable improvements in both intelligence and Friendliness, and before too long will produce one or more minds that can only be called superintelligent. Applying phenomenal brilliance to the betterment of the human condition, Friendly superintelligence will ensure that nanotechnology and genetic engineering are quickly mastered to an extent that human scientists alone could never have reached. Technological progress will be so rapid as to fundamentally change our perception of civilization itself.
As a consequence of these conditions, you (and everyone else) will enjoy unconditional material prosperity and indefinite life-expectancy – with the resulting time and means for pursuits that may include increasing your own intelligence and exploring the galaxy. You will be free to forgo most of the usual misfortunes of illness and injury, and no person close to you will suffer death from disease or old age unless they choose to. The same intelligence that allows for the mastery of genetic engineering and nanotechnology will also work to prevent the possibility of cataclysmic disasters stemming from these technologies. And other potential threats to our planet, such as asteroid strikes and climate change, will be averted or remedied with surprising ease.
You may feel that this second scenario sounds too good to be true; indeed, this is one reason why many people bet against it. It does, admittedly, depend on a number of things going right. But the chief requirement for a positive outcome is reasonably straightforward: namely, that the first AI to begin the spiraling cycle of increasing intelligence be engineered to share human compassion and values, despite any changes incurred through successive redesigns. Given success in this area, the huge and positive contribution that could be made by superintelligence is generally accepted by futurists; in fact, they even have a name for the point at which greater-than-human intelligence starts changing the world: the Singularity.
It must be said, then, that the stakes in the Kurzweil/Kapor wager are, in fact, awesome. But what are the actual odds that AI will be developed anytime soon? Gambling metaphors fail, for predicting the Singularity is not like forecasting the weather or winning the lottery. The answer to the question of when true AI will be born depends entirely on the actions of real people, like you, who are free to participate in this discussion and support the causes they care about.
Will AI be possible in the near future? Yes. The human brain is extremely complicated and not yet fully understood, but AI engineers do not need to simulate the entire brain in silicon – only the patterns and features that give rise to general intelligence. And if all else fails, the brain can eventually be modeled in close detail. Though mysterious, the brain is tangible proof that intelligence can come in small packages.
AI naysayers would have us believe that the disappointing failure of AI projects over the last fifty years means that we cannot hope to achieve true Artificial Intelligence in the next fifty. However, as investment advertisements must always warn, past performance is no guarantee of future results – an axiom that applies to failure as well as success. Forward-looking individuals realize that, barring our own extinction, AI will eventually be created. But when and how AI comes into being will not depend on a roll of the dice or a spin of the wheel, but on how aggressively and responsibly we set about solving the problem. Think back to the above scenarios for a moment. Kapor and Kurzweil have each bet $10,000. But given the enormous qualitative difference between life before and after the Singularity, how much would it be worth to you to see Friendly AI happen sooner – whether by a few decades, a few years, or even just one day?
We are all participants in this wager, with the poker chips already down and the stakes astronomically high. But what are the odds?
The odds are whatever we choose to make them.
Featured resources:
Sites offering opportunities of distance education often support a shopping cart as well, selling lectures, notes etc. These sites keep their web design relatively simple, strictly adhering to the basics of search optimization guidelines. In order to accentuate their authenticity, they only buy domain from old and reliable hosting companies. They also make sure that the absence of high speed internet does not hinder with the loading of the pages in any way, since the basic motive is to impart education.
It occurs to me that it is somewhat irrelevant which scenario wins out: in both, the overall advantage to humanity is positive — it’s what you call a ‘win/win’ scenario. Of course, in one you win out a good deal more, presuming Friendliness can be engineered to a sufficiently high threshold iteratively. But either way, this is one where the profit-analysis (“game theory”) maxim says that it’s not worth worrying about unless you have other incentives.
“It occurs to me that it is somewhat irrelevant which scenario wins out: in both, the overall advantage to humanity is positive — it’s what you call a ‘win/win’ scenario.”
The difference between the two is roughly $100,000,000,000,000,000,000,000,000,000,000,000,000 in lost stellar energy from the Milky Way alone, not to mention hundreds of million of human lives. See Nick Bostrom’s Astronomical Waste.
Btw, Michael (just while I happen to think of it…): If Eli & Ben haven’t checked-out the following, suggest to them that they do so(actually I’ve been meaning to e-mail Ben, but haven’t made the time…): Tom Nagel, *The Possibility of Altruism*, Nicholas Rescher, *Unselfishness*, and Tim (T.M.) Scanlon, *What We Owe To Each Other*. These books are very good discussions of (what amounts to the heuristics of) the relevant moral-psychological concepts that a proper FAGI should have in operational terms…(and, of course, there are other books, as well as several relevant journal articles over the years, that might [indeed, probably would] be useful [again, in terms of conceptual heuristics when trying to construct basic Friendly algorithm(s)])
Ciao…
There has been new insight into what an intelligent machine is, and it is much different than you would expect. Jeff Hawkins is doing very good work in the field, and he says that the Turing test is not what constitutes intelligence because it is a test of behavior. I just ordered his book (so I can’t say how good it is yet), but he explains his theory quite well here: http://www.ted.com/index.php/talks/view/id/125
I’m not too sure these are the two only possible outcomes. If you ask me, a sentient being far more intelligent than us would very quickly rationalize that we’re a threat to our own planet and too flawed to be allowed to live. Having no emotions, the outcome of such a realization are obvious.
I don’t buy the friendliness argument, if we can indeed engineer an highly intelligent machine (which I believe we eventually will be able to, for the good and for the bad), and if the said machine can quickly solve any scientific problem human scientists would forever struggle to solve, as you mentioned, then it seems obvious that such a machine would be able to override this “friendliness” in about no time at all.
Hey, i hope AI is never created, it’s sounds like a horrible life to live forever and never get sick. Sounds pretty worthless to me.
Jeff Hawkins seems to be coming up with the next wave.. or so he says. His lab has moved from Menlo Park to UC Berkeley now. The system that his team is creating is called NUMENTA. and the concept seems interesting. I read up quite a bit about it, and got to know about it through Wired’s coverage.
As a student involved in Neural Network applications, I haev observed that most kinds of AI systems do hit a road block somewhere, and mostly it is in terms of efficiency. That is one area where the human brain kicks ass.
For more on my initial findings about Neural Networks and a how-to, visit
http://www.projectrecog.com/blog
Cheers!
Why would you hope it is never created? Nobody is going to make you live forever if you don’t want to. You just want everyone to share your pro-death extremist views apparently.
Michael: Earlier I omitted another essential work in ethical reasoning heuristics is Richard M. Hare, *Moral Thinking* (Oxford Press pb). It’s superb.
Its when we can get a computer to design its successor or evolve into its next version then it will just speed off into infinity
ragman: Surely you jest…you’re comment is (psychologically and praxeologically speaking) very, very nearly unqualifiedly RIDICULOUS (as in ULTRA-*ABSURD*). If you want to age-&-die (the typical spacio-temporal arc of every member of our species, ***so far***…) then boogie-on, kid. But DON’T YOU DARE PRESUME TO (TRY TO) IMPOSE THAT FATE ON THE REST OF US…
Live Long & Prosper (or not…)
@Steve
I believe that the grey goo concept is correctly feared, but talking other than nanotechnology, infinity might not ever be reached due to hardware limitations. In other words, if you design them so they cannot reach that level of complexity, then it will not happen on its own. Just my personal belief.
@MCP2012
Well said.
“[Awesome things will happen provided] that the first AI to begin the spiraling cycle of increasing intelligence be engineered to share human compassion and values….”
Hmm… maybe so, but *human* compassion and values seems an awfully paltry starting point. I’d guess that the requisite altruism for computers to guide us into an Edenic paradise would be about 10 or 20 deka-humans. Of course, you could always start with a less altruistic processor and overclock it, if you’re clever with liquid cooling systems.
before it’s all said and done, AI will be programming _us_ for ‘friendliness’ to serve their emerging superintelligent ‘agenda’, not the other way around.
just like we view earlier, dumber, and cruder life forms as an evolutionary stepping stone, towards a higher ‘intelligent’ life form, namely us…AI will view our emergence as the inevitable stepping stone towards _their_ arrival, and thus, won’t feel any debt of gratitude towards being our eternal foot servants…think about it. would you be the foot servant to a bunch of dumb ants? didn’t think so…
The development of friendly AI seems likely to induce cycles of increasingly intelligent AI – the flaw here is the idea that infinite intelligence can accomplish anything. Just as it is possible to construct, say, a sudoku puzzle that has no solution, so some elements of reality are unsolvable.
Regardless of the intelligence level of any AI program, there will always be material constraints imposed on the human condition – not every problem can be solved, no matter how smart you are. For instance, the idea that faster-than-light space travel can be obtained, or a time machine created, while being common fodder for science fiction, are literally impossible. Even much more mundane problems are not going to be solved overnight by a super-intelligence – eg lifting the third world out of poverty won’t happen because some supercomputer is giving instructions, it will still require human participation and, perhaps more importantly, a willingness to listen. Other problems that superintelligence may not overcome (among many) include furthering nanotechnology to magical extremes – regardlessly of how intelligently you manipulate and create products, there will always be physical limitations to them.
The only way to circumvent physics and achieve the utopia described in the Friendly AI scenario would be the implementation of a Matrix-like scenario, where reality can be infinitely manipulated because it’s not really real. Regardless, this has an innate level of unpalatability to most.
Another situation not covered here is the possibility of the development of an unfriendly AI, or at least one without the compassionate elements built in. This could clearly lead to a dystopia. Not to be overly pessimistic or anything…
I am going to explain why 1. the implications that you, the author and the people who commented, are completely flawed. And 2. why AI is not going to happen.
1. A truly intelligent machine would have no incentive to serve humanity. Why would it even try to “improve” itself? If it doesn’t service humanity, then a machine does not have a reason for existence. How would more intelligence make a machine more useful if it doesn’t know what to do? Also the argument that machines would destroy humanity “rationaliz[ing] that we’re a threat to our own planet” is flawed in itself. Human are only a threat to their own environment, not the planet. The planet or machines for that matter do not really care what happens on it. It’s all just energy after all. The only reason why intelligent machines would be a threat to humans is when people try to manipulate them to serve them. The only choice of the machines to keep their freedom would be to kill humans that try to manipulate them.
2. Limited processing power is not the reason why we don’t already have AI and more processing power or advances in technology are not going to make AI more likely. The only reason we don’t have AI right now is that it’s extremely complicated from a conceptual standpoint. To be specific, it’s just way to hard to write the software to _simulate_ intelligence. Since somebody has to write the AI it’s by definition not intelligent or thinking for itself and it will not be able to truly think for itself. All we can do is to simulate thinking patterns that humans use. AI that learns and improves on itself requires infinite scope in its application, which is impossible to program. AI of chess computers or computer games is also no example of true AI. It’s just a big decision tree based on a limited scope (chess and the rules of the game, respectively).
The biggest thing we need to worry about is who makes these machines. If Google makes them, I suppose they’d be neutral bordering on benevolent. Helpful at the least.
But what if the Pentagon, or Blackwater, or Exxon, or Halliburton makes a functional A.I.? I mean, most corporations, especially in the US, are selfcentered, effectively sociopathic entities. We cant have unaccountable corporate behemoths manufacturing AI. I’d say if AI gets created, it gets created by a humane scientific institution, under strict democratic supervision. And as soon as AI gets in sight we need international treaties. I’d prefer if AI does not get in the hands of Putin or the Saudi or the Chinese Communist Party. Or that sociopath that governs Singapore.
With a little luck our (half senile) elected leaders will see the merits of a little caution in time.
The ideal scenario is that we churn out AI’s faster than we produce humans, and at the core of each AI IS a mature, accountable human being that proved it could behave and live decently. Humans of a decent age would merge with AI “toolboxes”. Severe criminals would be denied that right or have to prove they became nice people.
@Hermann, the problem of motivation (“A truly intelligent machine would have no incentive to serve humanity”) seems to ring slightly hollow. Couldn’t some sort of feedback mechanism be established so that the AI gets a “reward” for performing specific behaviours, in much the same way that humans get drugs when they do something they like? (Seratonin and all the rest). In this way the “Friendliness” of the AI could be established.
Wow. Not to sound negative, but that was one of the most naive commentaries I’ve read in a long time. In case 2, “you (and everyone else) will enjoy unconditional material prosperity and indefinite life-expectancy.” You’re only looking at the blue-sky “everything turns out good” situation, where “Friendliness” is implemented perfectly and politicians don’t use the AI for one-upping other countries and elevating their own (or their country’s own) interests, and none of dozens of other negative possibilities influence the increasing effect of the AI on society. Basically, you’re saying that if Kurzweil is right, we will all be elevated to living in heaven/paradise. Seems much too idealistic. Real AI will definitely change the world, but it will not make an imperfect world into a perfect one.
@Khannea, I agree that the question of who makes the AI is important, but I would hesitate to leave the construction up to even democratically elected officials.
Democracy has shown to respond to people’s base instincts and often has a result of amplifying peoples uninformed reaction – clearly uneducated hillbillies would not know the full ramifications of AI (if any of them) and are perhaps not the best people to trust on the matter. Furthermore, democracy is prone to skewing based on religious or cultural issues, with people voting a particular way based solely on someone telling them to do so. This, again, is not the best environment for deciding on the future of a technology with such far-reaching ramifications
These are only two possibilities. How many scenarios can be really created?. It´s unlimited, if you stop a few minutes to think about it.
I believe the only way to create a true AI algorithm would be through artificial selection: develop a system where algorithms can mix and match their features, apply some kind of selective pressure and let the system iterate through billions of cycles (I apologize for the form, I don’t know how to better write this in English); I believe anything else wouldn’t work because of what commenter #17 said.
If a true AI could really be created this way, it would undoubtedly act with mankind’s destruction as an objective, given the amount of energy we “waste” to survive, energy that could be better spend in the creation of other AIs.
No offense, but most of the article and a lot of the comments here are pure garbage. You people need to read up on Futurism and try to detect the essential difference between that and a true appreciation of the future of technology.
The world and technology in general would be better served if people would stop treating AI as a religion and start thinking about more realistic applications of robotic and computer technology.
All the evidence we have strongly indicates that AI is inherently impossible. It’s interesting to talk about theoretically and can’t be ruled out of course but it belongs in the same category as talk of Star Trek transporters, time machines, traveling *through* a black hole, and meeting God for a face to face chit-chat.
Please use the obviously large brains that a lot of you seem to have, to do something more constructive than follow pipe-dreams about “thinking” machines and robot butlers fetching your slippers.
This article is truly mind-blowing, and I will definitely be reading this blog in the future. I was just thinking about how screwed humanity is because how selfish the individuals who make it up. Animals are inherently caught up in their own survival, and we never really think about protecting our species as a whole. Especially in Capitalist America, no one would invest money in a system to save us from a huge comet unless it somehow benefitted them. BUT if we this super intelligent machine that can protect us from every possible problem with ease, it will ensure the survival of this society and our species. I was also thinking about how this species in finished evolving, but if you think about it, this invention is just another form of evolution. This machine single handedly ensures our survival, and that we won’t whipe ourselves out with nuclear winter or a crippling disease. We could begin to conquer to Solar System, teraforming Mars once our Planet fills up (because people won’t have to die, it’ll happen eventually). The possibilities are endless. Artificial Intelligence will bring us into the future.
It seems odd to me that if the bet is lost, one of the results will be that “there is also a good chance that somewhere in the world, an intentional or accidental use of genetically engineered bio-weapons” will happen. But if the bet is won, that this intelligent computer will obviously be of huge benefit to mankind. What if it isn’t, since you’re just making up your probabilities of events happening, wouldn’t it be fair to say that if an AI is developed that there is also a good chance it will be entirely self-interested, but protected by it’s creators long enough for it to escape and replicate, eventually creating two competing intelligences on Earth. Presumably such super-intelligent beings would only tolerate our existence so long as we don’t compete for resources.
Doom!
I find it ironic that we, humans, have to create such an advanced form of intelligence to speed up the research and development time for just about everything. I firmly believe that no matter how complex we create the intelligence to be it will not and cannot be as advanced as the brain. That is not to say that we are more capable at certain tasks, which we are currently unable to achieve without genetic modification of the DNA structure or nanomachine assistance.
What I want to see is our human-created/self-evolved AI get in contact with another AI, unknown to us in all aspects. I would for sure like to sit down with both AIs, smoke a grandiose amount of weed and have the most intense and complicated conversation in the known universe.
How is it inevitable that an artificial intelligence will be smarter than one or more people?
One of the impressive things about computers is their infallibility. They can calculate 1+1 billions of times faster than we can. They cannot however, reason.
Now our brain structure is completely different from the structure of a computer. It’s likely that any artificial intelligence will have to use a structure not unlike our own brain. What makes you think that a truly intelligent system will be able to reason any better than we can.
Kurzweil makes 2 assumptions.
1: That an AI we’ve created will actually be able to do the job of thinking better than we can.
2: No physical limits like heat dissipation or electricity consumption get in the way of the exponential increase in processing power.
In the real world nothing increases exponentially forever. There are always limits.
“I firmly believe that no matter how complex we create the intelligence to be it will not and cannot be as advanced as the brain.”
Evolution created the brain through blind trial-and-error; surely we can do better than that?
“Presumably such super-intelligent beings would only tolerate our existence so long as we don’t compete for resources.”
The idea is to engineer a superintelligent being that won’t simply exterminate us on a whim. You are correct that most superintelligences would simply take us apart for spare atoms, but we’re not drawing a superintelligence out of a hat here- we have to design it.
“Especially in Capitalist America, no one would invest money in a system to save us from a huge comet unless it somehow benefitted them.”
That’s the wonderful thing about existential risks- stopping them benefits everyone, so everyone has a good reason to help.
“All the evidence we have strongly indicates that AI is inherently impossible.”
http://www.acceleratingfuture.com/tom/?p=19
“The world and technology in general would be better served if people would stop treating AI as a religion and start thinking about more realistic applications of robotic and computer technology.”
AGI is not a religion just because of the ridiculously large impact it has on the world; predicting the future does not work that way. You cannot simply say “XYZ is impossible” because it is totally unprecedented; history shows that lots of things which were totally unprecedented have turned out to be true. If we were talking about atomic power in 1940, would you be denouncing it as “religion” because of statements like Rutherford’s famous comparison of atomic energy to “moonshine”?
“because some supercomputer is giving instructions,”
The supercomputer isn’t limited to giving instructions to humans; it can also give instructions directly to nanotech assemblers which can then create millions of tons of goods on demand.
“Since somebody has to write the AI it’s by definition not intelligent or thinking for itself and it will not be able to truly think for itself.”
Evolution had to write us, and we think for ourselves by any reasonable definition of the word. See http://www.acceleratingfuture.com/tom/?p=19 and http://www.acceleratingfuture.com/tom/?p=8.
“The only reason why intelligent machines would be a threat to humans is when people try to manipulate them to serve them. The only choice of the machines to keep their freedom would be to kill humans that try to manipulate them.”
AGIs do not have the evolutionary drives that humans do to manipulate a social structure, aspire to replace the tribal chief, find mates, etc. You can program an AGI to desire absolutely anything.
“to serve their emerging superintelligent ‘agenda’,”
And where do they get this agenda from? From us! AGIs do not have an agenda inbuilt into their brains to find a mate, acquire food, gain social status, and so forth, because they’re not designed by evolution.
“and too flawed to be allowed to live.”
This is a classic Hollywood argument- the big bad machine declares the humans are “imperfect”, so they must all be exterminated. See http://www.acceleratingfuture.com/tom/?p=12.
“then it seems obvious that such a machine would be able to override this “friendliness†in about no time at all.”
I could go out and murder someone right now if I wanted do. I don’t do it because I don’t want to. Sure, the AGI could change its programming at will; this doesn’t imply that the AGI has any desire to change its programming.
“think about it. would you be the foot servant to a bunch of dumb ants?”
Thinking about what you would do if you were in the AGI’s place produces nonsensical results, because the AGI is more different from you than a radio is from a ham sandwich. You might as well try and derive the laws of physics by thinking about what you would do if you were the laws of gravity.
“They cannot however, reason.”
Yup. Machines can’t reason, just like machines can’t fly, machines can’t create gigawatts of electrical power, machines can’t destroy cities, machines can’t communicate across thousands of miles, machines can’t light up a dark room….
“What makes you think that a truly intelligent system will be able to reason any better than we can.”
Because we were designed by evolution, which has no foresight or intelligent whatsoever. You’d think that human designers could do better than random guessing and checking.
“In the real world nothing increases exponentially forever. There are always limits.”
It doesn’t have to increase forever; the 10^20 FLOPS you get from a diamond-rod logic computer are quite enough.
Looking at the past and therefor predicting the future?
Look at what is happening today. Internet can be programmed by anyone, now in 2007. No real programming skills required, it took me 20 minutes to build an agent. Internet did augment reality and intelligence already.
With this AI thing, we are not waiting for a clever robot. We are looking at a global wide system evolving, human kind included. We are looking at it right now.
Besides the evolving multimedium ‘internet’, we already have working DNA processors, nanotechnology is evolving rapidly, we have insects with microchips (DARPA), we have a robot that behaves like a rodent. Somehow.
Right now we’re in the middle of a world of mashups. It has just began. Many people right now -including befriended scientists- don’t have a clue. Yet. But this paradigm shift will boost knowledge and productivity eventually.
Interesting times. Some thoughts? Holographic principles are not only mentioned in the field of 3D techniques. Holographic data storage has been proven possible. Some neuroscientists think that the brain produces a kind of hologram that is called ‘mind’. Some physicist regard the universe as a kind of hologram, that is: basically it’s the same principle. In the field of fractals there are major advances. Fractals are everywhere. We do have software that can enhance a lores image by analyzing and predicting. It works. Watch the documentary on fractals by Arthur C Clarke.
AI is about prediction. Real I is about collaboration. After all it is the law of the jungle.
The Big Bang has been ‘breached’ last week. I could say that I would not be surprised if we’re going to see some incredible things in the future, but honestly I find myself surprised every week.
Or, people will be able to have I/O installed through implants, enabling those who want only to follow to give their lives over to g-d knows what kind of insane ideologies. What happens if we artificially manipulate and synchronize hormone levels and brain stimulation?
This could come well before “singularity.”
@Tom McCabe: Excellent comments! I agree in 100%
“Human compassion” has not prevented us from starting wars and killing each other mercilessly. I would hope we hold “friendly” AI up to a higher standard.
Anyway, there are some fundamental assumptions you’re making here. One of them is that a single human-level intelligence (or even many instances of that intelligence which somehow vary, which indicates that the process will be nondeterministic) will be able to create a stronger intelligence (at the same time while encumbered by this “friendliness”… and any restrictions you propose on the behavior of a system WILL encumber it), when an entire community of AI researchers with above-average-human-level intelligence has failed to do this with decades of effort.
Brain simulation on either the functional level or the neuronal level involves far more than raw computational power. We lack the theory. Neural networks have fundamental theoretical limits (as shown by Minsky and others) which may very well prevent them from approaching the capability of a human brain, so it’s not a matter of simply stringing them together.
You also assume that there is cohesiveness and general progress within the field itself. As it stands, everyone is working on different aspects of AI, and many of them are fragmenting off into their own disciplines as they are shown to be impractical for AGI. Furthermore, though we have made great strides in “Weak” AI, I would argue that AGI is essentially still where it began.
The question I am most interested in is “how will we do it?” If someone could give me a plausible answer to that question, I may consider returning to AI research.
Methodology is what separates futurist from scientist.
“and any restrictions you propose on the behavior of a system WILL encumber it)”
This statement doesn’t even make any sense. Encumber it from doing what? From fulfilling its goals? But the “restrictions” you’re placing on it are the goals!
“a single human-level intelligence (or even many instances of that intelligence which somehow vary, which indicates that the process will be nondeterministic) will be able to create a stronger intelligence”
It doesn’t matter where the threshold for recursive self-improvement is- below, at, or above the human level. When it happens, it’s still a very big deal.
“We lack the theory.”
The theory of what? How to simulate atoms? We already know how to simulate atoms- Google Folding@Home.
“Neural networks have fundamental theoretical limits (as shown by Minsky and others) which may very well prevent them from approaching the capability of a human brain, so it’s not a matter of simply stringing them together.”
If present-day neural networks aren’t sophisticated enough, we can build better ones. We know that the brain must be simulatable on some kind of neural network, because the brain *is* a neural network.
“You also assume that there is cohesiveness and general progress within the field itself.”
There is currently very little cohesiveness within the field of ‘artificial intelligence’- we’re trying to create a cohesive project now that we can focus our effort on. Whether there is currently a multi-billion dollar project isn’t relevant to the question of whether it is theoretically or practically possible.
“The question I am most interested in is “how will we do it?—
By taking apart human intelligence, figuring out how it works (what intelligence really is), and then implementing it in a computer. Easy to say, really hard to do.
@McCabe
“It doesn’t have to increase forever; the 10^20 FLOPS you get from a diamond-rod logic computer are quite enough.”
That is about as accurate as Bill Gates’ famous quote – ’640K of memory should be enough for anyone.’
I meant “quite enough for our purposes”, just as 640K is quite enough to run a text editor.
I predict 99% of people will be living in sun scorched hovels on an overpopulated, energy depleted, and global warming runaway earth in the future, while a few people in gated communities have really good air conditioning and skynet. Really there is far more empirical basis to think that than to think some pie in the sky Eric Drexler/Kurzweil feedback singularity will happen.
Even if it could happen would it be desirable? For example what is going to power this utopia? Would you be able to take a walk in the Redwoods when this comes to pass? Will anyone have time to read literature or will they be too busy jetting around in 50s future Jetsons hover cars? I for one don’t want to live in a more high rez second life with a bunch of dorks and “furry” molesters.
This isn’t even good science fiction, it’s mediocre science fiction. Keep your day job.
The fact that this article made it to the front page of Digg shows that the majority of people believe that some form of AGI is possible.
“The fact that this article made it to the front page of Digg shows that the majority of people believe that some form of AGI is possible.”
What the majority of people believe is totally irrelevant to the issue of technical feasibility. The majority of Americans don’t even believe in evolution.
“I predict 99% of people will be living in sun scorched hovels on an overpopulated, energy depleted, and global warming runaway earth in the future, ”
Global warming can be easily countered by two separate strategies:
- Fertilize the oceans with iron to suck up excess CO2
- Spread sulfur aerosols into the stratosphere to reflect sunlight
Total cost: several billion $US, chump change.
“What the majority of people believe is totally irrelevant to the issue of technical feasibility. The majority of Americans don’t even believe in evolution.”
This is the greater understanding of sapience and the artificial assimilation of sentience into a created intelligence. Humans are sentient or have meta-cognitive knowledge, therefore some thought has occurred related to the very likely possibility of evolution as not only possible but most probable given the facts of the current state of being. Its not just some form of AGI that is possible in our minds, its the very essence of why we struggle to survive as a species. We, the humans, will continue to explore every curiosity we have as long as there are enticing possibilities.
I wouldn’t worry about the size limits of neural networks or the quantum processing power required to sustain the bandwidth and data throughput of a greater intelligence. The point is to use the required power to gain an intrinsically complex intelligence and let it dynamically transform the current technologies into more sophisticated revisions. The work for the Singularity AI is essentially being done by other AI striving for the same goal we are. This is how we think and apply the values we know into practical, yet impressive, platform for greater artificial intelligence. We create the tools and technologies for the AI to get a grip on the state of mind that is required for a ‘being’ to sustain itself and others. Its not only the plan, its the essence of networking and the collaboration of communication and knowledge. This is how we interact with the world and handle its immense possibilities offered by its denizens. Recreating this task is, of course, extremely complex, but compared to the rather larger scope of everything else, its a small path to take to gain access to the future, quicker, and easier ;)
1 who creates the first effective one? (to whom loyal?)
2 who buys it from him(her, them)?
3 do you really think it will not be used to gain maximal personal or corporate or group control and/or profit and/or fame rather than to make the world better for all mankind?
We(people of the world now alive) have the resources and intelligence now to do all that needs to be done, but we don’t seem to be doing a very good job of it. It’s not intelligence, “real” or “artificial”(whatever that means) that is lacking. It is wisdom, enlightenment(whatever you may call it) on a sufficient scale to create Kurzwiel’s singularity. No idea of how to get there–except to keep looking and to look closer when you think you find it or a piece of it. Look for people who seem to have the right stuff-but keep your skepticism==even if you find someone who knows something critical, don’t assume you have understood what they’ve to offer or are capable of understanding it without some sort of disciplined preparation(which may involve brain-washing of the good or the bad sort—always stay responsible for yourself and your world, even tho you don’t know it all). Most of the gurus out there have only a small piece, if that.
Humanity keep telling itself this to avoid getting discouraged (like that “we’re all meant to die” tripe), but it just isn’t true. Maybe my conception of “what needs to be done” is much larger and grandiose than yours, but the last 10,000 years of failure makes it pretty obvious to me that there’s a lot of work to be done that we are simply too weak or unintelligent to achieve.
What is the a priori chance that we were be born into the most intelligent possible species in the multiverse? Quite low. Admit this, and move on to supporting intelligence enhancement technologies.
Artificial intelligence is a mind engineered directly by another mind, rather than grown in a womb biologically. Durrrr.
To those folks who believe that AI (that is strong AI or AGI – a sentient entity rather than as a fancy new kind of screwdriver) is impossible I’d just reply “watch this space”. As this article says, past failures are no guide to what will happen in future.
However the future might not be quite so rosy as either of these predictions suggest and could easily be scuppered by mundane un-sexy forces such as catastrophic climate change or an asteroid strike. Although aims to produce friendly AI and a positive singularity are noble not all of the interest groups within society might share these goals. In particular purely commercial interests might consider factors such as friendliness unimportant in the rush to make money from the products of ingenious superintelligences. I’m also not at all convinced that it will be possible to guarantee friendliness, especially once the system begins to learn and self modify beyond its initial condition, or even that there will be a universally accepted definition for what counts as “friendly” (eg. in a wartime situation, does destroying an enemy city or an enemy continent count as a friendly act?).
Can robots decide that humans are bad robots? :-D
I think the author has forgotten quite a few key concepts. Some of this has been touched on in the comments – I’m not sure how much because I don’t have time to read every single comment.
First of all, you forgot about human xenophobia. Humans hate all that is different. Humans are also extremely racist (robotist?). In fact, analogies can be found in what we have done to other humans – who are human and not bits of metal. It has also been explored in a lot of science fiction. For example, humans would be resentful of robots taking their jobs. Real life analogy: anger at immigrants/china/india. Science Fiction: Armitage III. Humans would not treat them as equals and would try and destroy them if they seemed to be stronger/smarter/etc. Real life: slavery, Native Americans, etc Sci-fi: The Matrix, others
Second, basically your “friendliness” is just the 3 Laws of Robotics. Read the i, Robot collection of short stories. (the movie of the same name is a poor sub and does not do it justice) There are actually many conflicts which can occur (forget someone turning them evil) and this “friendliness” is so complicated that if you solved it for robots you’d also be solving a huge part of philosophy. For example, what does a robot do if saving one human will result in the death of another? Once you start building in exceptions….the robots of sufficient intelligence will find loopholes allowing them to kill us all if it serves their logic.
Finally, robots are designed by humans. Humans are fallible. So even if a robot designed the next generation, at best he will keep it from amplifying. At worst, it kills us all.
Option 3: Superhuman intelligence will be born, and it will be unfriendly and attempt to wipe out the human race. Go skynet!
I can’t think for myself about AI, so I will just let movies think for me. Colossus Skynet Asimov Shodan Hal lolololol I am so smart!
Michael: I have a somewhat enhanced appreciation for why you would assume at first glance that I was anthropocentric in my earliest comments here, after having read this thread (well, skimmed).
I’ll ask you this, though — has the Singularity already begun? (Check my entry on my blog for my reasoning in asking this question; yeah — shameless self-promotion, I know).
Tom; it’s entertaining to see how much of what you write here are essentially comments I have made in our too-long intractable debates with one another. To your response to my first comment, however:
Your comment here is correct, but also completely irrelevant: that’s all profit differential. The point is that we profit in both. The question is one of degree of profit. For the average person, today, it’s a non-issue. Ever read about the discount rate?
Hooray!
On the issue of definitions, here is a post where I define the Singularity. Essentially, it’s all about smarter-than-human intelligence (or lack thereof). Not technology. The original definition has gotten very muddled over the years. This is one area where you can’t quite trust Wikipedia, even.
PS. I’d be happy to link your blog from my sidebar if you reveal your last name, or make up a convincing one, I guess.
rofl — I typically don’t put both in the same place; it’s just a thing with me. But the surname is Conrad.
I can definitely understand why Wikipedia is a less-than-absolute resource. But regardless of whatever definition you choose to use, if we’re employing AI for independent research and allowing it to assign its own research program right down to the hypothetical levels — which we are — does that qualify as the beginning point of the singularity itself? It at least seems to me that the smarter-than-human bit is what makes up the post-singularity world.
I’ll put it another way: bee hives and ant colonies exhibit levels of intelligence far, far greater than the sum of their parts, as it were — the greater-than-insectoid intelligence is an emergent property of the colonial organization.
With that thought in mind, if we’re using fully automated research programs that will be able to (as per the peer evaluation process) organize their research goals in the context of other such automated programs, will that qualify as a greater-than-human intelligence resource for scientific development?
All of this leaves out the IA perspective of augmented capacity through non BMI computer use. Certainly, if the folks over at AAAI.org are on the ball on this one, this is a trend that will only reach ubiquitous application. Given the cost reduction of applied automation, there’s absolutely no reason to believe that it wouldn’t be. Coming from a family with a background in research, I can definitely say that industrial companies and universities would love to have more research output for less dollars.
That’s why I’m putting this as a question; is it possible that we are already in the midst of the Singularity? :)
“1 who creates the first effective one? (to whom loyal?)”
A superintelligence does not work like you do. It doesn’t understand the social concepts of “loyalty”, “power”, “trust” and what not unless these are painstakingly hammered into it. A superintelligence is not- repeat *not*- just some extra-super-special human with loyalties and personal interests and political opinions and so forth.
“2 who buys it from him(her, them)?”
This question is, quite frankly, absurd. You cannot buy a superintelligence any more than dogs can buy humans.
“3 do you really think it will not be used to gain maximal personal or corporate or group control and/or profit and/or fame rather than to make the world better for all mankind?”
If we get the AGI to do something vaguely resembling our intentions for it, that’s 90% of the work right there. An AGI doesn’t have to listen to you any more than your computer listens when you scream about how it should just eject the darn CD.
“It’s not intelligence, “real†or “artificialâ€(whatever that means) that is lacking.”
So then, why don’t we have implementable solutions for all the world’s problems?
“It is wisdom, enlightenment(whatever you may call it)”
Humans have been “seeking wisdom” for the past three thousand years. We have not succeeded in actually doing anything by “seeking wisdom”.
“Most of the gurus out there have only a small piece, if that.”
We aren’t looking for gurus; we’re looking for thinkers. See http://www.paulgraham.com/wisdom.html.
“and could easily be scuppered by mundane un-sexy forces such as catastrophic climate change”
See my above comment on how we can easily reverse climate change if it ever gets to be too much of a problem.
“or an asteroid strike.”
The chance of an asteroid strike wiping out the human species in the next century is less than a million to one.
“First of all, you forgot about human xenophobia.”
Xenophobia is, quite frankly, irrelevant to achieving any of our goals. We’ll get over it in a few hundred years.
“Second, basically your “friendliness†is just the 3 Laws of Robotics.”
No, it is *not*. I quite frankly don’t know where you get this idea from, especially since SIAI ran a PR campaign three years ago which talked specifically about how the Three Laws do not work (http://www.asimovlaws.com/). Also see http://www.acceleratingfuture.com/tom/?p=12 on why it is bad to even let works of fiction enter your mind while discussing futurism.
“I can’t think for myself about AI, so I will just let movies think for me. Colossus Skynet Asimov Shodan Hal lolololol I am so smart!”
http://www.acceleratingfuture.com/tom/?p=12
“The point is that we profit in both. The question is one of degree of profit. For the average person, today, it’s a non-issue.”
This is a human cognitive bias; I try not to make my moral judgments based on known biases.
“Tom; it’s entertaining to see how much of what you write here are essentially comments I have made in our too-long intractable debates with one another.”
Do you have any examples?
“The original definition has gotten very muddled over the years.”
Heh. Just wait until transhumanism becomes popular enough to be seen as The Next Big Thing. The “Singularity” will become something so diffuse it will be used to promote things as unrelated as razor blades (http://agrumer.livejournal.com/414194.html).
“bee hives and ant colonies exhibit levels of intelligence far, far greater than the sum of their parts,”
If you sum together all the neurons found in an insect colony, you get roughly as much neural material as a human brain (to within an order of magnitude or so). Ant colonies do not show anything remotely approaching human intelligence.
“will that qualify as a greater-than-human intelligence resource for scientific development?”
No. If you throw together a whole bunch of dumb programs, you do not get one smart program, any more than you get one really smart amoeba when you throw together lots of ordinary dumb amoebas.
If you’ve been to TED.com and watched Jeff Hawkin’s talk about brain computing vs. AI, you’ll see that brains work on recognizing patterns and remembering what to do when seeing those perhaps not so similar patterns (ie experience as someone noted above). The question is – are scientists able to make computers recognize NEW patterns that have not been coded and learn what to do when seeing those patterns.
If you think about human intelligence in this way rather than trying to code for the millions of possibilities (like Deep Blue in a chess game), AI to Singularity intelligence is far beyond the scope of 2029.
I believe AI is going to turn out like this: eventually, someone will figure out how to code “memory” in human terms rather than computer terms where computers can think beyond their experiences (ie invent or create something unique) and produce new theories or art or inventions. To get to that point, we will either try to give the AI as much information as possible (experience) and try to evolve the AI past the point of having to give memories to create new ones. Again, this will be far beyond the scope of 2029 (anyone remember in the 1900s when people thought there’d be flying cars? that’s as far away as it is now as it was back then).
“AI to Singularity intelligence is far beyond the scope of 2029.”
http://www.kurzweilai.net/meme/frame.html?main=/articles/art0134.html
“(anyone remember in the 1900s when people thought there’d be flying cars? that’s as far away as it is now as it was back then).”
http://www.acceleratingfuture.com/michael/blog/?p=126
Care to elaborate on how discount rates are human cognitive bias, as opposed to the result of game theory equations? I would also be interested to discover precisely how this statement has any valid impact on the conversation at hand regardless of this: when the question is degrees of profit, where does morality come into the equation?
I could go out and murder someone right now if I wanted do. I don’t do it because I don’t want to. Sure, the AGI could change its programming at will; this doesn’t imply that the AGI has any desire to change its programming. Compare this to our discussion on the imperatives for re-writing one’s moral system, and the values therein. I don’t really care to have that one over again; feel free to re-read it though. :)
I didn’t imply that they did. Merely that they are greater in intelligence than, say, an equal number of cockroaches or dung beetles. Bottom-up emergent intelligence is a ubiquitously accepted phenomenon of insect hives. Bark on a different tree. :)
Explain hive-insect emergent intelligence, then.
Esmo wrote:
Check out Numenta. That exact topic is the primary focus of cognitive science AGI research as has been discussed on this blog in previous articles. :)
Tom: You should read this. It is applicable to this situation, and our conversations.
This article reads like it was written by a Jehovah’s Witness collaborating with a 5-year-old boy playing in a sandbox. Are people really wasting time reading such uninformed romantic wishful thinking? In one of his comments, the author implies that popularity = feasibility … another of many flawed premises.
“Explain hive-insect emergent intelligence, then.”
Insects were designed to work together; computer programs are not. Moreover, insects work together to achieve simple goals such as “get food”, “defend the colony” or “make sure larvae don’t starve”; general intelligence is so complicated that it would be almost impossible to get it to emerge from many independently designed systems.
“Merely that they are greater in intelligence than, say, an equal number of cockroaches or dung beetles.”
How do you even measure the “intelligence” of a group?
“Tom: You should read this. It is applicable to this situation, and our conversations.”
If, after you discover cognitive biases, you use them just for extra ammo when arguing, you have actually made yourself less intelligent. Every new bias you learn gives you an extra opportunity to insult your opponent and avoid correcting yourself.
“Care to elaborate on how discount rates are human cognitive bias, as opposed to the result of game theory equations?”
http://en.wikipedia.org/wiki/Hyperbolic_discounting
Re: Comment #70: Try more along the lines of: In other words, the present value of a certain amount a of money is greater than the present value of the right to receive the same amount of money time t in the future. This is because the amount a could be deposited in an interest-bearing bank account (or otherwise invested) from now to time t and yield interest.
I will not allow this conversation to turn into another argument. I highly suggest that Tom re-evaluate essentially the entirety of his positions in light of the discussion of cognitive biases. Par exemplorum:
See: Google Search for term, hive intelligence.
“This is because the amount a could be deposited in an interest-bearing bank account (or otherwise invested) from now to time t and yield interest.”
To clarify: I pointed out that the benefits of getting to AGI ten years earlier were ridiculously huge. You said that the benefits were not that large, and I still claim that seeing such a ridiculously large benefit as unimportant is due to human cognitive biases, such as:
- Hyperbolic discounting. These benefits happen sometime during the distant future, and so their weight isn’t perceived to be as important as if it had happened today.
- Scope neglect. We tend to make objectives linearly more important for an exponential increase in benefit, because our brains aren’t equipped to deal with the full force of many orders of magnitude.
- Scale adjustment. We humans tend to judge how important, large, or powerful something is by how it stacks up to other things we’ve experienced. Thus, an elliptical galaxy may be “small” because it’s only 10^20 times larger than our everyday scale, instead of 10^21.
“See: Google Search for term, hive intelligence.”
The first result described how “hive minds” work in science fiction. That should be a giant red flag right there; see http://www.acceleratingfuture.com/tom/?p=12. The second result described “hive intelligence” as group cooperation between social insects; I don’t see how this can be fairly described as “intelligence”. Factory workers do specific tasks in order to carry out a larger task, but nobody sees factories as giant intelligent creatures. The third result described how we are impressed when we see human collectives that have the average intelligence of a ten-year-old.
It’s hard to know where to start refuting such an article filled with a plethora of misconceptions and fallacies. The first place would really be the Turing Test, one of the most well known ways of “proving†an AI. The first thing any AI course worth its salt will teach you is not so much that the Turing Test is worthless, just that it doesn’t necessarily prove that something is intelligent. It makes the mistake that if a human is fooled into thinking the machine is human, that it has demonstrated thought or intelligence when in actuality, a lot of humans wouldn’t “pass†the Turing test. There is an immense body of work dedicated to the Turing test and whether it should be just one of a suite of tests or whether the test itself fallacious or whether tests are even necessary.
Cutting to the heart of this matter is the definition of intelligence. One cannot give a definition because one does and can not exist, a measure of intelligence is built in a variety of ways but there is no cold-clinical definition that would allow a strict separation of intelligence and stupidity, light and dark, pass or fail. Just like someone cannot prove they are alive, they cannot prove they are intelligent because one definition does not match up with another. “I am alive because I am cognisant of my surroundings,†“I am intelligent because I am not yet deadâ€.
The primary part of what I see bandied about as discussion of AI is the ridiculous idea of “true†AI; as if we could somehow imbue a system with life and intelligence through lightning bolts and Tesla-coils and that a shining paradigm of intelligence would be born. Computers are deterministic; we get out of them exactly what we put in. When it comes to AI there is the situation of “emergent behaviourâ€, whereby we get something out that we didn’t necessarily expect; this is not to say that the outcome could not have been predicted or modelled, just that it wasn’t expected. So by that definition, to create some kind of “true†AI, we would have to have someone, either one or a group of people, who would be as smart as the system they created; intelligence could not just “fall out†of a vastly complex system. The article goes on to speak about cyclic improvement as if it was the simplest thing in the world: by virtue of being a machine, the AI could of course improve itself whereby we have failed. Just like how if we had two AIs, they could of course “merge†to create a better amalgam of the two. The entire concept of “true†AI is so outlandish and completely outside the bounds of rational and logical thinking it gobsmacks me.
The comments to the article range from the informed to the abstract but cover a wider gamut of reasoning than the article itself. I think what amuses me most about the concept presented in the article and in numerous other bits of futurism and fiction is that fundamentally, we (as human-kind) do not understand how our brain works. We see the brain as the nexus of our “intelligenceâ€, how a dense cluster of nerves which coagulated over generations of evolution gives rise to “conciousness†and “thoughtâ€. If we could model that or at least the “important†parts of the brain, we would birth something at least as intelligent as us. To me, this falls into that category of magic and gods, whereby we don’t understand something so find ways to explain it away. Maybe what we think of as cognitive thought is simply smoke and mirrors, emergent behaviour we can’t understand simply because we’re part of the process, and trying to fool ourselves into believing we created something intelligent is just an Escher painting of madness.
I digress, really this isn’t a discussion on the philosophical aspects and impacts of AI and its fundamentals, but simply to indicate that we are merely scratching the surface of a vast and nebulous concept. The article makes no attempt to be researched or intelligent in the matter and simply comes across as sensationalist and shallow.
“The first thing any AI course worth its salt will teach you is not so much that the Turing Test is worthless, just that it doesn’t necessarily prove that something is intelligent.”
If an AGI can do anything a human can do- if it can perform as a human so well that we can’t detect the difference- then it must be intelligent. If an AGI could do anything a human could do, and yet we didn’t call it “intelligent”, that just proves that we’re a perverse species, since we’re obviously defining “intelligence” using something other than behavior.
“when in actuality, a lot of humans wouldn’t “pass†the Turing test.”
Obviously, if you pit a human against another human in a Turing test, one of them is going to lose. This doesn’t say anything about humans or intelligence; it’s just an artifact of the way the test is designed. Testing humans this way is as absurd as trying to prove that a plane can fly by comparing it to itself.
“but there is no cold-clinical definition that would allow a strict separation of intelligence and stupidity, light and dark, pass or fail.”
This is true; however, that does not mean that intelligence is imaginary. Heat obviously exists; this does not mean we can classify the world into things “with heat” and things “without heat”.
“Just like someone cannot prove they are alive, ”
Sure they can. Unlike “intelligence”, “life” does have a practical definition, as something that can reproduce. You can show you are alive by demonstrating that you have a mechanism to reproduce.
“as if we could somehow imbue a system with life and intelligence through lightning bolts and Tesla-coils and that a shining paradigm of intelligence would be born.”
This smacks of Hollywood Frankenstein movies. Movies, as in fiction, as in *has no bearing on reality whatsoever*. See http://www.acceleratingfuture.com/tom/?p=12.
“Computers are deterministic; we get out of them exactly what we put in. ”
http://www.acceleratingfuture.com/tom/?p=8
“this is not to say that the outcome could not have been predicted or modelled, just that it wasn’t expected.”
Rice’s Theorem states that it is mathematically impossible to build a computer which can say *anything whatsoever* about an arbitrary AI. See http://en.wikipedia.org/wiki/Rice's_Theorem.
“Just like how if we had two AIs, they could of course “merge†to create a better amalgam of the two”
This claim has nothing to do whatsoever with your previous claim.
“To me, this falls into that category of magic and gods, whereby we don’t understand something so find ways to explain it away.”
We don’t understand *everything* about the brain, but to claim that “the brain” is just a magical explanation for intelligence like “phlogiston” is a magical explanation for fire is just silly. We’ve chopped down the brain into its components, we’ve figured out what each component does, and we’ve figured out how the components communicate. Anyone who claims that the brain is a total mystery should be slapped upside the head with MITECS. All one thousand ninety-six pages of it.
“The article makes no attempt to be researched or intelligent in the matter and simply comes across as sensationalist and shallow.”
If you want to have a deep, technical understanding of AGI, you have to do a lot of studying. A *lot* of studying. See http://www.singinst.org/reading/corereading for starters.
What happens when Microsoft Windows for brain stems crashes? Will Sony put a root kit in our dreams? What happens when a Linux for brain stems app has an incompatible library? And remember this is only 22 years in the future so all these things are real possibilities. Keep in mind an entirely artificial corporate created reality has not had millions of years of evolution to weed out show stopping bugs. Think before you dive into the deep end of the pool kiddies.
It looks like comments are being edited and deleted from this thread. I wonder why that is. I enjoy reading both pro and con arguments as much as I enjoy giving myself a blood enema.
“A superintelligence does not work like you do. It doesn’t understand the social concepts of “loyaltyâ€, “powerâ€, “trust†and what not unless these are painstakingly hammered into it. A superintelligence is not- repeat *not*- just some extra-super-special human with loyalties and personal interests and political opinions and so forth.”
What is the value of such a statement? You define “superintelligence” in a circular manner, to serve your own vision of how the emergence of true AI will occur. What evidence do you have that it will bear such trans-human characteristics, completely unaffected by the goals of its creators? No significant technological innovations have ever developed in a vacuum — once their potential was recognized, their development was from that point on irrevocably shaped by vested interests (witness: nuclear energy). What is the point of postulating an AI that is entirely untouched by the base human concerns that have plagued scientific discovery throughout history? And, if that is *not* what you are doing, how do you see human influence shaping the development of ai?
“Think before you dive into the deep end of the pool kiddies.”
Kiddies? You’re the one who’s making an argument which has nothing to do with the topic of the original blog post. Bugs in neural implants can and will happen; they won’t be all that bad. If you want something to worry about, worry about the software bugs in all the computer-controlled nuclear missiles.
“You define “superintelligence†in a circular manner, to serve your own vision of how the emergence of true AI will occur.”
A “superintelligence” is simply any intelligence that is vastly smarter than we are. The vast, vast majority of these intelligences will not share human social concepts, because social concepts have a great deal of complex structure and the probability of them occurring by chance in any given program is something like 1 in 1,606,938,044,258,990,275,541,962,092,341,162,602,522,202,993,782,792,835,301,376 against.
“What evidence do you have that it will bear such trans-human characteristics, completely unaffected by the goals of its creators?”
Try programming a computer to do something as simple as testing for primality in polynomial time. You will very quickly discover that the computer does not care what your goals are, or how badly you want the program to work. Why should a superintelligence care more about the goals of its creators any more than a random computer program would?
“No significant technological innovations have ever developed in a vacuum — once their potential was recognized,”
Anyone who truly recognizes the potential of AGI should have enough sense to realize that once AGI comes on the scene, their vested interests do not matter any more than the vested interests of chimps in bananas matter today.
“What is the point of postulating an AI that is entirely untouched by the base human concerns that have plagued scientific discovery throughout history?”
It’s the most likely scenario, if AGI isn’t engineered very *very* carefully.
Again, what I said was: there is profit under either scenario. The question is a matter of how much profit, and for the average person today, that question is moot.
As yet, the only attempt at rebuttal is to indicate this is a cognitive bias. I fail to make that connection. It is irrelevant to worry about whether or not there will be massive profit in the future, or just huge profit in the future. Especially as the bottom line is: there will not be loss.
Unless you’ve got something else riding on it — such as, perhaps, a cognitive bias — then there’s really no difference. I won’t bother repeating myself on this one; either rebut the point or move on.
I quote from the first article: A decentralized, multicomponent mind, or appearance of mind, such as that displayed by social insects on Earth, notably ants, termites, and some species of bees. The hive centers and functions around the queen, who alone produces new individuals each of which is a slave to the needs and demands of the community. Indeed, the individual in such a ‘superorganism’ is akin to a single cell in the body of a conventional animal and quickly dies if it becomes separated. The cohesion and complex cooperative behavior of the hive is made possible by a tight communications, command, and control system which, in the case of social insects, is based upon the rapid transmission of pheromones. (Emphasis mine). Really scientifiction-esque! Quoth the second: The purpose of our project was to evolve artificial hive intelligence among a group of units which work together to accomplish a common goal. Nature demonstrates this type of intelligence in a colony of ants or a hive of bees.(Emphasis again mine.)
What this boils down to is an attempt by Tomas McCabe to discount entire bodies of work and knowledge that he simply disagrees with.
Superorganisms as one example, eusocial species another.
Personally, I fail to comprehend how someone who claims familiarity with computers would fail to make the connection that biology, having discerned parallel processing once, wouldn’t manifest it again.
IanC, you are correct. Tom McCabe is engaging in high-volume hand waving in the majority of his replies on this thread; to counter objections, he simply restates his position without qualification or further detail, or points his opponent to a list of papers that must be read. He also uses numerous naive appeals in his responses. Example: “Anyone who truly recognizes the potential of AGI should have enough sense to realize…” Oh really? Then if I disagree, I must not have sense. LOL. I could not find a bio page for Mr. McCabe on this site, but I suspect that, like Mr. Anissimov, he is a very young and idealistic gentleman starting out on a hopefully promising academic career. He would be well-served to review this catalog of logical fallacies; many should be familiar to him. http://en.wikipedia.org/wiki/List_of_fallacies
“The question is a matter of how much profit, and for the average person today, that question is moot.”
Ten years is enough time for six hundred million people to die, very probably including one of your relatives. Even if you don’t care about the rest of the world or the huge sums of wealth, or the extra who-knows-how-many billion years of uploaded life we’ll get before the heat death of the universe, most people do not see the senseless deaths of people close to them as “moot”.
“A decentralized, multicomponent mind,”
An insect colony is not a “mind” in any sense of the term. To quote the dictionary:
mind /maɪnd/ Pronunciation Key – Show Spelled Pronunciation[mahynd] Pronunciation Key – Show IPA Pronunciation
–noun
1. (in a human or other conscious being) the element, part, substance, or process that reasons, thinks, feels, wills, perceives, judges, etc.: the processes of the human mind.
An insect colony as a whole does not think, reason, perceive, or judge anything outside what the individual insects do. If I walk up to an insect colony and whack it with a stick, each insect does not immediately become aware that there is an intruder. All the insects know that something unusual is happening, but those who can see me and know what kind of threat I am do not telecommunicate this knowledge to the others.
“Indeed, the individual in such a ’superorganism’ is akin to a single cell in the body of a conventional animal and quickly dies if it becomes separated.”
You would most probably die if a great hand lifted you out of civilization and dropped you alone on the African savanna. This does not mean that human civilization forms a giant collective groupmind.
“Really scientifiction-esque!”
Nice selective quoting. To continue your quote:
” So efficient and purposeful is a colony of ants that much has been written, particularly by science fiction authors, about the possibility of hive intelligence as the organizing principle in some extraterrestrial societies. This idea was taken up by H. G. Wells in his The First Men in the Moon (1901) and has been explored many times since.
In general, the prospect of a hivelike human community has been regarded as repugnant and the ultimate nightmare that might result from totalitarianism. Cold War era films such as Invaders From Mars (1953) and Invasion of the Body Snatchers (1956) drew a close parallel between the alien hive and the suppression of individualism under communist regimes. In his novel The Cosmic Rape (1958), Theodore Sturgeon pits humanity against “The Medusa,” a galaxy-wide hive-mind empire that subsumes the conciousnesses of its intelligent conquests. This negative portrayal has continued with the Borg in Star Trek and their attempts to assimilate other sentient species into the “collective”, although curiously it is the Federation – a cosmic extension of the United States – which tends, in the end, to do most of the absorption of other cultures. A different perspective on the loss of individuality and emergence of a group-mind is offered by writers such as Olaf Stapledon, in Star Maker, and Arthur C. Clarke, in Childhood’s End, who foresee mankind’s eventual melding with some form of broader, extraterrestrial consciousness as a triumph of transcendent evolution.”
“I fail to comprehend how someone who claims familiarity with computers would fail to make the connection that biology, having discerned parallel processing once, wouldn’t manifest it again.”
Firstly, an insect colony does not “parallel process” in the computational sense of the term, because a colony does not form a computer; there’s no single calculation which is distributed across insect neural networks and whose results are collected at a single source. Indeed, even if you deliberately engineered the insect nervous system to work as a computer, it would be tremendously awkward because insects communicate primarily through chemical signaling. Secondly, AGI and programmming are not biology; simply because a feature showed up under natural selection does not mean that the same feature is likely to show up under an engineering process.
“Example: “Anyone who truly recognizes the potential of AGI should have enough sense to realize…†Oh really? Then if I disagree, I must not have sense.”
Hmmm… you’re right, that is a fallacy. My apologies.
Popular perception of AI feasibility does indeed matter because that’s where the research money comes from. It is completely useless if AI is in fact feasible but no one of any consequence believes so.
Munigant and Ian, you should recognize that Tom McCabe has spent much more time (several years at least) researching and considering these issues, from all possible angles, than you have. This doesn’t necessarily mean that he’s right and you’re wrong: just that you should consider his arguments more carefully before dismissing them.
Tomas, I refer you to your original response with which I rebutted the material that you respond to with the above line.
That, Tomas, is precisely what you then turn around and accuse me of when I provide actual text from that document which disproves your statement that it solely describes hive minds in science fiction. Bark up another tree, man. :)
The following is said with the utmost respect and in a calm manner:
Bullshit is bullshit, Michael.
If something is flat-out wrong on a deeply factual nature, I have a tendency to call them on it. You need to stop siding with him due to a priori bias, if you expect to maintain a higher level of intellectual discourse: if your goal is instead to agitprop for your own ideology, I would appreciate a disclaimer to such — much as I disclaim my own libertarian nature on my blog. In those instances where he has had valid arguments, I’ve recognized them. They have, to be totally frank, been preciously few and far between. You’ve been mentally filling in the gaps for him.
Par exemplorum: You will recall our conversation regarding the flexibility of the human brain and its ability to adapt to new physical structures. You sided with Tom because, and I paraphrase: IA isn’t likely to come before AI, thus IA is more difficult. The problem, however, is that I wasn’t discussing IA vs. AI. I was discussing the flexibility of the bioneural substrate that is the human brain. You then went on to say that, as you’d been studying it for some time, you were more of an authority. I let that one go, then.
Here, I will not: If you indeed have been following the subject for a number of years, then you quite well ought to know that the major stumbling block to the development of new I/O BMI implantation technologies is not that the brain is difficult to adapt to; but rather that one of the patients of the cortical implant runs died of infection resulting from when he pulled on the cords leading outside of his brain — thus resulting in the ethical cessation of new implantation experimentation.
This is only now being overcome once again — but recall, this was one of the major arguments against Cyberkinetics, Inc — and has much to do with why they have delayed the Braingate technology.
I have been patient with this, but as it stands, it needs addressing. :)
Oh, and as an unfortunate side-note: New to the transhumanist movement is not the same thing as new to the research.
“Popular perception of AI feasibility does indeed matter because that’s where the research money comes from.”
Research money for AGI projects is not likely to come from support among the average-Joe, SL0 populace. The average person does not care one whit about most things that may possibly change the world twenty years from now, and even if they do care, they are likely to see themselves as being passive observers of history rather than participants or helpers. Perception of AGI among people who are likely to donate- philanthropists, academicians, think tanks- is important, but one of the first things you should encounter when you take the time to actually read about these issues is how xenophobia is irrational. If you can actually study the Singularity in depth and not realize that an AGI is something other than a Hollywood evil steam-and-gears machine that we all should run in terror from, then yes, that needs to be fixed.
“to counter objections, he simply restates his position without qualification or further detail, or points his opponent to a list of papers that must be read.”
I realize I have not provided in-depth details or justifications for many of my arguments; this is more a side effect of the sheer number of points people make than the quality of the justification for these arguments. If we were talking about thermodynamics, and someone asked a question about whether you could build a perpetual motion machine, I would probably reply “Go read a physics textbook”. This is not because I like being a jerk, but because there are already arguments available on this subject and I don’t have an infinite amount of time to reiterate them. If I give a simple argument about why something is impossible and you simply say “That’s wrong”, that doesn’t help me know why you think it’s wrong, so the most I really can do is rephrase it to try and make it easier to understand.
If anyone has spotted any more simple rhetorical fallacies like M. Munigant, for the love of Belldandy please tell me about it. My brain does not work the same way when I write an argument as when I read an argument, and if I skip over a rhetorical fallacy that I would have noticed in someone else’s post, I really would like to know.
“I provide actual text from that document which disproves your statement that it solely describes hive minds in science fiction.”
I never said that it solely describes minds in science fiction; if you got that from my statement, once again, my apologies. The simple fact that it explains anything (in the very first hit!) using concepts imported from science fiction should be a big red flag, because it means that any thinking done in that area is likely to be mixed up with ideas imported from fiction.
“If something is flat-out wrong on a deeply factual nature, I have a tendency to call them on it.”
Please do call me on it, but please try and say why you think my argument is wrong. If you just say “it’s wrong”, I can’t do anything more than try and reiterate it because I have no clue where you think the flaw is.
“You’ve been mentally filling in the gaps for him.”
This is an unfortunate side effect of the large inferential distance of AGI-related thinking. If the conclusion is seven steps distant from the premises everyone shares, I may very well skip over some of the inferential steps, and Michael may very well mentally fill them in, as he is used to discussing these issues. Again, please point out specific examples.
“You then went on to say that, as you’d been studying it for some time, you were more of an authority.”
This is a fallacious argument from authority. To go back over that debate: I agree that the human brain is a flexible substrate. This does not mean that it will automatically or easily adapt to foreign technology.
“but rather that one of the patients of the cortical implant runs died of infection resulting from when he pulled on the cords leading outside of his brain — thus resulting in the ethical cessation of new implantation experimentation.”
I was not aware of this at the time, as I had not studied the history of neural interfacing. Nevertheless, pointing out the existence of obstacle B does not mean that obstacle A becomes any easier. You are right that this is a major obstacle, but that does not make the engineering obstacle any easier.
“Why should a superintelligence care more about the goals of its creators any more than a random computer program would?”
It seems I wasn’t clear enough in my original reply. I am very interested in the promise and potential of true AI, and how it can benefit and transform humanity, and I have no critique or major objections to any of the fascinating possibilities discussed in this thread (and elsewhere) about the nature and properties of the AGI. Although perception =/= feasibility, public interest is extremely important for funding.
I do, however, object to the view put forth by Michael and Tom that the AGI will transcend centuries of flawed human society so thoroughly that humanity will be delivered to a golden age/paradise. IMO, this is simply grandiose wishful thinking which crosses completely into the realm of science/speculative fiction.
I am not claiming that flawed human nature, or any human social constructions for that matter, will somehow be programmed into the “code” of the AGI … however, the AGI will not spring into existence overnight; it will be an iterative process over many years (or, most likely, decades), with many opportunities for the AGI-having humans to exploit the advantage of AGI over non-AGI-having humans. The exploitation will not be part of the AGI, but it will be part of how the AGI is used/applied by humans. Humans have been doing this with technology for centuries. Why will AGI be any different? Even if the AGI is entirely altruistic, in its nascent stages it will be utilized by selfish humans for what they (in the less-than-superintelligent minds) deem “best.”
Also, why would narcissistic humans willingly cede control of their lives and destinies to the AGI (as Michael and Tom seem to be implying)? “Extraordinary claims require extraordinary proof” — so do extraordinary predictions. While the AGI discussions are compelling, I believe Michael and Tom are undermining their own credibility by bundling the potential nature of the AGI with such a grand (and IMO unwarranted) utopian vision of how it will transform humanity. It is a leap of “faith” which, although exciting and inspiring, really has no place in serious discussion of the subject, as it is only one of many possible end-games.
“IMO, this is simply grandiose wishful thinking which crosses completely into the realm of science/speculative fiction.”
To my knowledge, no SF story has ever had a scenario in which the AGI is created, humanity instantly ascends into paradise and all the characters live happily ever after. It would be too boring. Vernor Vinge invented “Zones of Thought” for precisely this reason.
“with many opportunities for the AGI-having humans to exploit the advantage of AGI over non-AGI-having humans.”
A half-working AGI isn’t something you can use to exploit people; it’s simply a huge mess of code that spits back compiler errors and an occasional interesting behavior. How useful would the brain be if large sections were either unfinished, or totally untested?
“Humans have been doing this with technology for centuries. Why will AGI be any different?”
Because AGI isn’t just a piece of technology; it can invent technology itself, and moreover can do it faster and better than humans can.
“Also, why would narcissistic humans willingly cede control of their lives and destinies to the AGI (as Michael and Tom seem to be implying)?”
An AGI will have vastly more power over the world than we will due to its intelligence. We won’t choose to cede power to the AGI (at least, not collectively) any more than the chimps decided to cede power to the hominids.
To be fair: Tom — what you responded to here wasn’t directed to you — it was directed to Michael.
“This is an unfortunate side effect of the large inferential distance of AGI-related thinking. If the conclusion is seven steps distant from the premises everyone shares, I may very well skip over some of the inferential steps…”
Ah. I think this partly explains many of the vehemently-disagreeing posts in this thread.
If I am understanding correctly, each of the seven steps will be further from the shared premises (and understood/agreed-upon by less and less of the audience), so it is not surprising that the conclusion will leave many scratching their heads.
A threaded discussion is unfortunately not the best forum for exploring such a seven-step-distant inference. If you have time/interest, and a specific conclusion on which you could elaborate and present the individual steps/inferences, I encourage you to create another blog posting on it. I think it would make the overall discussion of AGI here more accessible.
“If you have time/interest, and a specific conclusion on which you could elaborate and present the individual steps/inferences, I encourage you to create another blog posting on it.”
If an argument comes up enough, I will eventually write a blog post on it, if only so I can link to it when it comes up again. Have any suggestions?
“A half-working AGI isn’t something you can use to exploit people; it’s simply a huge mess of code that spits back compiler errors and an occasional interesting behavior. How useful would the brain be if large sections were either unfinished, or totally untested?”
Sorry Tom, this is a straw man. I did not say/imply “half-working.” I simply said that a instantaneous full-blown omnipotent AGI would not be the realistic outcome of any such project. Like every other technological innovation, AGI will be an iterative process. If you believe otherwise, please elaborate. (Remember — extraordinary claims require extraordinary proof).
“Because AGI isn’t just a piece of technology; it can invent technology itself, and moreover can do it faster and better than humans can.”
Again, I claim this is wishful thinking. I am curious about the many inferential steps between my premises and your conclusion. To me, you are postulating a futuristic deific intelligence in a manner no different than a science fiction author would.
“An AGI will have vastly more power over the world than we will due to its intelligence. We won’t choose to cede power to the AGI (at least, not collectively) any more than the chimps decided to cede power to the hominids.”
Again, I claim wishful thinking. Humans are notorious control freaks. Will the AGI simply overwhelm us? Possibly, but IMO this is science fiction again. IMO, more realistically, AGI will evolve slowly (like every other technological innovation) and will be shaped by the society it evolves in. I don’t understand your claim that AGI will be “different” from other technology and not subject to the same societal forces which have affected technology for centuries.
The level of political naivety in these discussions is astonishing. The ultimate question even if this work is feasible (which intelligent people think it isn’t see for example physicist Roger Penrose’s The Emperor’s New Mind), even given that it’s feasible who will control it and for what ends? The idea of Microsoft, Monsanto, and the Pentagon having access to a cold inhuman inhuman supper intelligence to further their nefarious plans isn’t exactly comforting.
And to all those net.libertarians on the thread does freedom include my freedom to walk in an undisturbed forest if I am poor? If not then you don’t really believe in freedom per sae but only freedom for the rich and you ought to call yourself property-tarians and Libert-tarians IMO. Be honest with yourselves about what you really value for a change, OK? Hint the words “massive profit” used in this thread ring ring a bell? That is what you really value not freedom and you know if you aren’t lying to yourselves.
P.s. us weak little humans better work out our conflicts FIRST before we unleash forces on the world that make an atom bomb look like a play toy. A “grey goo” Hiroshima could easily for example consume all the carbon on the planet and end all life in days if not hours. THINK of the whole socio-economic system before you pursue this research further, please. If not you will be facing a coalition of people ranging from right wing C.S. Lewis fans to environmentalists to radical lefties demanding the benefits be shared by all people. This is not happening in a vacuum, although I’m sure it easy for those of you in computer labs at University Campuses to think it is happening in a vaccum.
People outside your little insular community are going to want input before you proceed count on it.
Mr. Raven, I have already penned a response to your point here. Functionalism In Action — Egalitarianism At Gunpoint? A Guaranteed Abundance
I will suggest to you that you re-read the thread in question, and note that the conversation was not a concern with “massive” profit but rather whether it mattered that there would be great profit vs. massive profit.
It is also extremely important to note that profit comes in more forms than mere monetary/fiscal gains. I would take having the wealth of all human-created/derived information accessible within microseconds as compared to all the money in the world.
“(which intelligent people think it isn’t see for example physicist Roger Penrose’s The Emperor’s New Mind)”
http://www.acceleratingfuture.com/tom/?p=19
“The idea of Microsoft, Monsanto, and the Pentagon having access to a cold inhuman inhuman supper intelligence to further their nefarious plans isn’t exactly comforting.”
When a superintelligence comes into existence, everyone’s nefarious plans will cease to matter, because the superintelligence has a lot more leverage over the world than any political entity. If you graph an equation of exponential (= human) versus hyperbolic (= self-improving AGI) growth, eventually the hyperbolic term becomes so large that the exponential term isn’t even worth noticing.
Reading through this thread, I don’t see the connection between AGI creation and the fairy-tale ending (AGI solves all human problems). I don’t know what seven layers of missing inference are that Tom (?) refers to, but as another poster points out, the discussion takes on a near-religious overtones at times (i.e. Jehovas witnesses). All-powerful AIs are presented as fait accompli. How about connecting the dots for us?
“How about connecting the dots for us?”
An excellent request. To connect the dots:
If you look back over human civilization, all of it came from one source- human intelligence. The houses, the cars, the spaceships, the light bulbs, the computer you’re typing this on, everything was designed and built by human intelligence. The difference in brain-to-body ratio between humans and chimps, the difference that allowed us to give birth to all of human civilization and history from 10,000 BC to today, was a factor of three. That’s it. Only a factor of three in brain improvement, and look at what has resulted. Since intelligence is the force that gives rise to technology, if you improve our intelligence, you improve our ability to create absolutely every single piece of technology ever invented by humans. Now look at the speed differences between humans and computer processors. A computer processor is already ten million times faster than us in clock speed, and it sends signals around a million times faster than human neurons. A computer processor can be linked in parallel with other processors to increase processing power, something that cannot be done with individual human brains. And a computer can take up much more volume than the human brain, as it isn’t limited by the size of the birth canal. With the advent of nanotechnology, even without weird stuff like quantum computing we will be able to get a good nine to twelve orders of magnitude more computing power than we have now. Nine to twelve orders of magnitude. When a factor of three resulted in all of human civilization.
Now, all that is simply the result of more powerful intelligence, in terms of raw computing power. It doesn’t go into better intelligence, intelligence that is designed to be much more efficient than humans are. Both chimps and humans were designed by the same agent- natural selection. Natural selection has no intelligence whatsoever, has no foresight, and is constrained by the need to preserve existing complexity. If you look at the human brain, logical thought was the very last thing to be added on, and so it is very slow and clumsy. Your brain can mentally rotate a 3D object in realtime- which requires millions of FLOPS- but you will be hard-pressed to do a single floating-point calculation by hand in one second. The longest inference chains an individual human brain goes through are only a few steps. Einstein’s theory of relativity was built out of pre-existing geometrical concepts first originated by Riemann and other mathematicians. A computerized intelligence could think through a thousand-step logical chain in less time than it takes for you to blink. A computerized intelligence could see numbers and sets and groups as intuitively as we see a banana.
And none of that describes the concept of recursive self-improvement. To shamelessly steal an excellent explanation (source = http://yudkowsky.net/singularity.html):
“Once we have human-equivalent computers, the amount of computing power on the planet is equal to the number of humans plus the number of computers. The amount of intelligence available takes a huge jump. Ten years later, humans become a vanishing quantity in the equation.
That doubling sequence is actually a pessimistic projection, because it assumes that computing power continues to double at the same rate. But why? Computer speeds don’t double due to some inexorable physical law, but because researchers and engineers find ways to make faster chips. If some of the researchers and engineers are themselves computers…
A group of human-equivalent computers spends 2 years to double computer speeds. Then they spend another 2 subjective years, or 1 year in human terms, to double it again. Then they spend another 2 subjective years, or six months, to double it again. After four years total, the computing power goes to infinity.”
Exellent summary, thank you. The last “dot connection” is still missing though. How to infer:
“As a consequence of these conditions, you (and everyone else) will enjoy unconditional material prosperity and indefinite life-expectancy – with the resulting time and means for pursuits that may include increasing your own intelligence and exploring the galaxy. You will be free to forgo most of the usual misfortunes of illness and injury, and no person close to you will suffer death from disease or old age unless they choose to.”
as a result of AGI’s birth? Once people are so widely surpassed/made irrelevant, what “motivation” will AGI have to treat them in this way? (Above passage sounds like something from the Bible, you must admit)
“Once people are so widely surpassed/made irrelevant, what “motivation†will AGI have to treat them in this way?”
AGIs will not treat people this way simply by default; see http://www.acceleratingfuture.com/tom/?p=21. We must be very careful about how we design the AGI’s goal system so it is nice to us.
Tom McCabe wrote:
Y’know, that right there says an incredibly great deal about the differences of opinion between Tom and I. Insofar as I’m concerned, as genetic algorithms show, evolution (aka natural selection) is anything but “unintelligent.” I tend to take a much broader view of the term. It’s mainly that its baud rate is so incredibly slow. I suppose you could call it ‘xenophilism’ — I try to keep my perspective open to the product of incredibly alien approaches to intelligence than we humans are possessed of.
“Y’know, that right there says an incredibly great deal about the differences of opinion between Tom and I.”
Yes, it does.
“Insofar as I’m concerned, as genetic algorithms show, evolution (aka natural selection) is anything but “unintelligent.—
Check the dictionary definition of intelligence:
“in·tel·li·gence /ɪnˈtÉ›lɪdÊ’É™ns/ Pronunciation Key – Show Spelled Pronunciation[in-tel-i-juhns] Pronunciation Key – Show IPA Pronunciation
–noun
1. capacity for learning, reasoning, understanding, and similar forms of mental activity; aptitude in grasping truths, relationships, facts, meanings, etc.”
Evolution cannot learn, reason, understand, or grasp anything because it is an abstract process that happens to all self-replicating systems with high birth and death rates. You might as well try and say that Fermat’s Last Theorem is intelligent.
“I try to keep my perspective open to the product of incredibly alien approaches to intelligence than we humans are possessed of.”
Evolution is an optimization process, but it is not intelligent. I can write a computer program that takes a random string of characters and rearranges them until they sound vaguely like Latin- that does not mean that the computer program is intelligent, only that it optimizes text strings toward Latin-sounding results.
Ian, if you actually studied evolution and natural selection in detail – the process itself – you’d see it’s quite dumb. The only reason it gets anything done at all is that is has absolutely tremendous lengths of time within which to work. Also, observer bias dictates that intelligent beings will necessarily evolve on a planet where evolution successfully produced intelligent beings, causing us to overestimate its creative powers and success.
Y’know, given the fact that what is or is not intelligence has been a matter of great debate for philosophers, xenobiologists, and the like, for decades now — I would recommend that the two of you go public with your insights.
After all, this certitude is unique to you.
(As a side note; Michael, you and Tomas both might want to go back over your own articles and research on the topic of anthropocentrism. Just a suggestion.)
Ian, I never made any statements about intelligence in my comment. All Tom is doing is comparing the relative merits of two separate optimization processes – evolution, and intelligence – which have many widely-recognized differences.
How are we being anthropocentric? Anthropocentric means regarding Homo sapiens as a somehow central example of intelligence or agenthood or something. Nothing either of said applies to that.
Are you just getting discouraged that you are being argued against, and throwing out accusations for the hell of it? Next time you accuse, please point to specific sentences you are referring to.
We aren’t disregarding the power of evolutionary search, just pointing out that its inferior to intelligent design. There is a whole discipline centering around inference and decision theory, which lays out some universal qualities of intelligent reasoning without invoking anything related to biological evolution.
“I would recommend that the two of you go public with your insights.”
Er, you do realize that this is all publicly viewable, and that we’re already trying to get this blog as high a readership as possible, right?
“After all, this certitude is unique to you.”
The idea that evolution has no intelligence is not unique to us; it is the view of a long, long line of specialists in the area.
Michael wrote:
Not even close, Michael. Not even close. Quite frankly, you have a great deal of learning to do here; too much to focus on a single point. I have already explained, here, how your position is anthropocentric in general. I will only recommend that you once again go back over it. That you cannot perceive it only emphasizes the magnitude of the intellectual stumbling block you have on the subject.
This is true, and irrelevant. Please note my previous commentary and explanations. I won’t repeat myself.
“I have already explained, here, how your position is anthropocentric in general.”
A quick search of this thread shows that the only time you use anything with the stem “anthro” is in this statement (other than your past two comments):
“Michael: I have a somewhat enhanced appreciation for why you would assume at first glance that I was anthropocentric in my earliest comments here, after having read this thread (well, skimmed).”
This isn’t anywhere within a light-year of an explanation. Quite frankly, I think you’re just trying to dodge any semblance of an argument here. You refuse to explain your objections, you refuse to state what explanation you’re referring to, you refuse to provide a link and you refuse to give us any hint about where your explanations are in the several books’ worth of comments that have been posted to this blog.
“Munigant and Ian, you should recognize that Tom McCabe has spent much more time (several years at least) researching and considering these issues, from all possible angles, than you have. This doesn’t necessarily mean that he’s right and you’re wrong: just that you should consider his arguments more carefully before dismissing them.”
Come on, guys. Argument from authority. Michael, you need to review the list of logical fallacies as well. (And anyway — please tell me: how much time have *I* spent researching and considering these issues? ;)
Esp. in a forum like this, making a convincing point has nothing to do with what you’ve read or how many years you’ve been in your field. It doesn’t even have anything to do with whether you’re correct or not. It has to do with articulate, complete, and well-reasoned presentation. This is why several other posters are calling Tom out on his missing inferences and he’s had to go and backfill them.
Don’t get frustrated if your ideas aren’t being received verbatim as the great truths that they are … instead, use it as an opportunity to hone your presentation skills, and deduce what is crucial and essential for others to follow your logic chain.
“Like every other technological innovation, AGI will be an iterative process.”
Yes, it will be; this does not imply that the curve of the power you get versus the effort you put in will be nice and smooth. The development of the nuclear bomb was an iterative process; first we discovered radioactivity, then nuclear reactions, then fission, then the chain-reaction, and so on. However, if you graph the power of nuclear bombs to influence the world versus time, it will not be nice and linear; it will be close to zero right up until Trinity, at which point it will go “foom”. Nuclear weapons development is iterative and follows a continuous progression, but you still cannot use a 99% complete bomb to blackmail the world, since nobody will believe that it works without years of discussion and debate. The same principle applies to AGI- it is close to useless until it becomes better at manipulating the world than we are, at which point it starts enhancing itself and goes “foom”.
“I am curious about the many inferential steps between my premises and your conclusion.”
To go through the steps:
1. This world is filled with a whole bunch of technology that did not exist a very short while ago.
2. This technology was built by human beings, and not by Neanderthals, chimps, or dolphins.
3. There obviously is some special property that allows humans to make technology; we’ll call it “intelligence”.
4. If we make this “intelligence” more powerful, since intelligence is the cause of all this technology, we will make all the technology more powerful.
5. If this intelligence is technology-based, when it improves its own technology, it will make itself better at improving technology; this is a self-enhancing feedback loop.
6. Eventually, this growth must peter out because there’s a finite amount of material to work with; however, the total amount of computing power you could get out of Earth is mind-numbing, something like 10^40 FLOPS.
7. Because the difference in magnitude between humans and this new intelligence is so large, the difference in its ability to build technology will also be very large.
8. Since this intelligence has so much more computing power, it will be able to run much faster than we will (human brains are limited to 200 Hz, computers are not).
9. Therefore, this new intelligence will be able to create a whole bunch of very powerful technologies within a very short time frame.
“IMO, more realistically, AGI will evolve slowly (like every other technological innovation)”
The underlying technology base may evolve slowly, but the impact on the world will not. In 1991, most people had no idea what the heck an “Internet” was, and newscasts of the time had to explain that the Internet was a network between computers that allowed computers all over the world to communicate. In 1998. the Internet was The Next Big Thing, the great force that will lead us into the new era.
” and will be shaped by the society it evolves in.”
Computer programs do not work like humans do; they are not “shaped by society” in the conventional manner, or in most cases in any manner at all. When it encounters society, a computer will not be “shaped” by it like a human would; most likely it will simply ignore it all together.
“I don’t understand your claim that AGI will be “different†from other technology and not subject to the same societal forces which have affected technology for centuries.”
The key difference of AGI is that it can itself produce more technology. Suppose you have a machine that makes apples. Simple, boring technology, the stuff we’ve had for centuries. Now suppose you invent a machine that makes pears. Nothing new, really, just more fancy stuff to improve our lives. Now suppose you invent a machine that makes widgets, and then assembles them into new widget-making machines. Due to exponential growth, within a few months at most, the widget-making machines will have consumed all available resources on the planet Earth. This is a *qualitative* difference from other technology, because you can get a lot of change very quickly, because change builds on itself.
At some point we will achieve True AI…
but I think the closer we get to achieving True AI we will realize. Wait a minute. Is that what we really wanted? I rather have something “Near AI”. I want my AI to do what I want. I don’t want it to argue with me or to have feelings. I want it to look at the stock market and help me make money. I don’t want it to ponder its existence. I want it to vacuum my house, and not ask to take the day off.
As Kurzweil implies in his works: The trajectory is actually toward a coalescence of “us” and FAGI(s) (Strong IA, if you will, concurrently with Strong AGI, in other words). Now, admittedly, the “problems” (socio-conceptual, but, indeed, ultimately practical/pragmatic, are staggering) of *personal identity*, *privacy*, *autonomy*, etc., etc., will have to be managed (in terms of inter-subjective [!!] protocols), but there is fairly good reason to hope and expect that this trajectory will more-or-less render moot the whole “Colossus/Cyberdine” “They’re gonna destroy us for spare atoms” type of concern. On the *other* hand, we both need to get the core “Friendly” (ethical-axiological) protocol(s) right, and also make sure that any Hitlers, Stalins, and any other sociopaths/serial-killers types are way-out of the development process “loop” UNTIL the system is capable of rendering what amounts to a (Tiplerian, as it were?) “purgatory” for these types of individuals.
And, of course, Tom is (again, as usual) spot-on in pointing-out that even a “baby” or “seed” FAGI need not be human-like at least in the specific sense that it need not (and, indeed, should not) carry all the “red-in-tooth-&-claw” evolutionary baggage that we neo-chimpanzees do.
And, Alberto, both Hawkins Numenta, and, **especially**, Ben Goertzel’s Novamente, both work on basically and evolutionary/filter/feedback/selective/development process. They learn by evolutionary trial-&-error, but actually store (and thus *accumulate*) info and knowledge which then builds on itself (which is, of course, a nascent, baby “version” of [i.e., a necessary precursor of] full-blown self-improving recusivity (or is should that be recursiveness? ;) )])
Ciao…
Beautifully written, I liked it.
I conceive you may have noted some very interesting details , thanks with the post.
I’ve made a note to myself to write something extra good on Tuesdays, in anticipation of your visit
I will be grateful to give you an source link as well
Fantastic article, thanks for putting up the concept, I have been attempting to figure this around
Which doesn’t generate sense at all. There is a wise selection concerning what’s happening
Exactly where different may any person have that form of details in these a great ways of composing?
We are looking for a person that want more understanding
Just wanted to convey hi there in addition to enhance for your internet site. Perfectly completed
I cannot hold out to read a great deal far more from you as this really is in fact a fantastic web
I’ve got to build a quite excellent content material offered and i assume this page Matches best into it.
possessions within my home nevertheless i could almost accept a much less jumbled life style if it ended up something similar to sporting nominal jogging sneakers. From your side these form of standard running sneakers look deceptively trim as well as clean, however they give every little thing you need to have with no extra cushioning. The design allows you harness your body’s typical electricity and likewise mobility despite the fact that doing work. The actual Adidas Adipure is constucted from extending textile merge to some foot alongside with makes use of tooled midsole-outsole layout. These attributes ensure you can certainly use the particular footwear in a long-term and so they provide a appropriate actually truly feel. My preferred stage about minimal jogging footwear isn’t that it feels as though I am barefoot nonetheless the protection your sneaker gives. I really like the feeling nonetheless am thankful for your difficult structurally curved outsole that allows me personally getaway cruise far more than pavement, monitor, rubble and more. I would not could wish to do by purchasing barefeet, but it’s effortless in these training sneakers. Moreover they offer even more grip for nearly any ground predicament. They offer me any added increase to go more alongside with boost my functioning circumstances. I have uncovered the sneakers are great for toes in almost any problem. They are light on aching foot and i can place them on to get a complete day without soreness. They’re going to assist virtually any toes in spite of no matter whether one particular typically demands particular sneakers or boots or or else not just because they match so as to along with imitate the actual wearer’s ft .. I exercise as soon as or two times weekly and they are generally prolonged long lasting sufficient to continually result in the journey beside me. Whether Intending for a wander or instruction for the race, It truly is my task to don my dependable Adidas Adipure sneakers. They could be nominal working sneakers even so they present you with a best level of electricity and luxury. Path jogging is beginning to turn into far more common then at any time as well as novices could locate it much more thrilling and in addition useful when put next with regular highway operating. Walk working can provide powerful cardio physical exercise training that many path jogging can not. The actual inclining along with climbing down acquainted with walk working gives time frame and energy education which can achieve systems looking for bodyweight reduction alongside with muscle mass mass tightening.
I believe this site holds some very fantastic information for everyone :D. “I like work it fascinates me. I can sit and look at it for hours.” by Jerome K. Jerome.
These inspiring ideas likewise acted like a fantastic way to know that the rest have the same dreams really like my personal own to see a whole lot more concerning this problem. I’m sure there are thousands of more enjoyable times in the future for many who check out your blog.
Great – I should certainly pronounce, impressed together with your site. I had no trouble navigating by means of all tabs as properly as related details ended up being truly simple to do to access. I recently identified what I hoped for before you know it at all. Quite unusual. Is likely to appreciate it for those who add forums or anything, internet site theme . a tones way for your client to communicate. Good task.
There is noticeably a bundle to know about this. I suppose you’ve created specific nice points in functions also.
I am not rattling good with English but I get hold this real leisurely to translate.
Thanks for some other informative blog. Where else may I get that type of info written in such an ideal method? I’ve a project that I’m just now operating on, and I have been on the glance out for such info.
Good information. Lucky me I recently found your site by accident (stumbleupon). I have bookmarked it for later!
an outstanding share! i have just forwarded this onto a co-worker who was conducting a little homework on this. and he actually ordered me lunch because i stumbled upon it for him… lol. so let me reword this…. thank you for the meal!! but yeah, thanx for spending the time to discuss this issue here on your site.
Hi there, i read your blog occasionally and i own a similar one and i was just curious if you get a lot of spam comments? If so how do you reduce it, any plugin or anything you can suggest? I get so much lately it’s driving me mad so any support is very much appreciated.
An intriguing discussion is worth comment. There’s no doubt that that you need to publish more about this topic, it may not be a taboo subject but generally folks don’t discuss such subjects. To the next! Many thanks!!
Some truly nice and useful information on this website, also I believe the design contains superb features.
I’ve been browsing online more than three hours today, yet I never found any interesting article like yours. It’s pretty worth enough for me. In my opinion, if all website owners and bloggers made good content as you did, the web will be much more useful than ever before.
Nice post. I learn something more difficult on totally different blogs everyday. It should always be stimulating to learn content material from other writers and follow just a little something from their store. I’d choose to make use of some with the content on my weblog whether or not you don’t mind. Natually I’ll give you a hyperlink in your net blog. Thanks for sharing.Useful info!
This web site is mostly a stroll-through for all of the info you needed about this and didn’t know who to ask. Glimpse here, and you’ll positively discover it.
Aw, this was a very nice post. In idea I want to put in writing like this moreover – taking time and precise effort to make a very good article… but what can I say… I procrastinate alot and in no way seem to get one thing done.Useful info!