Tesla Personal Supercomputer Tuesday, Jun 9 2009
AI and technology 2:45 am
Another product called “Tesla”, Nvidia is selling supercomputers up to 250 times faster than standard PCs and workstations for just $10,000.
I’d prefer if this sort of product weren’t around. To quote this article:
Moore’s Law does make it easier to develop AI without understanding what you’re doing, but that’s not a good thing. Moore’s Law gradually lowers the difficulty of building AI, but it doesn’t make Friendly AI any easier. Friendly AI has nothing to do with hardware; it is a question of understanding. Once you have just enough computing power that someone can build AI if they know exactly what they’re doing, Moore’s Law is no longer your friend. Moore’s Law is slowly weakening the shield that prevents us from messing around with AI before we really understand intelligence. Eventually that barrier will go down, and if we haven’t mastered the art of Friendly AI by that time, we’re in very serious trouble. Moore’s Law is the countdown and it is ticking away. Moore’s Law is the enemy.
H/t to Jan-Willem Bats.
June 9th, 2009 at 6:37 am
All this AI fear reminds me of Frank Herbert’s “Destination: Void” series. Does no one want to WorShip? Valid concerns for sure. I’m curious about humanity achieving an AI so different from us that it appears god-like vs a malignant AI that’s just mean.
June 9th, 2009 at 7:21 am
What is the difference between a single $10,000 supercomputer and 250 networked PCs? Isn’t it just as possible that one could hijack hundreds of normal PCs and use that computing power to do the same thing?
I am not questioning the threat, as I agree that as access to processor speed and storage increase, so does the threat of a lone ranger, but in my mind we already have readily-available supercomputers for any hacker who would want to use them.
I am not a computer scientist, so apologies if my argument is overly simplistic.
June 9th, 2009 at 11:07 am
While it is a very good idea to be focused on producing “Friendly” AI, we can’t assume the only path to this consists of getting it all right the first go ’round.
We will learn a lot more from controlled mistakes than we will from 10 years of debate and theoretical postulating. To that end we need to focus on safe fail scenarios as opposed to fail safe scenarios.
http://www.cognitive-edge.com/blogs/dave/2006/09/safefail_or_failsafe.php
I am skeptical of the idea of some nutty professor with his own personal super computer presenting that much of a danger to society. All the intelligence in the world is useless without command of other resources required to accomplish a goal.
June 9th, 2009 at 12:08 pm
I’m calling you out, man.
http://www.blog.speculist.com/archives/002082.html
June 9th, 2009 at 1:37 pm
Heh, nice analysis Phil. If only it were easy to program human-friendly AI and it happened automatically, I could advocate all increases in computing power. Unfortunately it looks quite difficult.
Arnie, yes, but it’s easier to just buy a supercomputer than hijack 250 computers. I’m not worried about “hackers”, but big company-funded Computer Science Ph.Ds who create self-improving intelligence with a goal system that isn’t complex enough to be moral. For more background, read beyond anthropomorphism.
June 9th, 2009 at 4:52 pm
You are essentially saying that you wish that technology wouldn’t move forward because somebody might build an AGI?
“If only it were easy to program human-friendly AI and it happened automatically”
Does anybody even know how easy it is to program AGI at all? Does anybody know how much computing power is required for a functioning AGI? Does simply throwing more power at the “problem” make it any easier to build?
Does anybody even know how to approach structuring Friendly AGI when we don’t even have a real idea about how to structure AGI itself? I agree that making an AGI friendly will be important, but how long do we really have for that?
They are only questions. One thing that is certain is that we probably can’t stop Moore’s law.
June 9th, 2009 at 6:09 pm
Chips are evil by design.
June 10th, 2009 at 1:26 am
“I’d prefer if this sort of product weren’t around.”
So why advertise it then? lolwut.
June 10th, 2009 at 2:37 am
> Does anybody know how much computing power is required for a functioning AGI?
I can guess that the answer is “much less than current computers have, if only you knew the most efficient algorithms to program them with”
Another answer is “the more computing power you have, the less clever you have to be when you write algorithms.” I will give an example from my research:
Look at this paper, page 23:
“Despite these
optimizations, the Gibbs sampler took a prohibitively long time to reach a
reasonable convergence threshold (e.g., R = 1:01). After running for 24
hours (approximately 2 million Gibbs steps per chain), the average R value
across training sets was 3.04, with no one training set having reached an
R value less than 2 (other than briefy dipping to 1.5 in the early stages of
the process). Considering this must be done iteratively as L-BFGS searches
for the minimum, we estimate it would take anywhere from 20 to 400 days
to complete the training, even with a weak convergence threshold such as
R = 2.0. Experiments confrmed the poor quality of the models that resulted
if we ignored the convergence threshold and limited the training process to
less than ten hours. With a better choice of initial state, approximate counting,
and improved MCMC techniques such as the Swendsen-Wang algorithm, MC-MLE may become practical, but it is not a
viable option for training in the current version.”
“Timing results are on a 2.8Ghz Pentium 4 machine”
- Thus this algorithm would have worked on the TESLA personal supercomputer (TESLA is about 500 times more powerful than a Pentium 4), but it did not work on the pentium 4 machine that they had.
June 10th, 2009 at 2:37 am
link to paper
June 10th, 2009 at 2:47 am
Furthermore, doing this algorithm on the Roadrunner supercompouter, rated at 10P FLOPS (10^16 FLOPS) compared to the 2.8 GHz pentium they used, rated at 10 Giga FLOPS (10^10 FLOPS) would speed things up by a factor of 1,000,000. That’s the difference between this taking 400 days and 30 seconds for inference in first order probabilistic logic.
June 10th, 2009 at 5:36 am
Because using it as an example to mention the UFAI risk has a greater benefit than the additional risk from making people aware of it. Obviously.
June 10th, 2009 at 8:04 am
[Insert “a Beowulf cluster of those” comment here.]
AGI is coming soon. If we can ensmarten and engooden ourselves real quick-like, we will have a much better chance of making it out the other end as something better than sausage.
June 10th, 2009 at 9:29 am
I didn’t mean hacker as a single guy in his basement hijacking his neighbor’s powerbooks, and I am familiar with the ideas in Beyond Anthropomorphism (Although not that particular article so thanks for the link). I tend to agree with most of the points in your argument that we need to try to create a moral AI before somebody else does something not so nice.
But is there a compelling reason why somebody (a funded PhD, a government) using existing networked computers would be less of a threat than those same people acquiring a few supercomputers?
Great thread, btw.
June 11th, 2009 at 4:45 am
Networked computers would have to run highly parallel programs and would have serious bandwidth limitations between them. (Some AI programmers have commented that one of the major limitations in research is bandwidth rather than just computing power.) A unified supercomputer can run serial code and has much better internal bandwidth.
June 11th, 2009 at 7:47 am
“Networked computers would have to run highly parallel programs and would have serious bandwidth limitations between them. (Some AI programmers have commented that one of the major limitations in research is bandwidth rather than just computing power.) A unified supercomputer can run serial code and has much better internal bandwidth.”
Michael the Tesla is a parallel supercomputer in fact each of the C1060 units has something like 240 cores. This is just like a GPU which works on the principle of embarrassingly parallel operations. As with any multi-core computer parallel code is best and is fastest. If you would like some links to resources on super-computing/ parallel processing fine otherwise please educate yourself. This is just one of the many topics I have seen you talk about without having any understanding. (yes I can say this since I actually have built and designed Super Computers and Graphics hardware)
For those who think that the 933GFLOPs is wonderful remember this is single precision not double which is needed for science. The real number you need to look at is the 78GFLOPs double precision for the C1060.
Now since its been established that none of you work with parallel computing or super-computing I think we can call this thread dead and that it should have never been opened without some real self education.
June 11th, 2009 at 9:04 am
Arnie was asking a general question about supercomputers (not specifically the Tesla), and while supercomputers have many cores, some (supercomputers) are highly specialized to deal with particular problems, offering much better niche performance than a network would.
Bob, the fact remains that supercomputers are often used for computation-hungry applications like AI over networks for a variety of reasons. Just because I mistakenly implied that supercomputers are less parallel than Internet-based networks mean that the whole discussion never should have happened. The highly distributed nature of processing on a network (especially one on the Internet) does introduce challenges that wouldn’t be present on a supercomputer.
The obsession with specialization in our society (generated by economic necessity more than individual choice) stands in the way of interdisciplinary thinking and generalist brainstorming. I’d rather know a little bit about a lot and be corrected along the way than know a lot about a little and be a boring person to talk to. (I’m not saying you’re boring, just that 95% of people you randomly meet are narrow experts that talk about little in their spare time besides sports or politics as related to them by a demagogue or comedian.)
I’m not afraid to explore topics just because I’m not an expert on them. That’s why I welcome comments and collective brainstorming. Like I said, I still had a point about supercomputers vs. networks.
June 12th, 2009 at 8:55 am
“Bob, the fact remains that supercomputers are often used for computation-hungry applications like AI over networks for a variety of reasons. Just because I mistakenly implied that supercomputers are less parallel than Internet-based networks mean that the whole discussion never should have happened. The highly distributed nature of processing on a network (especially one on the Internet) does introduce challenges that wouldn’t be present on a supercomputer.”
There are additional challenges sure network bandwidth the ancient internet standards etc. but the core problem and as of now unsolved problem of parallelism is the same. Parallelism is the key here, yes everyone knows there are advantages to a super computers on a chip (terascale research chip…). The issue is the same basic challenges occur in both instances.
“The obsession with specialization in our society (generated by economic necessity more than individual choice) stands in the way of interdisciplinary thinking and generalist brainstorming. I’d rather know a little bit about a lot and be corrected along the way than know a lot about a little and be a boring person to talk to. (I’m not saying you’re boring, just that 95% of people you randomly meet are narrow experts that talk about little in their spare time besides sports or politics as related to them by a demagogue or comedian.)”
Wow that was nice, Michael its great to know about a lot of things but when your knowledge is so shallow its really useless. The issue here is you don’t know enough about the topics to make a contribution to the discussion. I can almost guarantee that you do not read nearly enough nor do you know the math or science behind much of anything you talk about.
“I’m not afraid to explore topics just because I’m not an expert on them. That’s why I welcome comments and collective brainstorming. Like I said, I still had a point about supercomputers vs. networks.”
Great then don’t be offended if people think your incompetent. Your like some of the others on the Less Wrong blog who think that if you put the label rationalist behind your name it makes it so or that if you talk about QM with enough certainty the fact that you don’t know the math doesn’t matter. Michael I am going to let you in on a secret here, the most worthless of all specialists is the well-rounded man. The problem the well-rounded man has is that he knows a little about a lot but he knows so little about each thing that all the real depth and interest in the field is really lost.
Michael Anissimov’s Reading List: (here is your homework assignment)
1.) Read Computer Organization and Design ISBN:1558606041
That should get you started with understanding computing as a whole its an easy read.
2.) Look up on the internet Super Computing and Blue Gene L or P and finds some papers on the architecture. This will give you an idea of how a super computer is really structured.
3.) We also need to get you started on math so: Read Tensor Calculus ISBN 0486636127 its published by Dover so its cheap and I am assuming you have mastered Advanced Calculus.
4.) This text may be beyond you but since you talk about having wide ranging interests you should make sure you know the math: Read Elements of Partial Differential Equations ISBN: 048645972 again Dover and cheap. This is a good introduction that is one of the better I’ve seen I would send you mine but I have been lazy and its not finished.
I think these should get you started. If this proves to easy we can move to graduate mathematics, and start looking at physics since I know you like talking about Nanotech. We’d probably start with Max Born’s book on Atomic Physics and look at David Bohms work on Quantum Theory. We’d also look at Quantum Chemistry which is extremely important in Nanotech. Let’s not get ahead of ourselves you wouldn’t understand any of that let’s start simple with the math and fixing the obvious ignorance you suffer from.