Closing the Loop Saturday, May 19 2007
singularity 7:24 pm
I made the above image while idly listening to the podcast mentioned previously. It describes the Singularity idea pretty straightforwardly. If technology can be used to improve intelligence, even a little bit, then that will lead to further advances in intelligence enhancement technology, and so on, until there are superintelligent gods, right there on our front porch. Thus, it’s counterproductive to work on the really big projects ourselves, when we can ’simply’ invent intelligence enhancement technology, use it to make ourselves smarter, and then use our superintelligence to much more effectively pursue them.
Skepticism around the idea, explicitly or implicitly, generally takes the form that human beings are pretty much the smartest and fastest thinkers that can possibly exist, therefore intelligence enhancement technology will only provide tiny gains. Considering how far intelligence has come since the beginning of life on Earth, I think it’s pretty bold to suggest that we’re the endpoint of the process of intelligence improvement. Some also think it’s “betraying humanity” to advocate superintelligence, but really, I personally love humanity a lot, but think we should avoid xenophobia about greater intelligence. The world needn’t be a zero-sum place where humans automatically lose just because a smarter species shows up.
In the past, the process was incredibly slow, because evolution takes millions of years to make appreciable changes. This time, the process will be incredibly fast, partially because minds are substrate-independent and can be cognitively accelerated by better hardware, but also due to the qualitative smartness factor.
May 19th, 2007 at 8:01 pm
It almost makes you a little wary of abundant nootropics and/or other mass-producable intelligence enhancers that could pass a ‘threshold’ where a large sect is intelligent enough to begin recursive self-improvement, yet either specifically unwise to the implications, impulsive, or simply uneducated/exposed to the worldwide unintended (or worse, intended) consequences of several god-like powers that certain technologies could exert.
May 20th, 2007 at 7:10 am
Great diagram. Here is a thought, add a Venn diagram, they really are the king of diagrams in my opinion.
May 20th, 2007 at 7:40 am
David, you can’t recursively self-improve a human brain more than a little without the kind of nanotechnology that would probably require a superintelligence to control in the first place. People on nootropics could start building an AI, of course, but that’s no different from the general UFAI risk.
May 20th, 2007 at 9:03 am
I was alluding to those methods, Nick. It would be arbitrary enhancements, that would lead to deeper, external ones. And it is a bit different from the UFAI risk, but mainly in unintended acceleration. But yeah, I know, the brain was in no way meant for modifications. Even nootropics are like throwing things into a soup wondering which’ll make it taste better.
May 21st, 2007 at 7:13 am
It’s a really bold statement to say that minds are substrate-independent… that’s what it seems to be at the moment, but until I’m shown a human being running on silicon or any other non-biological medium, I’ll remain a skeptic.
May 21st, 2007 at 8:24 am
Well, if you’re a physicalist, substrate-independence follows naturally because the laws of physics are computable. If nothing else, you could do a quantum chemistry simulation of the whole brain.
May 21st, 2007 at 9:23 am
I don’t like the wording of “minds are substrate-independent” in the least. To me, it’s better to say that “the capacity of minds to exist is greatly/massively substrate-flexible.”
I say that because the platform, naturally, has its limitations on what *range* of minds can be constructed on it. I don’t think you’ll ever find an unmodified, genetically “normal”, human being with an IQ of 10,000.
May 21st, 2007 at 10:42 am
It is quite possible if we think that the biggest obstacle to more intelligence is not the ability to establish correlations, (at least on the persons intelligent enough to make them in the first place), but the capacity to access the memory. If the technology allows you to access your memory banks which are hidden to the average human, it will give you the quantum leap you mention.
May 21st, 2007 at 12:48 pm
“Great diagram. Here is a thought, add a Venn diagram, they really are the king of diagrams in my opinion.”
I still have nightmares about Venn diagrams from when I was in elementary school (shudders).
“without the kind of nanotechnology that would probably require a superintelligence to control in the first place.”
It doesn’t require superintelligence to build a computer with enough FLOPS to enable recursive self-improvement; heck, we’ve managed Moore’s Law even with constant intelligence.
“It’s a really bold statement to say that minds are substrate-independent… that’s what it seems to be at the moment, but until I’m shown a human being running on silicon or any other non-biological medium, I’ll remain a skeptic.”
Conjecture: Humans can be run on any Turing-compatible substrate.
Proof:
1. Humans are made of atoms.
2. Atoms follow the laws of quantum electrodynamics.
3. QED is Turing-computable.
4. Therefore, humans are Turing-computable.
QED.
“I don’t think you’ll ever find an unmodified, genetically “normal”, human being with an IQ of 10,000.”
You’re correct, but that has very little to do with the neural substrate itself. After all, we have IQs of 10,000 or so on a frog’s scale, yet we share the same substrate.
May 21st, 2007 at 1:31 pm
Neuromancemeister: “If the technology […]” just put you outside the realm of what I was trying to discuss.
Tom: That just illustrated my point; that the substrate itself is radically significant to the *range* of minds possible on said platform.
You *can* have a computational IQ of 10,000 with something with the “overall” brain-power of a frog… but that “mind” will never be anything resembling that of a human’s. I.e.; there is mind-substrate *compatability* dependence.
May 21st, 2007 at 2:05 pm
Tom, I was only saying that recursively self-improving a human brain needs SI-controlled nanotech.
May 21st, 2007 at 5:52 pm
Since when has quantum electrodynamics been computable–moreover efficiently computable? And since when has it been reconciled with relativity? It hasn’t. Sorry to rain on the parade, but we don’t fully understand all the principles of our universe yet.
I’d really like to see this grand plan on simulating quantum dynamics on deterministic computers!
May 21st, 2007 at 5:57 pm
Why does everyone flatly assume that these magic words “self-improving” automatically imply exponential growth or improvement? How do we know that there aren’t fundemental limits? How do we know the curve we are on is not an S-curve and we’re approaching the knee? How do we know we aren’t on a tangetial to linear growth, or tangential to an upper bound? After all, all of those curves are monotonically increasing. Haven’t you people ever seen an arc-tangent function?
May 21st, 2007 at 6:21 pm
Well, one reason is that the last time intelligence improved a bit, separating humans from our primate ancestors, human civilization was the result, and that’s huge.
Another reason is that minds can be accelerated by better hardware because they are substrate independent. (This applies most to AI.)
Another reason is that exponential manufacturing is not far off, and that could be used to convert huge amounts of matter into advanced technology very quickly, including intelligence enhancement technology.
Calm down.
Obviously, if clueful people have been considering this for upwards of two decades, there might be something to it. I can’t explain everything in every blog post, y’know.
May 21st, 2007 at 7:39 pm
Quantum electrodynamics has always been computable, and it doesn’t matter if it’s not efficiently computable, because that was just a reductio anyway - nobody thinks that’s how minds will actually be simulated. Relativistic effects are not important in the brain. Dualism could be true - I have yet to see a convincing refutation of this argument - but if it’s not, then minds are necessarily substrate-independent.
Regardless of what fundamental limits there may be, there exist designs for nanocomputers that are plenty powerful enough to allow the Singularity and uploading. I wish I had a link. Any limits are way, way up there.
May 21st, 2007 at 7:46 pm
It should be noted that even if some form of dualism is true, this doesn’t mean the Singularity can’t happen. A computer can reason perfectly well, which is all you need, even if it can’t have subjective experience; and if weird physics is actually necessary to do efficient reasoning (really unlikely), we can always duplicate the brain’s weird physical structures in another substrate.
May 21st, 2007 at 10:53 pm
The majority will comfort itself, that the intelligence is not substrate independent.
That it’s “to bold to conjecture otherwise”. The minority will do the job anyway.
Great fun, to read both sides.
May 21st, 2007 at 11:55 pm
Minds are not substrate independent in the sense that they are influenced by the substrate. For example someone with a headache may fall into bad mood and poor thinking.
I think what is important is how one *wants* his mind to function, or in other words the important features that one wants to keep.
If my mind could today be translated to another substrate, I would wish to retain my core sense of identity, high level thinking, sense of humor, love etc., but give up the propensity to unchosen mood.
May 22nd, 2007 at 12:18 am
Apologies if I came off as overly grumpy on the subject, but after viewing centuries of human misery having been populated by intellectuals who believed that this, that, or the other new emerging magic or technology was going to grant immortality and radically change the world, and that we would have the knowledge, wisdom, and power to wield whatever said technology, I am very cynical on the topic.
It’s all well and good that you guys want to imagine what the future is going to be like, and its great possibilities for progress. There is nothing wrong with that in and of itself, but I think the transhumanists have amplified their imaginations far beyond what technology will deliver in the absurdly short time frames.
I simply don’t believe we know enough–about anything–to issue in this vast superintelligence. The AI field is marked over and over with moving goal posts and busted expectations. Even Turing thought that we were close, in his time, to having human-level intelligence. And that was 60 years ago. (I know, I know, the equivalent of those 60 years of progress is supposed to happen in 5 years nowadays, yadda, yadda, yadda).
Is it completely inappropriate for me to just end my parade raining with “Dream on guys”?
May 22nd, 2007 at 12:45 am
I understand your sentiments, but as you might imagine, I’ve heard them many MANY times and have persuasive counterarguments for all the obvious points. The mind-blowing explosion of Internet communication and adoption in the last five years alone may be among the most obvious of many signs that progress is accelerating, and quite rapidly. If you follow the science and tech news in some detail, you will also be flabbergasted by the types of advances happening in our labs and the increasing pace at which they are hitting shelves.
But anyway, if you disagree with the basic premise of this site, please don’t hang around out of boredom and make negative comments - indeed, click the “unsubscribe” button. It’s that easy.
May 22nd, 2007 at 9:32 am
Substrate independence doesn’t mean, that you think no matter what your brains do in the meantime.
Substrate independence means, that a headache or a bad mood indeed, can be archived in silicon also.
May 22nd, 2007 at 7:45 pm
“Minds are not substrate independent in the sense that they are influenced by the substrate. For example someone with a headache may fall into bad mood and poor thinking.”
The reduction-to-QED shows that any substrate can be simulated, in theory, in any other Turing-computable substrate.
“but after viewing centuries of human misery having been populated by intellectuals who believed that this, that, or the other new emerging magic or technology was going to grant immortality and radically change the world,”
A whole bunch of emerging technology *has* radically changed the world. Or did you not notice that this blog could not have existed fifteen years ago, because there was no such thing as HTML?
“but I think”
“I simply don’t believe”
“I am very cynical on the topic.”
Your personal feelings on the topic are not an argument. Please tell us exactly why you feel badly about transhumanism, or else there’s no point posting: you might as well write a bot that posts “I think 2 + 2 = 7!”.
“Even Turing thought that we were close, in his time, to having human-level intelligence.”
Reference? Alan Turing died in 1954, well before even the most primitive AI systems came online (in fact, if I remember correctly, the term “AI” hadn’t even been invented yet).
May 22nd, 2007 at 8:35 pm
(From Nick Tarleton’s link):
“We then rely on massive quantities of computation and luck to construct just the right computer that happens to read the meteor shower as a program, such that it happens to be equivalent to a zombie’s brain.”
Aha! Gotcha. Any data from a meteor shower would essentially be random. The chances of this randomness forming any kind of conscious brain in a Turing machine are astronomical, I’d guess around 2^1000000 against. Not going to happen before the universe dies. Therefore, in order for the data to run as a workable consciousness, it must be heavily fine-tuned by the computer before use. But in order to fine-tune it, you have to know what the target is, and so therefore you’re inputting consciousness data into the computer. The computer is conscious, but the meteor shower is not because you’ve radically changed the stream of information.
“Does even the possibility[9] of this computer give the meteor shower consciousness, if only for a moment[10]?”
No. Even if the meteor shower did line up in exactly the right way so as to be a representation of a conscious brain, consciousness is not just a static pattern of information; the information must change according to a set of rules. The meteor shower is obviously not going to follow these rules, even for a moment. But if you were very, very careful, and put millions of meteors in exactly the right place, and they all bonked around in a very precise fashion within a closed container, then yes, you could have a conscious meteor shower.
“This time we are looking for a computer (which we’ll call Ralph) such that, when you point it at our overworked meteor shower, it implements an emulation of itself (which we’ll call Fred), as well all the brains and their environment.”
Again, you’re obviously cherry-picking the computer; the relevant Kolmogorov complexity is therefore found within the computer rather than the meteor shower, so the computer can be conscious without the meteor shower being conscious because the computer has a ton of information the meteor shower doesn’t.
“I suppose they might believe in Ralph more than Fred.”
Nope. If the computers are, in fact, completely identical, right down to the quantum level, then each level is just as real as the previous one by any empirical definition of the word “real”, and the system collapses into an infinite recursion. This is an excellent argument for why a system can never perfectly simulate itself.
“Why is this information sanctified into some higher state of being by having a processor just look at it?”
Because when a processor looks at it, it creates new information by processing the old, and so information can flow by in time. This is an essential attribute of consciousness; it is like a wave, in that it can never be fixed into one value of spacetime and still retain its properties. A wave only exists if it is moving; consciousness only exists if information is changing.
“I claim that to a Martian, a Macintosh is the same sort of thing as a toaster or a rock.”
No, it isn’t. A Martian could take a screenshot of the Macintosh using his Martian-brand computer, compute the Shannon entropy of the photo, and conclude that the screen generates highly nonrandom images and therefore must have been built by some optimization process.
“Even if you somehow came up with a confirmed computer, you would have to have a rigorously objective answer to the question: What would make an object a non-computer?”
If I did the above test on all of the Macintosh’s systems and concluded that it really was no different from a rock.
“If you don’t want to rely on a mushy brained human’s intuition, you’ll have no choice but to bring a confirmed computer on stage to analyze your proposed computer. ”
True. In order for the concept of a computer to exist, you have to have a computer to think of the concept. Therefore, a universe with no computers cannot possibly determine whether something is or is not a computer. I don’t see why this is a problem. You might try and argue that, because a computer cannot be defined until one exists, the definition is arbitrary. But this goes deeper than computers- obviously an empty universe can have no definitions, so therefore in order to define existence, something must exist.
“Everything a specific, physical computer can be observed to do[14] can be understood without having to think of it as a computer.”
Every single thing. But taken together as a whole, the series of computations a computer performs cannot be done except by a computer.
“Relativity is necessary to explain the observed universe, while computer science is not.”
Sure it is. The answer to “Do computers exist?” must always be “Yes”, because the asking of the question implies the existence of a computer.
“No one has thus far been able to define a notion like “complexity” sufficiently well that we could someday hope to build a complexity detector.”
Sure we have. We already have a working complexity detector; you can download the Debian package for i386 computers at http://mirrors.kernel.org/debian/pool/main/e/ent/ent_1.0-6_i386.deb.
May 23rd, 2007 at 8:31 am
Thomas, stop using commas! Just eliminate them from your writing. It’ll make what you say a lot clearer.
Tom, you should really blog on this domain. My friend Dan is doing it, but he hasn’t posted much yet. I can set you up lickety-splitsky. Your words are being wasted in the comments section.
May 23rd, 2007 at 5:07 pm
“Thomas, stop using commas! Just eliminate them from your writing. It’ll make what you say a lot clearer.”
Could you recommend any books/websites on improving writing?
“Tom, you should really blog on this domain. My friend Dan is doing it, but he hasn’t posted much yet. I can set you up lickety-splitsky.”
Please do! How do I get started?
May 23rd, 2007 at 6:28 pm
I was talking to the other Thomas about the commas, not you. Elements of Style is the classic writing book, though.
I’ll send you an email…
May 27th, 2007 at 7:48 pm
Tom, re: comment #9, the premises don’t necessarily lead to the conclusion (I tried to add some missing premises, but it just became tautological (story of my life (nested parentheticals, haw many can YOU keep track of? (:)(now I’m just fucking with you))))).
Substrate material may be human brain-meat or silicon, but there are other substrata to be considered, such as the software environment on which the mind runs. Atoms and software environments can be formed into non-Turing-computable substrata, such as pushdown automatons.
I’m as much of a fan of the consequences of functionalism as the next person here, but let’s please stay away from the ‘faith-based logic’.
P.S. Does anyone else feel uneasy about the terms ‘non-biological’, ‘non-organic’, or ‘non-carbon-based’ as used to describe alternative substrates for minds (the mind would be ‘alive’, have ‘organs’, and may be run on carbon rod-logic)? Any proposed alternatives? Simply ‘non-human’, ‘artificial’, or ‘non-typical’?
P.P.S. ‘The Apostrophe’ - Friend or Foe?
May 28th, 2007 at 11:07 am
“Tom, re: comment #9, the premises don’t necessarily lead to the conclusion”
We already know that QED can calculate the future probability distribution over arbitrarily large groups of atoms. We also know that humans are built of atoms, and that the equations used for QED can be computed on a Turing machine. Therefore, any Turing-compatible substrate could, in theory, do anything a human could do by using QED and a quantum-mechanical RNG to calculate out what the humans’ atoms will do.
“Atoms and software environments can be formed into non-Turing-computable substrata, such as pushdown automatons.”
True, but anything on a Turing-computable strata will still be able to do anything a pushdown-automoton-based intelligence can do, just not the converse.