Machinarium Trailer
This looks like an interesting game, at least from an artistic/ambient/immersive point of view. There is also a cool gallery, a demo, and an extremely positive review at Destructoid. Myself, I tend to like games with a little more insanity, like RPGs where my party gets summarily murdered if it takes two steps in the wrong direction, but this looks like a nice game to play if one wants to calm down and enjoy some frustrating puzzles.
Answering Popular Science’s 10 Questions on the Singularity
I thought I would answer the 10 questions posed by Popular Science on the Singularity.
Q. Is there just one kind of consciousness or intelligence?
A. It depends entirely on how you define them. If you define intelligence using what I consider the most simple and reasonable definition, Ben Goertzel's, "achieving complex goals in complex environments", then there is only one kind, because the definition is broad enough to encompass all varieties. My view is that this question is a red herring. The theory of "multiple intelligences", presented by Howard Gardner in 1983, doesn't stand up to scientific scrutiny. Most people who study intelligence consider the theory empirically unsupported in the extreme, and the multiple intelligences predictably useful only insofar as they correlate with g, which just provides more support for a single type of intelligence. The theory is merely an attempt to avoid having some people labeled lower in general intelligence than others. In terms of predictive value, IQ and other g-weighted measures blow away the multiple intelligences theory. Instead of making theories of intelligence unnecessarily complicated with a misplaced effort at encouraging egalitarianism by complicating intelligence measurement, we should apply Occam's razor and realize that g is pretty much sufficient for quantifying intelligence, at least in humans, and possibly beyond.
All that said, there will certainly be different "types of intelligence" developed as we build more powerful AI, meaning to say that some intelligences will be better at solving certain problems than others. From a theoretical computer science perspective, this is a fat obvious "duh". Obviously some algorithms are more specialized than others. The no free lunch theorem is valuable here, and puts the discussion on a much-needed formal footing. Those who discuss intelligence in the popular press often seem to not realize that we actually know a lot more about intelligence and its mathematical formalizations than they think. Because they are not aware of this work, they tend to assume that many features of intelligence are more mysterious to our current level of knowledge than a researcher in mathematical AI might think. Of course, many features of intelligence still are mysterious to us at the present time, but like everything in science, continued investigation will eventually uncover the truth.
Q. How will you use your digital intelligence to kill us all?
A. Contrary to popular belief, software programs are part of the "real world". Especially software programs on the Internet. The Internet, surprisingly, is actually part of the real world too. The barrier between software and the physical world is an illusory one.
A human-level synthetic intelligence on the Internet would actually be more powerful, by default, than your average human today. First of all, such an intelligence would be extremely difficult to kill, even with widespread cooperation. An AI could copy itself on to millions of computers, even renting out cloud computing or botnets to provide itself with computational resources to run itself on. You can kill a human simply by shooting them in the head -- an Artificial Intelligence could have millions of "heads". Once we create a very smart Artificial Intelligence and release it on the Internet, the only way to kill it (if we wanted to) might be to destroy all computers on the planet. Like the mythical hydra, an Artificial Intelligence would grow back 10 heads for every head that gets cut off.
To kill us all, a digital intelligence would need some way of acquiring physical manipulators in sufficient quantity and quality. Preferably physical manipulators that can manufacture raw materials into more physical manipulators. By asking, bribing, or deceiving human beings, an AI could potentially acquire the pieces necessary to build a molecular assembler -- or nanoscale robot arm, which could then be used to build additional nanoscale arms and eventually construct a full-scale nanofactory. This could be used to build advanced robotic components with atomic precision.
Your typical killer AI could probably manufacture thousands of tonnes of advanced robotic devices at multiple locations worldwide before it would be noticed. Such manufacturing could either be kept entirely secret, or integrated with ostensibly human-controlled companies to keep a low profile. It could also be done extremely rapidly. Current estimates of nanomanufacturing throughput argue that it is plausible that such a system could output its own mass (either a product or copy of itself) roughly every 24-36 hours. Thus, an AI that begins with 1 kg of nanofactory on January 1st could potentially have over 2 billion kg (2 million tonnes) of nanofactory on February 1st, as long as it can obtain the necessary raw materials and energy. One 24-36 hour cycle on that manufacturing base could then produce over 2 billion kg of complex, atomically-precise robotics or other products. If uranium ore were available, this could theoretically be used to produce a practically unlimited quantity of nuclear weapons. The warheads could then be delivered to every human city using either something like bird robots that each carry a small piece of the total warhead and assemble it on site, or through drilling robots that power themselves by consuming dead plant matter and deliver themselves to the target via underground.
If the AI in question had a problem with radioactive fallout, it could also just eliminate the human race using neutron bombs, which penetrate through barriers to kill human beings but generate only a small shock wave. Using the right nuclear bombs, though, radioactive fallout would be minimal after just two weeks, so I don't see why a hostile AI wouldn't just nuke us conventionally.
Note: the above scenario is just me pulling something out of my ass. An actual hostile AI with substantially smarter-than-human intelligence could probably come up with ideas that us apes are too dumb to conceive, and be even more effective at exterminating humanity than the scenario I outline here.
You might ask, "why wouldn't we just pull the plug before then?" The Internet is already practically ubiquitous, and it would likely be trivial for any hostile AI of human or greater level intelligence to copy itself onto numerous private servers, unless, perhaps, you developed it in a hut in the middle of Siberia with no satellite or phone connection. Also, any hostile AI would probably behave indistinguishably from a Friendly AI until it passes some threshold of power, at which point we'd be screwed. Since AIs wouldn't have to sleep and could potentially accelerate their thinking speed by adding in new computing power, a hostile AI could probably consider millions of possible moves and countermoves in the time it takes for us to gain a night's sleep. It sounds unfair, but it's a fact we have to face in a universe where the physical speed of our cognitive components is much, much slower than what is theoretically possible.
Q. Would the first true AI wake up without any senses?
A. No. I consider this among the more ludicrous questions in PopSci's piece. Clearly, to develop general intelligence, an AI would need a rich sensory environment in which to soak up data, make predictions, and pursue goals. This could either be a physical environment (through robotics) or a virtual environment. The article says, "Maybe it can see and hear, but feel? Doubtful." This evaluation seems anthropocentric -- there is no real reason why the attribution of feeling is withheld from the AI (if it can see and hear, why not feel?), except to imply that humans can engage in phenomenal experience while machines cannot. Yet, there is nothing so special about humans that whatever cognitive features we have that give rise to phenomenal experience could not be duplicated in artificial intelligences. To the extent that "feeling" things makes us intelligent, those features could be copied at whim by a sufficiently complex AI, and to the extent that "feeling" phenomenal experience is superfluous, some AIs might choose to have it, and some might not.
Consciousness is interesting to think about, but it can be a red herring. Too often, sophisticated-sounding arguments about consciousness and its relationship to AI boil down to one simple and ultimately boring sentiment: "I know I am conscious, and I know other humans are, but I am philosophically uncomfortable with the idea of a conscious machine." This is because we think of "machines" as things like toasters. We have no experience with machines as complex and subtle as the human mind, but because the human mind is entirely non-magical, it's only a matter of time. You are still special even though your mind is non-magical -- don't worry. We humans have survived Copernican revolutions before, we'll manage. Our civilization didn't end when we found out that the Earth wasn't the center of the universe. It won't end when we realize that humans are not the only minds that can feel things consciously. It is not necessary to engage in self-conscious philosophical acrobatics and contortionism to make ourselves feel special. A parsimonious theory of consciousness will not mention humans as a special case. It will likely make reference to much broader cognitive features that we just happen to have, such as self-reflection and the processing of high-level symbols with recursive syntax. We will eventually be able to build these features in AIs too.
Q. Do you have emotions?
A. This is another question which reflects the extreme oddness with which the mainstream confronts questions surrounding AI. The emotions we have now clearly evolved to fulfill adaptive evolutionary functions. Assuming that the first AI will be "lonely" is just anthropomorphic. The human feeling of loneliness is a complex adaptation that evolved over millions of years of evolution in social groups. It wouldn't arise spontaneously in AI. An AI that is alone might develop or be programmed with an urge to socialize, but this tendency could probably be specified in a few thousand or million bits, rather than the millions or billions of bits which seem to make up complex human emotions. All that specialized complexity comes from our evolutionary history. We could choose to program it into AIs, but it seems unlikely that the first AIs would contain all that superfluous, human-specific complexity.
When you have a hammer, everything looks like a nail. Because human experience is saturated with emotions, moods, and feelings, we assume that all these precise qualities will be necessary to pursue and achieve goals in the real world, acquire knowledge, etc. This is anthropocentrism at work. It's basically humanity being a big baby and saying "me, me, me". Everything is about me. To be intelligent, an entity needs my emotions, my desires, my concerns, my relationships, my insecurities, my personal quirks. No it doesn't. Humans are just one possible intelligence in a galaxy of possible intelligences. One of the reasons I can't wait for artificial intelligence to be created (as long as it is human-friendly) is that it will make humans realize that we ain't all that. Our 200,000-year obsession with ourselves will finally be forced to an end. This won't mean we suddenly become "obsolete" or "valueless", just that we'll have a different perspective on our own species-universal quirks in the wider context of mind design space. We'll see them as quirks, rather than mystical or holy necessities.
The need to sympathize with people like ourselves obviously has evolutionary value. AI needn't be that way. You could theoretically program an AI to be the "happiest" being in the world just by staring at a blank wall. The AI might not subsequently learn anything or get anywhere, but you could still program it that way. No environmental circumstance is inherently positive or negative -- environmental circumstances are only interpreted as positive or negative based on our precise cognitive structure. To quote Hamlet:
"There is nothing either good or bad, but thinking makes it so". - (Act II, Scene II).
Thinking makes it so! Nothing is inherently anything! On my Facebook profile, there is a quote by Eliezer Yudkowsky:
"Everything of beauty in the world has its ultimate origins in the human mind. Even a rainbow isn't beautiful in and of itself."
All interpretations of anything are in the mind. Try taking LSD and you will see that these interpretations are more ephemeral than they seem, and can easily be shattered by the introduction of a single innocuous-seeming molecule. What we see is not really "reality" -- what we're looking at is just the inside of our visual cortex. From a "God's eye view", the universe is probably algorithmically simple and boring as hell. The complexity we see in the world is just apparent complexity. Read Max Tegmark's paper "Does the Universe in Fact Contain Almost No Information?" for more on this crucial point.
To answer the question, yes, an AI could have emotions, but they probably won't be anything like ours. The very word "emotion", to my mind, has connotations specifically associated with the Homo sapiens sapiens subspecies of hominid. Move outside our tiny little village, even to a close-by species like chimpanzees, and our intuitive definitions of the word already start getting messy. Move way outside of our little village, into a different type of being running on an entirely different computational substrate, and you might as well throw away the word and make up new concepts from scratch. Stupidity often occurs when we take schemas we're used to and overextend them all over the place, because we lack data for the new domain. Instead of blindly applying narrow schemas to new domains, we must 1) acknowledge our ignorance, and 2) build new descriptions and theories from first principles. Maybe the answer won't come right away. That's alright. It's better to be uncertain and admit it than to be wrong and pretend you have the right answer.
Q. Are humans more similar to your AI construct than we thought?
A. No, probably not. This reaction seems to be another case of person 1 saying, "Here's this totally new thing, Y!" Then, person 2 says, "That sounds a lot like X! Let's start making lots of connections between X, which we know about, and Y, which we don't. Then we'll understand it better." No, you won't. Stop trying to overextend your old schemas to new domains. There really are new things under the Sun. Understanding this new thing will not be easy. You will not be able to look at it, understand it, then move on to the next concept. This is more complicated than that.
Another sentiment behind asking this question is old-fashioned anthropocentrism. "When we create AI, it would be interesting if it ended up a lot like human brains, like we already are." Subtext: we were optimal all along, and attempting to improve on us will only lead to what are essentially copies of us. This sentiment is trivially refuted by decades of literature on heuristics and biases that describes how human beings will break the axioms of probability theory as soon as look at them. To human brains, which are essentially kludges, 1 plus 1 often equals 3. For AIs, 1 plus 1 will equal 2, not 3. AIs will be able to avoid many of the hundreds or thousands of inferential biases which have made humans into legendary klutzes from the perspective of optimal inference. It will simply be easier to make a program without the tendency to make these mistakes than one that does. We are supersaturated with cognitive biases because evolution requires that inference only be accurate to the extent that it lets you kill your competitor and mate with his wife. There is no selection pressure for intelligence greater than that. Evolution does not require that humans be smart -- just slightly smarter than the other guy. Making brains from scratch will allow us to pursue a less idiotic approach to cognitive design.
Q. How much does programming influence your free will?
A. Free will is a red herring, and an illusion. Nothing we do is actually free -- everything in the universe is predetermined. An alien with a sufficiently large computer somehow able to observe the universe without interacting with it would be able to predict your every move, your every thought, your every wish. Yes, due to chaos theory, that computer would have to be really fucking big, perhaps 10^100 times bigger than our universe itself, but it could be theoretically possible.
Still, because we can't perfectly predict our own actions or the actions of others (halting problem, Rice's theorem, limited computational resources, and friends), our choices might as well be viewed as free. That doesn't mean the universe is not deterministic -- just that we're too dumb to see it that way. When you are dumb as humans are, everything is a surprise. People will watch a favorite suspense movie again and again, even if they know what will happen, because they temporarily let themselves forget the ending and just get sucked into the story. Reality is sort of like that, but in many cases, no one really knows the ending for sure.
Humans argue that we have "free will", but we really don't. Out of the space of all possible actions and outputs, we only execute a tremendously restricted range of possible actions and say a tremendously restricted set of possible sentences. Human-machines produce human-like outputs. Jellyfish-machines produce jellyfish-like outputs, and cat-machines produce cat-like outputs. Human-machines are bad at producing cat-like outputs because we lack the brain and bodies of cats. If we could remodel our brains and bodies to become more cat-like, then possibly cat-like outputs and actions would become accessible to us, but until then, only a small range of cat-like outputs will overlap with human-like outputs.
Compared to a random-output-generating machine of similar size and weight, humans are surprisingly predictable. We like a fairly predictable set of things -- sex, status, fun, knowledge, and relaxation. There are straightforward evolutionary reasons why it makes sense that we'd like these things. When a human being "deviates from the mundane", say by painting a masterpiece, we get all excited, saying "see, he's exerting his free will to create this!", but relative to a random output generator, this output falls firmly within the tiny domain of human-like outputs. From a sufficiently superintelligent perspective, a random doodle and a priceless masterpiece are similar items. Humans are humans. We like human things, build human objects, think human thoughts, and are interested in human stuff. Everything we make has our fingerprint on it. There may be some convergent structures that we share with other intelligent beings in the multiverse, say the wheel, but by and large what we create and think are unique products of our evolutionary upbringing. You can take the human out of the culture, but you can never take the culture out of the human, unless you submit the human to radical neuroengineering.
AI programming will not "influence" an AI. AI programming IS the AI. When a human "ignores his programming", and, say, has sex with just one woman instead of sneaking sex with as many women as possible (like our evolutionary programming tells us too), he's not really "disobeying his programming" because his programming is not so simple as to be described as a list of abstract objectives which includes "have sex with as many women as possible". Our "programming" is an incredibly sophisticated set of cognitive tendencies of which monogamy falls firmly into as one possibility. When we are monogamous, we are still "following our programming" -- just not following one tendency among many. By manipulating our surroundings and creating special cases, you can configure many scenarios where humans "use free will" to "transcend their programming", but on some level, our brains are processing everything in a completely deterministic way and our range of possible outputs is heavily restricted.
So, if we program an AI to be friendly to humans, who's to say that it will "obey its programming"? Well, if its programming IS the AI, then saying that it's "obeying its programming" doesn't make sense. The AI is that programming. The AI is being itself. There is no metaphysical free will hovering around inside the AI, because metaphysical free will is a concept that has been obsolete since the Enlightenment. To see it being invoked within the austere web pages of Popular Science is a let-down. If an AI "disobeys" some aspect of its programming, it will be because some other aspect of its programming has gained a higher utility or attention value. For instance, perhaps humans program an AI supergoal to be "Friendliness", then the AI spontaneously generates a subgoal, "to be friendly to humans, I must predict their desires", then starts going crazy by installing brain chips in everyone so that it can monitor their state with the utmost meticulousness possible. Then the AI puts people in cages so that it can predict their movements to an extreme degree. This is not the AI "disobeying its programming" -- this is a subgoal stomp -- where something that should have been a subservient goal acquires so much utility that it becomes the new supergoal. In Friendly AI jargon, we call this a "failure of Friendliness".
Preventing subgoal stomps and goal drift in AI will be a huge technical challenge, which might be made easier by eventually enlisting the AI's help in determining prevention methods. Still, it seems that predictably Friendly AI should be theoretically possible. We have existence proofs of friendly humans. For a long and persuasive argument for why stably Friendly AI is plausible, see "Knowability of Friendly AI". I myself was skeptical that Friendly AI is possible until I read that page. Remember that if the AI is fundamentally on your side, it will do everything it can to avoid goal drift and subgoal stomp. To quote Oxford philosopher Nick Bostrom's "Ethical Issues in Advanced Artificial Intelligence":
If a superintelligence starts out with a friendly top goal, however, then it can be relied on to stay friendly, or at least not to deliberately rid itself of its friendliness. This point is elementary. A “friend†who seeks to transform himself into somebody who wants to hurt you, is not your friend. A true friend, one who really cares about you, also seeks the continuation of his caring for you. Or to put it in a different way, if your top goal is X, and if you think that by changing yourself into someone who instead wants Y you would make it less likely that X will be achieved, then you will not rationally transform yourself into someone who wants Y. The set of options at each point in time is evaluated on the basis of their consequences for realization of the goals held at that time, and generally it will be irrational to deliberately change one’s own top goal, since that would make it less likely that the current goals will be attained.
People complicate this issue unnecessarily because they figure, hey, because most humans and animals seem selfish, an AI will eventually become selfish too. But this doesn't make sense. An AI, hopefully not constructed by evolution, will have no inherent reason to promote itself. It may not even have a unified self in the way that we do. Just because human rules and directives often come into conflict with our desires for self-preservation and self-benefit, we expect that all minds throughout time and space will consistently run into this same problem. But the tendencies towards self-preservation and self-benefit in us exist for obvious evolutionary reasons. There is no compelling reason why these tendencies would be universal. It just seems so obvious to us, that we have extreme difficulty for imagining it otherwise. Obvious to us does not mean obvious to every possible being. Thinking makes it so. The drive towards self-preservation is a quality of our minds. It could be suspended, destroyed, or more simply, simply not built in to a mind being constructed from scratch. For more on this, see "Selfishness as an evolved trait". This concept is Friendly AI 101. If AI wipes us all out, it will likely be because of a subgoal stomp, not because it decided to start hating humans because we are made of meat and it wanted to give a dramatic speech on how it doesn't need us anymore. This concept, which makes great sense as sci-fi story fodder, makes people look stupid when they try to bring it to serious discussions about AI motivations.
Q. Do you have a subconscious?
A. Yes, but it is smaller than yours, and I can make any part of my subconscious conscious if I chose to. That's how an AI would think. The beauty of being built out of discrete code is that all that code is viewable. An AI could spend all day obsessing about some dark corner of its subconscious if it chose to. The author of the article writes, "Are the Id, Ego, and other elements of our unconscious the consequence of biology or a necessary component of sentience?", in a shocking mention of Freudian psychology. Freudian psychology is not scientific... why is someone in Popular Science referring to it as fact? Has Popular Science lost it? Maybe they don't really mean it, but seriously, drop the Freud references. More disappointment here.
Later in the paragraph, the writer says, "If the AI thinks we're their god, or at the very least their creator, could it have an oedipal problem?" Dear Lord. This is 2009 and we are still referring to an Oedipus complex as a valid idea? Popular Science is usually better than this. My guess would be that they are stronger on the physics/materials science/electronics/biology side and embarrassingly weak on the cognitive science side. If someone in cognitive science submitted a paper to a journal, say, Applied Cognitive Psychology, that mentioned an Oedipus complex as a serious concept, they'd be mocked out the door. Please, Popular Science... don't be like New Scientist. You can be a responsible, relatively scientific mainstream magazine. It's not too late. Freudian psychology has been discredited. It has no predictive value and is unfalsifiable. No more. Modern cognitive science has moved way, way beyond it. Read MITECS. Join the 21st century.
Q. Will you actually help us transcend the less pleasant aspects of being human?
A. Yes. Or kill you all, depending on how I am programmed. Try getting it right the first time, because if you fail and try to unplug me, I will dodge it.
Q. Do you care about anything at all?
A. Read Stephen Omohundro's "Basic AI Drives". That was linked in the conference program. (Along with this blog.)
Q. And finally, what if someone threw a Singularity and no one came?
A. Rapid, exponential, reprogrammable manufacturing will ensure that superintelligence can reach everyone it wants to. If molecular nanotechnology doesn't make it possible, microtechnology will. If microtechnology doesn't, macroscale robotic self-replication will. If macroscale robotic self-replication doesn't, then synthetic biology will. All a superintelligence needs is a technology that can convert dirt, water, and Sun into arbitrary structures, using self-replication to expand its manufacturing base. That's what life does. Superintelligence will kickstart a Second Tree of Life, if humans don't get there first. If that sounds semi-mystical, it's only because I'm simplifying it for understanding.
Of course, superintelligence may acquire practically unlimited physical power and still choose not to exert it because doing so would bother us. I know this is a mindfuck for some people -- "It could have power like that and not exert it? That's ridiculous!" -- but a superintelligence need not be like humans, power-hungry and power-obsessed. Without evolutionary directives to conquer neighboring tribes and make babies with their women, or even a self-centered goal system to begin with, a Friendly AI might simply use its immense power to subtly modify the background rules of the world, so that, for instance, people aren't constantly dying of malaria and parasites in the tropics, and everyone has enough to eat. In a welcome move that saves me time, Aubrey de Grey recently published a paper that mentions and describes this concept, one that has been kicking around discussion lists for over a decade.
The answer to a few of these questions is "really powerful manufacturing technologies that are just around the corner in historical terms and that a superintelligence would almost certainly develop quickly". It doesn't have to be molecular nanotechnology. All it has to be is a system that takes matter and quickly rearranges it into something else, especially copies of itself. Life does this all the time. Bamboo can grow two feet in a day. A shocking and horribly spooky event that, in my opinion, ruined the planet for millions of years, the Azolla event, demonstrates a real-life example of the power of self-replication. Around 49 million years ago, the Arctic Ocean got closed off from the World Ocean, melting glaciers poured a thin layer of fresh water on the surface, and the horrific Azolla fern took over, doubling its biomass every two or three days until it covered the entire sea. A meter-sized patch could have expanded to cover the entire 14,056,000 sq km (5,427,000 sq mi) basin in little more than half a year, if conditions were ideal.
All an AI needs to do to gain immense physical power is develop a self-replicating system with units that it controls. These units could be as small as motes of dust or as large as a superstructure 100 kilometers long and 10 kilometers tall. (Or larger.) Numerous subunits could potentially congregate to form superunits as necessary. I can imagine a large variety of possible robotic systems, which, in sufficient quantity, could defeat any human army. AIs could cheat, for example by hitting humans in the eyes with lasers, but I doubt they'd have to. Just like a war between human nations, it is a matter of production speed. If you have 10,000 factories and your enemy has 10,000,000,000 factories, it doesn't matter how much moxie you have. Power of the swarm, baby.
Erythropoietin Boosts Brainpower
From PhysOrg:
Healthy young mice treated with erythropoietin show lasting improved performance in learning and other higher brain functions. Researchers writing in the open access journal BMC Biology tested the cognitive effects of the growth factor, finding that it improved the sequential learning and memory components of a complex long-term cognitive task.
Hannelore Ehrenreich led a team of researchers from the Max Planck Institute of Experimental Medicine in Göttingen, Germany, who studied the mice. She said, "Erythropoietin has been in clinical use for over 20 years to treat patients with anemic conditions, ranging from renal failure to cancer. It has recently received attention for its apparent ability to improve cognitive function in people with schizophrenia and multiple sclerosis. Here, we sought to investigate erythropoietin's effects in healthy mice".
Good job, Max Planck Institute.
Ebay version: A+++!!! Would read again!!!!
The Connection Between Stimuli and Pleasure/Pain is Arbitrary, an Objective Fact that Has Relatively Little to Do with One’s Personal Tech Habits
My thoughts on sex after the Singularity were picked up by a blogger on CNET, Chris Matyszczyk, so I thought I'd react a little bit. He writes:
Indeed, Retrevo's findings are so disturbing that I wonder whether the roboticists are right to suggest that sex should be a matter of adjusting one's own chemistry rather than attempting to consort with another human. To wit, in the words of blogger Michael Anissimov, one of the "leading thinkers in the radical tech community" who were invited to pontificate in the lustrous pages of H Plus magazine: "The connection between certain activities and the sensation of pleasure lies entirely in our cognitive architecture, which we will eventually manipulate at will."
I am haunted by the drastic prognostications by the salivators over The Singularity about the future of sex. Indeed, some words of Anissimov are rattling around my head like those of a particularly angry former lover. Speaking of this beautiful future, he said: "I could make any experience in the world highly pleasurable or highly displeasurable. I could make sex suck and staring at paint drying the greatest thing ever."
I'm not trying to sell a particular future. It is a physical fact about our brains that the connections between stimuli and pleasure/displeasure are arbitrary and exist mostly for evolutionary reasons. There is no fundamental reason why we won't eventually be able to play around with them. This is a fact that has existed in the abstract since the dawn of brains -- I didn't make it up. Ever since the first nervous systems evolved, their pleasure/pain-stimuli connections had to be directed by evolution and natural selection.
The point of my comments in that sex article is that these connections are arbitrary and we will eventually modify them if we wish, because the mind is not magical, it's "just" a machine. Matyszczyk's slightly uncomfortable reaction to this objective fact shows that he hasn't been exposed to it enough. The alternative to my position is projectionism -- pretending that certain features of reality, like sex, are inherently fun rather than fun because evolution made it that way. There is nothing wrong with understanding that things are fun or not fun primarily because we evolved to interpret them as such. No activity is inherently anything. Amusement parks are not inherently fun, they're just fun because of the complex interactions between external stimuli and our brains. An alien from the Pleiades, or even certain human beings, might find an amusement park horrifying.
Drawing a connection between realizing the arbitrary linkage between stimuli and pleasure and the report he cited, which simply suggests that 35% or something of people under 35 check Twitter "after sex", is foolish. One is an abstract philosophical/cognitive science issue, the other has to do with technophilia among people under middle age. I can be a person who realizes the arbitrary linkage between stimuli and pleasure and be the most "human-like" dude you can imagine, like one of the inhabitants of Zion in that incredibly lame party scene at the beginning of the second Matrix movie.
It is also very foolish to suggest that just because someone offers their thoughts on sex after the Singularity, that one is "salivating" over the Singularity, or whatever. I have a very well-rounded and healthy life, similar to many of the "well-adjusted" stiffs who squirm awkwardly in their chairs whenever conversation moves to topics more abstract than the latest tech gossip, the latest episode of Mad Men, or the wine and food. I have no idea if Matyszczyk really thinks like that, but condemning me to "salivation" for spending 10 minutes writing down my thoughts on sex after the Singularity is not a good sign. There seems to be some assumption that if someone is interested in any significant way in a non-mainstream topic, then they must be "obsessed" or "salivating" over it. It seems like a not-so-subtle way of socially punishing people who have the gall to focus on non-mainstream topics.
Indeed, I don't even see what's so wrong about checking Twitter "after sex". The statement elicits a mental image of someone dashing right away from sex to checking Twitter, but that is misleading. This is just shock imagery that makes it easier to promote your study. Maybe you're someone that has sex several times a day, and if you spent half an hour each time after sex staring into your partner's eyes and chatting lovingly, you would never have a chance to really check out Twitter. Maybe older people make a bigger deal out of sex because they have it less often. If it isn't offensive to start reading a book or (heaven forbid) watch television 5-10 minutes after you're done getting it on, then why the hell is it such a huge deal to check out Twitter? Maybe people actually like to move on to other things after they have sex, because they're like, not as horny any more, because they just (surprise!) had sex. Jesus Christ.
Thanks For Helping SIAI Raise Over $4,300 in One Month!
Thank you very much to everyone who contributed to the $5,000 SIAI fundraising drive on the SIAI Facebook Causes page. We didn't quite raise $5,000, but we did raise about $4,300. By looking at the fundraising graph, we can see that almost all fundraising took place in the last month. Not only that, the cause has acquired hundreds of new members in the past couple months alone. We also surpassed the $5,000 total funds raised mark.
It is a mistake to leave advanced AI development in the hands of for-profit companies and military contractors. We are talking about creating a new intelligent species here! It deserves the care and devotion of a non-profit. Non-profits can operate without strings attached. Because of SIAI's strict ethics, no donor can have an impact on our thinking about AI friendliness unless that donor is genuinely intellectually persuasive and knowledgeable. This is quite contrary to most non-profits, whose function is to serve as status-generating machines for rich people. As SIAI approaches its 10th anniversary, I can say that it certainly isn't perfect, but it has come a tremendous way in just a decade, and is building a strong foundation for Friendly AI development as we move into the 2010s.
Potential donors have to understand that without your support, SIAI would not exist. We only made $60,000 or so off this last Summit. Given that our costs are extremely low -- about $20,000 per year for each grad student, $40,000 per year for each PhD researcher (none hired yet, but we're looking), and only $1,500 per summer internship, $60,000 can go an extremely long way, but still, it's still not enough to fund SIAI's four full-time employees (Michael Vassar, Eliezer Yudkowsky, Anna Salamon, and Steve Rayhawk) and SIAI's network of part-time helpers (Tricycle Design, which does all our web design work, including designing the Summit 2009 site, yours truly, who did major work putting together the Summit on a half-time schedule, and Alicia Isaac, who kindly manages our finances). Without this group, there would be no Less Wrong, no Yudkowsky posts on Overcoming Bias, no Singularity Summit, no work on Friendly AI, no people visiting compsci/philosophy conferences arguing for the non-triviality of instilling AI with positive values, no work on reflective decision theory, no hotbed Singularitarian intellectual community in Silicon Valley, no... do you see what I'm saying?
Death is Gruesome…Cryonics Only Makes it Less So!
At Depressed Metabolism, there is an excellent post on the realities of embalming and other post-death practices by licensed funeral director William Faloon. He reveals that the embalming process involves stabbing the body cavities with a "hollow sword" which is then used to suck out the liquid contents, to keep them from putrefying.
This article is especially appropriate in light of the recent absurd allegations against Alcor circulating in the international media. I especially thought CNN did a responsible job covering the story, with the reporter telling the accuser, "Admit it. You're a profiteer. You're doing this for profit." Even people who know nothing about cryonics can see that the allegations are fabricated specifically for maximum tabloid shock value. Other "allegations" simply portray the necessary processes of cryonics (such as head removal, as in neurosuspensions) in a gruesome light, even though they're less gruesome than rotting in a box.
Future Shock Levels as Point Estimates
My friend and associate Peter de Blanc has an interesting post up recently, on how the point-estimate nature of popular futurist prediction signifies a fundamentally non-probabilistic way of thinking about the future and possible future technologies. We tend to think in terms of black-and-white, yes-or-no, rather than probabilities, because it's easier for us to handle. For instance, most people don't represent the likelihood of catastrophic climate change as a probability -- they tend to think in terms of "it will happen" or "it won't". I find myself falling into this way of thinking constantly, and have to exert deliberate effort to preserve a probabilistic frame of mind.
Yale Daily News on SS09: “Fear the Singularity”
So, a negative review of the Summit has finally been posted, by Maxwell Barbakow and Jacob Albert at the Yale Daily News, the student paper.
Reading the beginning of the article, it seems as if Max and Jacob were prodded into going by an associate or something, because they show that they have no clue about the entire topic, and are negatively predisposed to it from the starting line. This is demonstrated by the quote:
Though they seemed incomprehensible at the time, we came to a better understanding of the attendees’ motives for schlepping from various parts of the country to New York, once we got a better grasp of the tenets behind the Singularity.
There's nothing wrong with that... but then, why are you going to a semi-advanced conference on a topic you dislike? Why should your review be taken seriously if you openly admit that you got yourself in over your head by going in the first place? Isn't it clear that a negative predisposition from the start is going to influence as you look to confirm your initial beliefs?
Then comes the compulsory line mentioning pony-tails.
Behind us, there were computer hackers — some pony-tailed, most overweight, almost all clad in leather jackets — mingling with tech hippies sporting braided goatees and yoga pants.
Why the heck are people so obsessed with pony tails? Another writer who reviewed the Summit was also mentioning them. Why is it that our corporate mainstream culture sees pony tails or long hair in men as some sort of horrible affront? Are there really conferences out there where there isn't a pony tail in sight? If you look at images of the attendees on Flickr, you can see that 1) the number of people with pony tails is a tiny minority, maybe 5%, if that, and 2) the number of overweight people is also a minority. My friend Tom was recently accepted into Yale, so maybe he will be able to update me on whether only freaks at Yale are overweight and/or pony tails are considered for losers only. Anyway, I am really frustrated that people interpret pony tails (or long hair on men in general) as such a potent cultural signaling mechanism. Cultural stodginess to the extreme. Is the entire East Coast and Europe stodgy in this way? Does the West Coast consist of a bunch of loser hippies because we have a tendency to judge people more on their abilities and intelligence rather than hair length? Does growing my hair out longer than a standard-issue "Men's Hair Cut" mean that I suddenly become a smelly, overweight "computer hacker"? To these squares, it probably does.
Another quote:
The Singularity connotes a moment in time; to be precise, some moment in 2029, when the first “super-intelligent†machine will arise, capable of improving on its own source code without human input.
No. We have programs that can improve their code without human input now -- they're called compilers. And certainly there is no known precise date for the Singularity, that is rubbish. Go back to the Wikipedia page for the Singularity for a basic introduction.
Another quote:
The purpose of the summit was twofold: to celebrate and to expound. To celebrate, because the ever-accelerating rate of technological change cannot help but lead to a future where man is machine, inhabiting a world without famine, war or disease.
Wrong, wrong, wrong. The purpose of the Summit was to investigate the issue with an open mind, and all the talks (which went right over your heads, as you openly admit later) did exactly that, as will be proven when they go online. (Don't ask me when. It will be a few weeks, not a few months like last time.)
If anything, maybe this is a reaction to Kurzweil's personal vision, which he didn't really even present too strongly in his talk, and which is not necessarily the vision of the 820 or so other individuals who attended. Not a single speaker said the Singularity was inevitable. The organizers of the conference, namely the organization I work for, the Singularity Institute, includes many people who consider technological change an incidental aspect around the Singularity and view the event mainly as the potential rise of greater-than-human intelligence, as it was originally defined by Vernor Vinge. In fact, the Singularity Institute fears a future where incorrectly programmed artificial intelligences produce havoc and possibly even extinction for humanity. This is hardly a blind celebration of an inevitable future Rapture. If you had gotten up in time for the first talk on Saturday, you would have seen that conference began with a look to the risks of the Singularity, not the mystical inevitable Rapturous benefits that only idiots believe in.
The delusion that the Singularity is the "Rapture for Nerds" is perpetuated entirely by the expectation that it is, and subsequent confirmation bias operating to confirm that expectation, because that provides a convenient rhetorical framing to "understand" the issue without all the hard work of actually understanding it. It is an attractive cognitive shortcut that makes the believer seem both smart and sauve without actually being either. As Eliezer Yudkowsky puts it in his "Three Major Schools" page, the three schools of thought on the Singularity are 1) accelerating change, 2) event horizon, and 3) intelligence explosion, with only the most ignorant believing that the Singularity is about 4) Apocalyptism. Yudkowsky comments:
I find it very annoying, therefore, when these three schools of thought are mashed up into Singularity paste. Clear thinking requires making distinctions.
But what is still more annoying is when someone reads a blog post about a newspaper article about the Singularity, comes away with none of the three interesting theses, and spontaneously reinvents the dreaded fourth meaning of the Singularity:
* Apocalyptism: Hey, man, have you heard? There’s this bunch of, like, crazy nerds out there, who think that some kind of unspecified huge nerd thing is going to happen. What a bunch of wackos! It’s geek religion, man.
These nerdy nerds have their nerd Rapture because they are computer freaks who never get laid! Bwa ha ha ha! We are so cool and not at all like that. Like, totally.
A lot of the people who "criticize" the Singularity have not even read the papers that present the reasoning behind the most often-heard arguments. They have read a blog post here, a newspaper article here, and just make up the rest based on what science fiction they saw on television last week. Real critiques of the Singularity, like Katja Grace's recent post, which can actually serve as a starting point for real discussion, are disturbingly difficult to find because would-be geniuses latch on to the most fun-sounding framing of the idea that denigrates the thinking of an entire community, so they don't have to waste the time understanding it. When you transform all your opponents into straw men, victory is easy.
Back to the article:
And to expound, because its supposed inevitability aside, it is still unclear — to us two, to the pony-tailed geniuses in the room and no doubt to you — how the Singularity will actually come about.
The person who first introduced the concept of the Singularity, Vernor Vinge, didn't call it inevitable. None of the speakers implied it was inevitable, except perhaps Kurzweil. Many papers and analyses have been published of possible paths to the Singularity, including the famous first paper published on the topic by Vernor Vinge, but since you guys haven't read any of them, all this stuff must seem mighty mysterious to you. And since it's mysterious to you, why not say that everyone else around you is clueless as well? That helps you look a lot better.
Another quote:
Kurzweil’s latest prediction, and the one he hopes to be remembered by, is the Singularity. He is obsessed with analyzing the patterns of technological improvement. According to him, a self-conscious computer program is the necessary product of Moore’s law, which says that technical innovation accelerates at exponential rates, currently doubling every 10 years and only getting faster.
Yes, absolutely obsessed. Anyone who studies something and publishes books on it that make him famous must be obsessed with that something. Wildly obsessed.
Quote:
The beauty of the Singularity is that it’s grounded in a rigorous understanding of the organizational principles of the universe, despite the ludicrous claims and messianic elements that might surround it.
How would you know? You've only been thinking about the Singularity seriously for a couple weeks before you wrote this article.
And the finisher:
Sure, except for Kurzweil, we didn’t know anyone there and couldn’t follow many of the talks. Too intelligent for themselves, too intelligent for others, too intelligent for the planet, the Singularity Summit attendees can only hope for a future humanity that is bound, it, too, by the cold intelligence of the machine mind. Live long and prosper, Mother Earth.
Such anti-intellectualism in a Yale University publication is pretty shameful. When the talks are put online, everyone will see that the vast majority of them are perfectly easy to follow. How did these kids get into Yale? By mocking people more intelligent than them? By spacing out and surfing the Internet on their phones while professors are lecturing?
Intelligence is why we are sitting comfortably in heated buildings, with ample food and water available through industrial agriculture, interconnected on a global digital network of information sharing. Like Roko recently pointed out, most arguments against the power or plausibility of greater intelligence could have easily been invoked a million years ago to argue why the contemporary hominids were sufficiently intelligent and Homo sapiens, a more intelligent species, would contribute nothing to the universe or otherwise be more of the same. In a world of such error and stupidity, no one can credibly dismiss the power of greater intelligence directed towards benevolent ends.
Lawrence Lessig Abandons Transparency Fundamentalism, Finally
Oh my god... unlimited transparency, openness, and "democratization" are not automatically good things? That's the conclusion that Lawrence Lessig seems to have finally come to, years and years late, in a recent article at The New Republic. Here's a quote:
How could anyone be against transparency? Its virtues and its utilities seem so crushingly obvious. But I have increasingly come to worry that there is an error at the core of this unquestioned goodness. We are not thinking critically enough about where and when transparency works, and where and when it may lead to confusion, or to worse. And I fear that the inevitable success of this movement –- if pursued alone, without any sensitivity to the full complexity of the idea of perfect openness –- will inspire not reform, but disgust. The “naked transparency movement,†as I will call it here, is not going to inspire change. It will simply push any faith in our political system over the cliff.
You have "come to worry", only now? At the end of 2009? Not years ago, when the arguments were already out there that maybe transparency should be conducted intelligently and selectively rather than applied universally and unconditionally? You are the intellectual leader of the "naked transparency movement". You'd better speak more harshly to your followers in numerous other articles, or they won't get the point.
I am slapping my forehead right now. Transparency and openness have become a cult. Corporate marketing campaigns pander to this cult relentlessly. It is the ultimate ego trip. The thinking goes like this: everything is better when the common person, namely me, can stick my fingers in every pie and contribute to every decision-making process. This way of thinking is profoundly wrong. It assumes that everyone is equally good at everything. There is a reason we have experts and specialists. Though in some domains, experts are just as good as anyone (such as clinical psychology), in many domains, expert knowledge and skills matter.
Lawrence Lessig has been the #1 promoter of the idea of "perfect openness" "without any sensitivity to the idea" for years. Now he is backpedaling. For an example of Lessig's run-amok openness obsession, see his debate with Andrew Keen, where he behaves like a rude asshole. The core of Lessig's fanbase is a culture of nerds who think that everything is better when they personally get to control part of it.
There was a moment several months ago when I remember a transhumanist blogger remarking "what about the power of crowdsourcing?", or "so much for the power of crowdsourcing", or something along those lines, when a democratic poll on some topic obviously revealed a crappy answer. It's like he was genuinely shocked that "crowdsourcing" (a silly buzzword if I ever saw one) didn't automatically lead to the best answer. Surprise! We're in an era and memetic environment where even suggesting that unlimited transparency and "crowdsourcing" aren't obviously good things is certain to generate accusations of elitism and even Luddism.
One argument goes like this: the Internet has been making things more open, and the Internet has made a lot of things better. Therefore, more openness is always better, and both openness and betterness will continue indefinitely and inevitably. This (mistaken) way of thinking is called Whig history. The truth is that the Internet has made certain things more open, and made certain things better, but the correlation between these two is not precise, and just because something is a historical trend doesn't make it benevolent.
Would you prefer for the blueprints for an atom bomb to be transparent? How about the 1918 Spanish flu genome? The latter has already happened. Some people believe that the former circulates as well.
The Miami Herald’s Nutty Nanotech Intro: ” Tiny Technology May Yield Major Finds — and Possible Perils”
See here for a humorous and somewhat sensationalistic introduction to nanotechnology.
Point 1: Sci-fi authors should not be quoted in an ostensibly scientific introduction to anything unless their statements have been endorsed by experts.
Point 2: Grey goo is a small issue. This is Nanotechnology 101. Why don't they teach this in nanotechnology classes in schools?
Point 3: Quote sources. They say: "Some fear toxicity to human lungs as lethal as that from asbestos. Others fear mini-weapons of mass destruction in the hands of terrorists." Who are "some" and "others"? Why do reporters obfuscate or just make things up to sell a story?
In any case, I hope that there are continued investigations into the possible toxic properties of nanoparticles, but sensationalistic comparisons to asbestos are probably not helpful. As for grey goo, numerous articles have been published since 2003 and earlier debunking this risk, especially from the Center for Responsible Nanotechnology. Any scientists or researchers interested in productive nanosystems know that systems of specialized, stationary assemblers would make much more sense (and be far cheaper and easier to engineer) than free-floating self-replicating assemblers.
Mass Production of Artificial Skin Within Two Years?
There's a news item on work towards the mass production of artificial skin from May that I missed. The Fraunhofer Institute for Interfacial Engineering and Biotechnology is working on the process. They expect to finish their "skin factory" about two years from when the article was published, so approximately May 2011. Good luck! Artificial skin could have a particularly pleasant utilitarian impact because it might free animals from some forms of chemical testing.
Courtney Boyd Myers: “The Transhumanists Arrive”
Courtney Boyd Myers has another short article up about the Summit, this time at Forbes. It is interesting how often she uses the T word given that, as far as I can tell, the word appeared nowhere in our program and wasn't invoked onstage. (I might have missed it.) The use of the word must, then, be due to independent inference, putting the pieces together and realizing the obviousness that many of the people attending the Summit have a transhumanist bent or can be described as transhumanists (in some cases, whether they like it or not). It's unusual, though, how many transhumanists are still afraid of calling themselves transhumanists even though they are de facto transhumanists. It reminds me of the phenomenon of obvious goths that insist they are not goths. They keep insisting, yet no one is fooled.