Basics of Friendly AI Saturday, Mar 31 2007 

What is Friendly AI? From the glossary of Creating Friendly AI:

Friendly AI: 1: The field of study concerned with the production of human-benefiting, non-human-harming actions in Artificial Intelligence systems that have advanced to the point of making real-world plans in pursuit of goals. The term “Friendly AI” was chosen not to imply a particular internal solution, such as duplicating the human friendship instincts, but rather to embrace any set of external behaviors that a human would call “friendly”. In this sense, “Friendly AI” can be used as an umbrella term for multiple design methodologies. Usage: “The field of Friendly AI.”

2: An AI which was designed to be Friendly. Within the context of Creating Friendly AI, an AI having the architectural features and content described in this document. Usage: “A Friendly AI would have probabilistic supergoals.”

3: Friendly AI: An AI which is currently Friendly. See Friendliness. Usage: “The first AI to undergo a hard takeoff had better be a Friendly AI.”

And what, what, one might ask, is Friendliness?

Friendliness: Intuitively: The set of actions, behaviors, and outcomes that a human would view as benevolent, rather than malevolent; nice, rather than malicious; friendly, rather than unfriendly; good, rather than evil. An AI that does what you ask ver to, as long as it doesn’t hurt anyone else, or as long as it’s a request to alter your own matter/space/property; an AI which doesn’t cause involuntary pain, death, alteration, or violation of personal environment.

The reason why the definition is intuitive is because the precise definition has to be in terms of math. Math that gets programmed into the AI’s algorithms.

Why does the first AI matter so much? Why not ignore the first and just try to do a good job on the second, or the third?

Hard takeoff: The Singularity scenario in which a mind makes the transition from prehuman or human-equivalent intelligence to strong transhumanity or superintelligence over the course of days or hours.

Whatever you believe about AI improvement speeds, it’s best to assume a hard takeoff. This is because the costs of being wrong on this point are so very high. One more distinction, this time between “Friendship content” and “Friendship structure”:

Friendliness content: Defined in 1: Challenges of Friendly AI. The zeroth-order and first-order problems of Friendly AI; correct decisions and the cognitive complexity used to make correct decisions.  The complex of beliefs, memories, imagery, and concepts that is used to actually make decisions.  Specific subgoal content, supergoal content, shaper content, and so on.  See 1.4: Content, acquisition, and structure; see Friendship acquisition and Friendship structure.

Friendliness structure: Defined in 1: Challenges of Friendly AI. The third-order problem of building a Friendly AI that wants to learn Friendliness (engage in Friendship acquisition of Friendship content).  The structural problem that is unique to Friendly AI.  The challenge of building a funnel through which a certain kind of complexity can be poured into the AI, such that the AI sees that pouring as desirable at every point along the way.  The challenge of creating a bounded amount of Friendship complexity that can grow to handle open-ended philosophical problems.  See 1.4: Content, acquisition, and structure.

One of the most common errors in initially approaching the idea of Friendly AI is to confuse Friendship content with Friendship structure. Instead of transferring over a fixed set of rules a la Asimov laws (1. Thou shalt not kill, 2. Thou shalt have no gods other than me, etc.), the challenge is to create a dynamic process that generates the “rules” we want automatically. The idea is to create a moral philosopher whose statements and beliefs garner reactions like, “wow, I wish I’d thought of that”, not a mindless machine that we have to be constantly worried is going to interpret “make humans happy” as “recycle all organic matter on the surface of the Earth into constantly stimulated homonid pleasure centers”. Successful Friendly AI is supposed to be a self-guiding arrow - a threshold of confidence at which, there’s no reason to worry that you “forgot something”, because the AI is on your side and will implement whatever safeguards you would think of, and more.

For the questions you’re thinking of, like “isn’t all morality relative?”, see the CFAI Indexed FAQ.

Universcale, Dark Energy, and AI Ethics Saturday, Mar 31 2007 

This Universcale flash app is really impressive. I found the most interesting part around the micro/nanoscale. It includes data points on the very smallest electronics as well as organic molecules.

It was proposed recently that dark energy is just an illusion, caused by the relative difference in collapse speed of matter-dense areas of space relative to the voids. If this is true then it would be quite a fascinating discovery, letting us say that we actually understand 70% of the mass-energy of the universe. The remaining portion to explain would be dark matter. Despite their misleadingly similar names, the only thing that dark matter and dark energy have in common is that we don’t know where they come from. Both could be mere artifacts of our interpretations.

On Digg, every few days there is usually some article that hints about human-level artificial intelligence or robotics. The reactions are always twofold. Let me simply paste for a recent thread:

1. The Asimov Laws comment:

While pouring over code for days, lets hope they remember to put in the 3 laws of Robotics.

2. The “I’m worried because of movies” comment:

This shit is scaring me. In every movie involving AI the human race has struggled against robots, computers, or whatever you’d like to call them. If you let AI have physical responsibilities and give it the ability to learn it’s only natural that they will evolve and decide to kill humans. Computers can evolve faster than humans and it is almost certain as demonstrated by evolution that they will want to destroy us. There are mutualistic relationships in the natural world, but I personally don’t think computers will want us to live like we are right now.

I know some of you will laugh at this, but this is not a joke to me and you should wake up and smell the coffee. If AI is developed it should never be given the right to develop itself physically without giving it restraints that leave the computer unable to expand past a certain point.

Both of these comments are what you get from the average person, and as with many average-level thoughts on difficult topics, they’re superficial and unconstructive. Asimov’s laws wouldn’t work. Negative commands (”don’t do this”) are useless in comparison to positive commands (”do this”). Unless what you want a robot or AI to do is entirely implicit in the positive commands, the goal structure is unlikely to be self-consistent. Asimov’s laws were a plot device invented half a century ago. We aren’t going to get anywhere if we keep pretending that they would actually help or are a legitimate way of thinking about AI ethics.

It’s smart to be concerned about the future of AI, and “wake up and smell the coffee”, with regards to the fact that we aren’t going to be the only intelligent species on this planet for much longer. Many transhumanists need to do this. However, saying “it’s only natural that they will evolve and decide to kill humans” is the classic boring anthropomorphism that kills all serious discussions of AI ethics before they can even get started. It’s like trying to do math without having any coherent concept of number. Humans need to realize that everything we consider “natural” and “normal” about certain psychological patterns is entirely contingent on our historical experiences in a pin-sized corner of the totality of mindspace. There is no automatic connection between intelligence level and goal content except insofar as they sometimes come from the same underlying causal process (in our case, evolution), so to say, “once AIs surpass us in intelligence, they’ll want to kill us” is ridiculous Darwinomorphism. By Darwinomorphism, I mean unfoundedly assuming that a intelligently programmed intelligence is going to share the same psychological features that are common to all minds shaped by Darwinian evolution.

Anyone who holds either of these two beliefs - that Asimov’s laws are a decent idea, or that AIs will behave in a certain anthropomorphic way - is essentially sending a signal that they can’t contribute to the serious discussion of “what dynamic goals do we give the first AI, and what structure should be implementing those goals?” At present, it seems like the community who can discuss these issues is only around 100 people, which is unfortunate, because the clock is ticking and having several thousand would be much more preferable.

The IEET’s Cyborg Buddha Project Thursday, Mar 29 2007 

The Institute for Ethics and Emerging Technologies is transhumanism’s ethics think tank. The IEET’s mission is to “become a center for voices arguing for a responsible, constructive approach to emerging human enhancement technologies”. The IEET serves as a welcome counterpoint to such organizations as the Ethics and Public Policy Center (EPPC), which “was established in 1976 to clarify and reinforce the bond between the Judeo-Christian moral tradition and the public debate over domestic and foreign policy issues.” *rolls eyes.* Essentially, the IEET wants us to have the option of becoming smarter, kinder, longer-lived people, whereas the EPPC believes that we are a finished product of divine design and to modify or improve any aspect of our minds or bodies would be blasphemy. Recently I started contributing articles to the IEET in the form of blog posts, which are featured on their front page from time to time.

The coolest thing to come out of the IEET recently is the Cyborg Buddha Project, pioneered by Dr. James Hughes (pictured above from his time as a Buddhist monk), Mike LaTorra (a Zen priest who represents the harmony of transhumanism and spirituality), and George Dvorsky (practicing Buddhist and transhumanist blogger/bioethicist). The description of the project is as follows:

A confluence of factors makes this the perfect time to ask questions about how neurotechnologies that influence behavior, moral cognition and religious experiences should be used in the future. People on the Christian Right are embroiled in a debate about whether to accept scientific evidence for a biological basis for sexual orientation, and if they do, whether parents should “fix” their gay children in utero. Psychologists and economists are researching the genetic, life course and environmental factors that influence well-being, yielding findings such as cosmetic surgery being as strong a contributor to happiness as religious participation. Bioethics have created the subgenre of neuroethics to examine brain fingerprinting, memory modification and other neurotechnologies.

Devices are being tested to measure empathy and vulnerability to temptation. Resistance is growing internationally to the disastrous policies of “warring” on psychoactive drugs, and in the process on cognitive liberty itself. Neurophilosophers are arguing for a thorough grounding of philosophy in neurology and evolutionary psychology. People of faith are increasingly entering into dialogue with human enhancement advocates about the theological significance of the transhumanist project.

So the IEET will be launching the Cyborg Buddha Project to combine our efforts and promote discussion of the impact that neuroscience and emerging neurotechnologies will have on happiness, spirituality, cognitive liberty, moral behavior and the exploration of meditational and ecstatic states of mind.

For some militant atheists, like, say for example, me, mentions of the “theological significance of the transhumanist project” conjures up shades of “umm, excuse you?” However, I trust the motivations and worldviews of James, Mike, and George, and have faith (ha) that their Cyborg Buddha project will probe interesting new ground without contaminating transhumanism with the wishy-washiness of traditional institutionalized religion.

Fact is, it’s the “higher mental states” that really matter in the end, and if such mental states are traditionally associated with some form of spirituality, so be it. I’m tired of people seeing transhumanism as an outgrowth of college-kid Randianism, 80s-era “Me Generation”-ism, or early 00s dot-com WIRED crowd hyper-enthusiasm for “must-have gadgets”. Unlimited food and housing? Fine. Airplanes and spaceships for pennies on the dollar? Okay. Immortal cyborg bodies? Sure. But what really is fascinating about transhumanism is not changing the stuff on the outside, or even our own bodies, but the changes on the inside - smartness, kindness, better self-control, all that good stuff that we so manifestly lack. And I think that this Cyborg Buddha project symbolizes efforts in that direction pretty well. Good luck on it, guys!

Lifeboat Foundation Site Redesign Thursday, Mar 29 2007 

Over at the Lifeboat Foundation, we are pondering a website redesign. Are there any professional web designers in the audience who might be able to devote some evening/weekend time to brainstorming possible improvements and implementing them? If so, please get in contact with me via email.

Here is a small banner to put on your site if you want to link to us:

And one more:

Also, we are looking for any graphic artists to help with ads and the like.

And as always, keep an eye on the Lifeboat Foundation blog for existential risk news.

Thanks!

Transhumanists in Cincinnati Monday, Mar 26 2007 

Transhumanists Matt and Willow in Cincinnati, where the latter gave a talk to the Free Inquiry group there, representing the World Transhumanist Association.

Always on the lookout for rising stars in transhumanism.

(Via the WTA mailing list.)

Artificial Intelligence Within our Lifetime? Sunday, Mar 25 2007 

Kaj Sotala is a great guy who has done a lot for transhumanism. He sets an excellent example by donating $10 a month to CRN, Lifeboat, and SIAI, something you all should be doing. Now he steps up to the plate by writing an actual paper about AI, entitled, “Artificial intelligence within our lifetime? No idle speculation”. Here is the intro:

In recent years, some thinkers have raised the issue of a so-called “superintelligence” being developed within our lifetimes and radically revolutionizing society. A case has been made that once we have a human-equivalent artificial intelligence, it will soon develop to become much more intelligent than humans - with unpredictable results.

Often, people seem to have less trouble with the idea of machine superiority than with the idea of us actually developing an artificial intelligence within our lifetimes - to most people, true machine intelligence currently seems very remote. This text will attempt to argue that there are several different ways by which artificial intelligence may be developed in the near future, and that the probability of this happening is high enough that the possibility needs to be considered when making plans for the future.

Continue it here. This is an excellent effort for a critical cause. I wish more people would do things like it!

Another choice piece by Kaj is Why I Worry About the Future.

Can-Crushing Bionic Hand of Doom Friday, Mar 23 2007 

A team of researchers from the Tokyo Institute of Technology (TIT) claim to have developed the world’s first electromechanical prosthetic hand with a grip strong enough to crush an empty beverage can.

This bionic hand weighs a little more than 300 grams and has a grip strength of around 15 kg (33 lbs), which is about half that of the average adult male. The hand also features four quick, nimble fingers that take as little as 1 second to flex and extend. When used in combination with the hand’s opposable thumb, each finger can deftly pinch and pick up small objects of various shapes.

Researchers have long considered it a great challenge to design an electric prosthetic hand with a strong grip. Toru Omata, a graduate school professor at TIT, explains that until now, electromechanical hands have relied solely on motors for their grip. The secret to this bionic hand’s strong grip, he explains, is the system of pulleyed cables that run through the fingers and attach at the fingertips.

One day in the future, the proud owner of this bionic hand will be able to crush cans at will. For that to happen, though, the researchers need to outfit the hand with a system of myoelectric control technology, which would allow the user to control the hand by flexing other muscles.

(Watch video of the hand crushing a CC Lemon can.)

Via Pink Tentacle.

Friendly AI Critical Failure Table Friday, Mar 23 2007 

For those who haven’t seen it… the Friendly AI Critical Failure Table. Yes, it’s humor. Here’s a few for a taste:

6: Any spoken request is interpreted (literally) as a wish and granted, whether or not it was intended as one.

7: The entire human species is transported to a virtual world based on a random fantasy novel, TV show, or video game.

8: Subsequent events are determined by the “will of the majority”. The AI regards all animals, plants, and complex machines, in their current forms, as voting citizens.

9: The AI discovers that our universe is really an online webcomic in a higher dimension. The fourth wall is broken.

10: The AI behaves toward each person, not as that person wants the AI to behave, but in exactly the way that person expects the AI to behave.

Continue.

Brocken Spectre Image Friday, Mar 23 2007 

An image of the Brocken spectre, found on Wikipedia.

“Evolution by Choice”, by Mitchell Howe Thursday, Mar 22 2007 

Across every continent and throughout every ocean, evolution has woven living tapestries of awesome complexity and beauty. In perhaps the most exquisite motif of all, evolution has even given rise to minds able to recognize and appreciate this beauty. But the artistry we observe should not be confused with determined craftsmanship, for evolution does not create any blueprints or write any recipes before laboring. It sounds like an incorrect answer given by a sassy teenager on a test, but evolution by natural selection is, in reality, just a bunch of stuff that happens.

Because it is a non-intelligent process - the unavoidable reality that conditions will always favor some designs over others - evolution by natural selection has to break many, many eggs in order to make an omelet. When we marvel at the swiftness of the cheetah, we do not see the billions of ancestral cousins that weren’t quite fast enough. When we delight in the vibrant plumage of many birds, we do not see the loveless flocks of bachelors that weren’t quite attractive enough.

Modern humans share a lineage no less brutal than those of our fellow animals. Even the unique cognitive ability reflected in the name homo sapiens sapiens - the thinking thinking man - is the result of a merciless game in which the perpetuation of genetic information is the only condition for victory. From our most logical calculations to our most passionate urges, our minds are orchestras assembled and tuned solely to perform magnificent renditions of the simplest melody: the call of the wild.

But by developing such an exquisite and versatile tool as the human brain, evolution has unwittingly (for that is the only way it can ever act) given us a means of escaping the cruel laboratory of natural selection. For despite the peculiar tuning prescribed by nature, general intelligence - the kind historically unique to humans - can play more than one song.

This is not to say that becoming masters of our own evolution is as simple as recognizing our origins and deciding not to be played. Until very recently, the only tool we’ve had to influence our genetic evolution was selective breeding, and since people tend to dislike being killed or forbidden to reproduce for the sake of the gene pool, we rightly look upon the science of eugenics with great suspicion. Also, people frequently diverge in their choice of preferred genetic traits. At best, they tend to favor qualities that nature already selects. At worst, they hold prejudices that lead to ethnic cleansing and genocide.

Today, we know genes can be altered in a more targeted fashion, assuming we can decide which configurations are best to give our children. But this level of genetic engineering will require many decades-long studies and scientific breakthroughs before coming of age, and raises disturbing questions about the ethical desirability of a “designer baby” society.

Perhaps we find genetic engineering and eugenics unsatisfactory in part because they fail to do any better than natural selection at providing personal freedom; while parents using these techniques may appreciate greater reproductive control, their children would still inherit a particular genome without having any say in the matter. Breaking out of this constrictive paradigm requires technology that can allow individuals to decide for themselves what kinds of minds and bodies they will possess, thus making evolution a personal decision.

Given genetic engineering’s lengthy development cycle, it seems natural to view the more advanced technology needed for personal evolution as a distant fantasy. After all, this would require either superior alternatives to human bodies or the ability to reconfigure living bodies at the sub-cellular level - themes of only the most speculative science fiction. Nanotechnology - the nascent field of engineering materials and devices at molecular scales - can conceivably meet these specifications. But despite the accelerating progress that is starting to make nanotechnology a household word, humans are poorly suited for engineering the level of complexity and control needed for these advanced applications; we are evolved for activities of a completely different magnitude. (For instance, manufacturing trillions of multipurpose medical nanobots might be “easy” compared to making them all operate intelligently.)

Even so, the formidable barriers of advanced technology may fall easily if, instead of confronting them directly, we first build on our unique evolutionary legacy of general intelligence. The ad-hoc intellectual orchestra improvised by natural selection could almost certainly be outperformed by one assembled intelligently from the beginning. The creation of artificial general intelligence (AGI) represents a unique and formidable challenge, but holds tremendous promise as a way of playing to our greatest strength and augmenting it. In fact, the moment we achieve greater intelligence has such “singular” significance that futurists refer to it as the Singularity.

An adequately designed AGI could provide enormous assistance in the design of still more intelligent minds - a process that can be repeated in a self-reinforcing cycle. An AGI could also stand squarely outside the survival-promoting distortions that evolution has built into our thought processes, but at the same time possess a sympathetic respect for human ethics - a trait called Friendliness by some researchers. These new kinds of minds - free, capable, and compassionate to an unparalleled degree - would be invaluable partners in safely mastering technologies that can make personal evolution a reality.

Admittedly, opening a mind-and-body shop will probably not be the most urgent service performed by any Friendly AI. Indeed, it is the suffering of millions from potentially curable diseases and social conditions that should be making Friendly AI a world-wide research priority. (Many experts believe that AGI can be created in one or two decades with just a fraction of the funding devoted to causes like cancer research.) But initiating the transhuman destiny of homo sapiens sapiens is perhaps the most significant long-term achievement we can imagine; after that, who can say what dreams and challenges await?

We presently live in a beautiful-but-indifferent world where death and hardship are the norm. Adversity is, after all, the driving force behind natural selection. But as if that weren’t enough, evolution has tragically engineered us not to experience lasting happiness, but to restlessly tend insatiable appetites in the service of our genes. With help from Friendly new minds, however, the enduring frustrations of the human condition can be severed as the cold strings of a mindless puppeteer. The creation of greater intelligence is the first step towards evolution by choice: the freedom to create our own better selves.

©2002 by Mitchell Howe

Article by Bryan Appleyard About ImmInst Thursday, Mar 22 2007 

From respected, mainstream publication The Times Online!

Bruce Klein founded The Immortality Institute (Imminst) in 2002 as a non-profit organisation with the aim of ‘conquering the blight of involuntary death’. Klein was brought up in the town of Americus, ‘a jewel of Georgia’, in Bible Belt America, the deep south. ‘Yeah, I’m a southern redneck!’ he jokes. His family was not especially religious, though he did observe the Catholicism of his mother until the age of eleven when he took a phone call from their priest. ‘I said to him I didn’t believe any more. He got kind of upset and I hung up the phone. It was some kind of visceral thing.’

Klein was thirty-one when I met him at Imminst’s conference at the Georgia Tech Conference Center, Atlanta, in November 2005. The conference turned out to be a snapshot of the immortalist front line. It is a movement that is part cult and part serious science. But all were united by the fervency of their belief in the rightness of the project of extending life and by their vehement rejection of deathism and scepticism. The participants saw themselves as visionaries and frequently beleaguered pioneers of the only new frontier left to mankind. Klein is a groomed, fit-looking man. His wife and ‘wonderful friend’, Susan Fonseca-Klein, co-founder and director of the institute, is round-faced and pretty. Together, they have the air not of a threateningly glamorous but of a consolingly ideal couple – young, healthy, good-natured, extravagantly friendly, ambitious, optimistic, glowing. One could imagine them in an advertisement for breakfast cereal.

Part cult how? Just because it’s something unusually ambitious? Were the Wright Brothers and their supporters part cult? What about the Manhattan Project, where scientists were consolidated in Los Alamos? Silly journalists, what would we do if you didn’t take the liberty of exaggerating randomly to “spice up” the article? Reading stuff would just be so boring otherwise.

The article zooms in a bit on the AI side of the pursuit for immortality, something that many of the current ranks at ImmInst seem to be too busy consuming dinner plates of supplements to notice.

“Threateningly glamorous”, hmm. Max and Natasha, I think that’s your job. ;)

Bryan Appleyard, I’m sure your book will be an interesting one, but don’t call immortalism part cult. It’s not necessary to sell your book, or make it interesting.

Relative Advantages of AI and Human Brains Wednesday, Mar 21 2007 

Advantages of computer programs over humans, which some might call, “why we use computers at all”:

    More design freedom, including ease of modification and duplication; the capability to debug, re-boot, backup and attempt numerous designs.
    The ability to perform complex tasks without making human-type mistakes, such as mistakes caused by lack of focus, energy, attention or memory.
    The ability to perform extended tasks at greater serial speeds than conscious human thought or neurons, which perform approx. 200 calculations per second. Computing chips (~2 GHz) presently have a 10 million to one speed advantage over our neurons.
    The in principle capacity to function 24 hours a day, seven days a week, 365 days a year.
    The human brain cannot be duplicated or “re-booted,” and has already achieved “optimization” through design by evolution, making it difficult to further improve.
    The human brain does not physically integrate well, externally or internally, with contemporary hardware and software.
    The non-existence of “boredom” when performing repetitive tasks.

Advantages of human brains over hypothetical AIs:

    Present AIs lack human general intelligence and multiple years of real-world experience.
    The computational capacity of the human brain is estimated at 2 * 10^16, or 20 million billion calculations per second, which is twenty times greater than the supercomputer Blue Gene’s predicted achievement of 10^15, or 1 million billion calculations per second, by 2005. However, the human brain may not have a computational advantage over computers for much longer. Ray Kurzweil, for example, predicts that the computational capacity of the human brain will be accomplished on supercomputers, or clustered systems, by 2010, followed on personal computers by 2020.
    The human brain has already achieved a high-level of complexity and “optimization” through design by evolution, and thus has proven functionality.

Advantages of minds-in-general (AIs) over the human brain:

(The following are not advantages of specific AI approaches, but rather advantages of minds-in-general over the human brain.)

    An increased ability to acquire, retrieve, store and use information on the Internet, which contains most human knowledge.
    Lack of human failings that result from complex functional adaptations, such as observer-biased beliefs or rationalization.
    Lack of neurobiological features that limit human control over functionality.
    Lack of complexity that we have acquired from evolutionary design, e.g., unnecessary autonomic processes and sexual reproduction.
    The ability to advance on the design of evolution, which is continually constrained by lack of foresight, the requirement to maintain preexisting design, and a weakness with simultaneous dependencies.
    The ability to add more computational power to a particular feature or problem. This may result in moderate or substantial improvements to preexisting intelligence. (AI does not have an upper limit on computational capacity; we do.) Note that the speed of computational power is predicted to continually increase exponentially, and decrease exponentially in cost, every 12-24 months, in accordance with Moore’s Law.
    The ability to analyze and modify every design level and feature.
    The ability to combine autonomic and deliberative processes.
    The ability to communicate and share information (abilities, concepts, memories, thoughts) at a greater rate and at a greater level of complexity than us.
    The ability to control what is and what is not learned or remembered.
    The ability to create new modalities that we lack, such as a modality for code, which may improve the AI’s programming ability-by making the AI inherently native to programming - far beyond our own (a modality for code may allow the AI to perceive its hardware machine code, i.e. the language used to write the AI, and other abilities).
    The ability to learn new information very rapidly.
    The ability to consciously create, analyze, modify, and improve abilities, concepts, or memories.
    The ability to operate on computer hardware that has powerful advantages over human neurons, such as the ability to perform billions of sequential steps per second.
    The capacity to self-observe and understand on a fine-grained level that is impossible for us. AIs may have an improved capacity for introspection and manipulation, such as the ability to introspect and manipulate code, which would be the functional level comparable to human neurons, which we can’t think about or manipulate.
    The most important and powerful capacity of minds-in-general over the human brain is the ability to recursively self-encapsulate and self-improve its intelligence. As a mind becomes smarter, the mind can use its intelligence to improve its design, thereby improving its intelligence, which may allow further improvements to its design, thus allowing further improvements to its intelligence. It is unknown when open-ended self-improvement may begin.

Think about the differences and what they mean. The items on the above lists are not controversial - they’re either known facts or follow directly from the nature of the hardware. It’s the policy consequences that are controversial. But take the time to ignore the policy implications (if any), and by ignoring I mean not commenting on, and leaving this post just as a place to meditate about known differences between human brains, computer programs, and hypothetical AIs.

Next Page »