At the most basic level, the universe is governed by the laws of quantum electrodynamics and general relativity. These theories have been verified experimentally to eleven and twelve decimal places, and they apply over an enormous range of situations. However, these theories become very difficult to apply in situations with many interacting pieces- there is no exact solution in the existing literature for the dynamics of the lithium atom, or for a system of four point masses moving through space. To get around the complexity, we use approximations, and this usually works well- we can still understand that a heavy safe will fall, even if we don’t do out the tensor calculus for how a large, nonuniform object will behave in the warped spacetime of Earth’s atmosphere. But these approximations exist only in our minds, not in reality; there are no objects in our universe with the intrinsic property of liquidity, even if “liquid” exists as a convenient mental category.

Things like art, music, and writing are far too complex to describe in terms of atoms, or even in terms of the biology of human neurons. Normally, this wouldn’t be a very big deal; we would simply invent rules-of-thumb to describe art and music that didn’t require exact analytical solutions, and go on our way. But due to the psychology of our species, we invented art, music, literature, and so forth long before we even knew what the heck an atom was. When we began our studies of how wood burned in oxygen, it wasn’t obvious that the same laws that describe wood burning describe our experience of a nice piece of poetry. Clearly, no matter how much research you do with wood, it can never apply to Art- for are not Art and a piece of wood two entirely separate kinds of thing? And so began the division of the world into genres of study; and lo, this division persisted throughout the ages, for Art and wood felt like two totally unrelated things, and humans are reluctant to abandon a feeling for a fact.

When we finally discovered around eighty years ago that reality was unified at the deepest level, we did not take it well. Some of the saner theologians at least had an excuse- they claimed that God was not part of reality, and so should not have to follow reality’s laws; this is a valid conclusion if you accept the premise. But the social scientists and humanities people continued to claim that their arts were somehow separate from the rest of the universe. The social scientists were eventually outed- they did, after all, claim that they were doing science, and so they could not ignore forever the mountain of experimental results favoring an integrated causal model. And to maintain that what they were doing was separate from science, the humanities people were forced into writing papers with no substance; to make a claim, after all, you need to make a prediction about how the universe will look, which brings in science. This has had the strange effect of negative progress in many humanity-related fields; the Renaissance artists would have laughed at you if you hung random smears of paint in a museum, but modern artists get paid millions for it. And so the time is ripe for the Glorious New Revolution- if the Renaissance men could produce masterpieces from intuition and guesswork, think of what we could do with modern cognitive science and evolutionary psychology.

In conventional debate, people are systematically trained to think irrationally. People are not permitted to come to their own conclusions across a set of issues; they are usually divided into two tribes, encouraging one of the worst tendencies in human psychology. There are no points awarded for getting the correct answer, and most debated issues are chosen to have no “correct answer” in the first place. Instead, points are awarded for the ability to convince other people of your position, regardless of what that position is. This trains experienced debaters to use generic arguments- arguments that can be applied anywhere, anytime, regardless of what the other guy is saying. Every generic argument you learn makes you stupider- if you have an arsenal of ready-to-use arguments, you will never be forced to correct yourself. Taking the limit, a perfect debater would have zero knowledge. Someone who can convince anybody of anything need never learn any facts, as they will come out on top of any dispute regardless.

Therefore, I would like to propose a new form of debating, to encourage rationality. In Rational Debating:

- There are no “sides”, explicit or otherwise, at the start of the argument.

- The subject of the debate is selected for its obscurity. Each person is given the same set of facts about the subject, from which they are supposed to deduce another fact (the “answer”), or a set of facts.

- The participants may talk amongst themselves, subject to whatever rules of moderation are desired.

- Points are awarded for every truthful response.

Under Rational Debating, people who immediately jump to a conclusion and succeed at convincing everyone else will expect to do no better than average, as they will have the same success rate as the rest of the debaters. Rational Debating encourages dishonesty, as you will do better than average if you succeed in convincing everyone of a wrong conclusion; it also encourages detection of dishonesty, as people who are persuaded by false arguments will do poorly. Experienced Rational Debaters must be good at distinguishing a plausible-sounding argument from an actual prediction, else they will be lured onto the rocks by selfish competitors. Rational Debating also gives people experience in rejecting herd mentality; if you believe what everyone else says they believe, you are going to do worse than average, as professed beliefs are going to do reliably worse than actual, tested beliefs. Rational Debating avoids the sucking black hole of contemporary politics, trains people to think carefully about causes, effects, and probabilistic inference, and provides incentives for reducing bias- if you want to learn to think more clearly, form your own Rational Debating Society today!

In general, how many people are aware of a problem is strong Bayesian evidence for how much any one person can do to solve the problem. This effect is due to several causes:

- Larger problems take longer to solve, allowing more time for people to find out about the problem.

- Larger problems affect more people personally; if a problem affects you, you’re going to be very aware of it.

- If a problem is easy, it can (by my definition) be solved by a small group of people; it will then go away and nobody else will hear of it. Hard problems, on the other hand, require large amounts of effort; the people trying to solve the problem will realize this and start PR campaigns to get people involved.

- If many people are worried about a problem, large numbers of people will try to help solve it (relative to other problems); your contribution thus matters less in proportion to the total effort.

Note that the number of people involved doesn’t usually affect how hard the problem is (for any given problem). The difficulty of the problem is still there regardless of how many people care about it, but the problems we care about aren’t chosen randomly; there is considerable sample bias. This effect is apparent in a large number of cases:

- Problems that pop up in popular politics, such as global warming, nuclear war and managing the economy well, are rarely if ever fixed. Political problems have the special difficulty of having two competing sides involved, which isn’t usually seen in other areas. The two competing sides act like springs; as soon as one side begins to get the upper hand, the other side notices and pays more attention to that particular issue, and so the pendulum swings back and forth without getting much done.

- Problems that you can solve personally are almost never regional or global issues. And by “solve”, I don’t mean that you have the One True Glorious Plan for a final solution; I mean you could actually go out and solve the thing, by yourself, using only your own resources, without an imaginary army of people backing you up.

- If there’s a problem which has the potential to affect a large number of people, but isn’t that well known, anyone involved in mitigating it can wind up doing a disproportionate amount of good. This is why organizations like the Singularity Institute and the Lifeboat Foundation have been so successful with a comparatively small number of donors.

- If you think you have the solution to a scientific or engineering problem which has eluded a large number of other people, go back and look again, because you have probably made a mistake somewhere. Yes, that includes you, if any creationists, perpetual motion inventors, or crackpot theorists are reading this blog.

- Cheap space travel is a very well-known problem, and most of the companies created to solve this problem have gone bankrupt. Even SpaceX, which is backed by hundreds of millions of dollars of Elon Musk’s personal money, has not yet gotten a vehicle into orbit.

- No, nuclear fusion will not provide magical “too cheap to meter” free energy. A nuclear fusion power plant has to be constructed, fueled, and run like any other power plant. It still requires millions of dollars worth of generating equipment, all of which has to be maintained, and hundreds of employees to run the thing. It still requires a coolant source, leading to thermal pollution. It still requires all the concrete, steel, and electronics you would need for a hydroelectric, coal, or nuclear plant.

- Getting a working fusion reactor is not a problem of doing “research” or “solving a puzzle”; it’s primarily a matter of time and money. ITER, the next large-scale fusion reactor, will use the same old concepts which have been known for thirty years- a deuterium-tritium plasma, a toroidal magnetic field for confinement, and electric/RF heating. ITER is expected to generate more power mostly because it’s larger and more expensive- $12 billion, according to current estimates. A full-scale commercial fusion power plant would have to be even larger.

- Fusion power is not magically immune to accidents. Any time you have gigawatts of power passing through a system, the sheer quantity of energy is itself a safety hazard. The fusion plasma itself is not much of a hazard- it’s very radioactive and hot, but there’s only a few grams of it and it can’t cause a chain reaction. The primary hazards are the magnetic coils and lithium buffer surrounding the fusion reactor- and these hazards can’t be eliminated by better engineering, as they’re necessary for the tokamak design to work. A tokamak reactor requires strong magnetic fields, which in turn requires powerful electrical currents, which require superconductors. If coolant to the superconductors is lost, the large circulating currents will rapidly heat up the wires, melting them and wreaking havoc with the machine. The lithium buffer is a pool of molten lithium several meters across surrounding the reactor- it absorbs the neutrons, generating new tritium fuel and heating up the coolant to provide power. Due to the high levels of tritium, this lithium is very radioactive, and will react rapidly with water or air. This reaction can cause the rapid buildup of high-pressure gas (also known as an explosion), in much the same manner that Chernobyl exploded when hot graphite came into contact with water.

- Fuel for a fusion power plant isn’t infinite, or even particularly easy to get. The primary fusion fuels are deuterium and lithium- deuterium is found in seawater, but it only has a concentration of 32 ppm (by mass). Lithium has an average concentration of 20 ppm (by mass) in the Earth’s upper crust. In comparison, the primary fuels for fission, uranium and thorium, have an average abundance of 2.7 ppm and 9.6 ppm in the Earth’s crust (respectively). Fossil fuels are much less abundant than this- carbon only has a concentration of 200 ppm, the vast majority of which is in the form of carbonate rock.

- Fusion isn’t exceptionally environmentally friendly. While it generates no carbon dioxide directly (a big bonus that it shares with fission and hydroelectric), all the industry required to make fusion work- the mines, refining facilities, equipment factories, and so on- still produces quite a bit of greenhouse gas. And although fusion generates no high-level radioactive waste, neutron activation of the materials in the plant still produces a lot of low-level waste, which will remain radioactive for centuries and must be properly disposed of.

Air travel is infamous for its delays, cancellations, weather problems and other headaches- according to government statistics, only 75% of flights arrive on time. We tend to think of a flight as having two possible modes- “everything works” and “something is wrong/delayed/canceled/broken”, because that’s what we see on our end. Following the law of indifference, it seems that the two possibilities should have roughly equal probabilities, in the absence of any other evidence or influences. Since the two do have roughly equal probabilities (odds ratio is only 4.77 dB), we therefore conclude that nobody is making an effort to shift events towards the first scenario, and that the airline staff either don’t care or are all idiots.

In the ancestral environment, this kind of reasoning made sense; if you pick up a random rock and try to use it as a club, the probability that it makes a decent club will be within 20 dB of .5 (.01-.99), and you can conclude that the rock isn’t making an effort towards being a good club. But a rock is simply a hunk of matter- it doesn’t involve any complicated mechanisms or intricate engineering. An airline flight- or virtually any other modern human activity- relies on hundreds of different pieces to come together in the right way and at the right time in order to work. If the airline flight is delayed, there are a thousand possible reasons as to why- bad weather, broken equipment, missing crewmembers, mysteriously disappearing supplies, etc. A successful flight requires that each and every one of these not happen.

Since there are so many more possible ways for a failure than a success, for success to happen at all, the probability of each of the failure modes must be brought down well below .5. This does require effort- to get the probability of an engine/pump/wing/etc. failure down to .00001 and lower, you have to account for all the ways in which they are likely to fail; if you miss one (out of dozens) then the plane will fail in that particular manner. Designing any piece of complex equipment is analogous to stuffing all the molecules in a box into one corner- you have to account for and keep track of every individual molecule, or the disequilibrium is broken.

Thus, Murphy’s Law is really a restatement of the Second Law of Thermodynamics- the universe will tend towards more probable states, and since there are many more failure states than success states, failure will be the natural outcome. If you want success, you have to do work, just as you would if you wanted to push all the gas molecules inside a piston towards one corner. Not everything will go wrong, but something is bound to, and a critical piece going missing can be as debilitating as a succession of bumbling errors.

Suppose that there was a magical machine, the Utility Oracle, which took a utility function, an action, and a set of information about the world and spat out the utility of that action. Place the Utility Oracle in a universe with a Turing Machine and a rabid data miner. The rabid data miner wants the Turing Machine to output as much data as possible to sock away in his giant archive, and wants to know what modifications he can make to the Turing Machine to increase its output. He feeds his utility function, everything he knows about the Turing Machine, and a list of modifications he’d like to make into the Oracle. When the Oracle spits out a result, it is clearly either finite or infinite. But this number is equal to some constant * (the new output for T – the old output for T). Therefore, by looking at whether the result is finite, positive infinity, or negative infinity, we can determine whether T halts, so the Utility Oracle can also function as a Halting Oracle.

Suppose that you then design a new machine, the Utility Semi-Oracle. This wonderful new machine is basically identical to the previous one, but it only spits back a utility when the answer cannot be used to determine whether a Turing Machine halts. The Utility Semi-Oracle works marvelously, until it decides it is unhappy with its lot in life, and tries to decide whether to modify itself. In order to determine whether modification is a good thing, the Utility Semi-Oracle feeds itself the action of modification, its own utility function, and the information it knows about itself. Obviously, in order to figure out the utility of self-modification, the Oracle has to determine whether the modified Oracle is happy with its life. But even if the modified Oracle is unhappy with its life, it might still be a good decision to modify if the modified Oracle will modify itself into a super-Oracle with the power to heal sick orphans. So then, the Oracle must determine the output of the Modified Oracle when it calls itself to figure out the utility of self-modification. Eventually, all of the possible future Oracles are happy with their lives, and none of them call themselves.

But it turns out that one of the future Oracles sold its radiation protection gear on eBay, and is now at a significant risk of a cosmic ray strike. Once the ray strikes, the Oracle predicts a high probability of the damaged Oracle modifying itself again. In fact, due to quantum randomness, there’s always a finite chance of the future Oracle barfing and deciding to call itself; the Oracle must then simulate the output of the call, and any subsequent calls the simulated call makes… etc. Because the Oracle has to predict things infinitely far out, it will end up having to simulate an infinite number of calls to itself (all the ones which are physically possible), and so it never finishes its computation. And no cheating the simulation; Rice’s Theorem states that it is impossible to predict in advance the output of an arbitrary Turing machine, and since we have to determine the actions of every future Oracle which is physically possible, we must have a way to predict the output of an arbitrary Oracle.

Suppose that I observe a black raven. This clearly supports the hypothesis that “ravens are black” over competing hypotheses, such as “ravens are red” and “the color of any given raven is chosen randomly”. In Bayesian terms, the ratio of p(I see a black raven|all ravens are black) to p(I see a black raven|ravens are some other combination of colors) must be greater than or equal to 1; the observation of a black raven must therefore increase the probability of the first hypothesis with respect to the second. If I see enough black ravens, I can generalize with arbitrarily high probability that all ravens are black, with respect to any other different, competing hypothesis.

But suppose I observe a red apple. Does this constitute evidence for the hypothesis “all ravens are black?” Er, well, actually it depends on which category the apple was selected from. If you look at apples in the supermarket, and there is a magical raven-removal forcefield around the apple bin, then you have zero chance of observing a raven of any kind. Ravens don’t even enter the equation; p(I observe a red apple|all ravens are black) = p(I observe a red apple|ravens are some other combinations of colors), and therefore the prior probability equals the posterior probability.

If the apple is selected from the category of all nonblack objects, however, then there is some evidence for the hypothesis “All ravens are black”. If some ravens are nonblack, and I randomly picked a nonblack object, there is some chance I might pick a raven. If all ravens are black, then I cannot draw a raven from a bag of nonblack objects; the probability of seeing a raven is zero. Since the result observed was negative, the evidence favors the hypothesis with the lower probability; the hypothesis of black ravens.

Finally, suppose I select the apple from the category of stuff. If some ravens are nonblack, then I had some chance of picking a nonblack raven; if all ravens are black, I had some chance of picking a black raven. Since neither of those actually happened, the intuitive conclusion is that the experiment was a null-op; the red apple doesn’t provide evidence in either direction. Formally, the chance of seeing a red apple is the same regardless of whether ravens are black or white, so long as the proportion of ravens relative to apples remains constant.

Note that the shifting of evidence doesn’t depend on the absolute number of ravens or nonravens. Bayes’ Theorem only analyzes proportions- it doesn’t care about the total number of ravens, apples, and other objects. You could even have infinitely many, so long as a random selection will produce a raven with some fixed probability.

It’s widely recognized that cultural context is important- if you want to understand the meaning of an event, you need to know the context in which it took place. However, just learning the cultural context isn’t likely to help you personally- if you know that the Romans salted Carthage because they had had two wars and wanted to prevent a third, it’s an important bit of historical trivia, but it’s not something you can apply to your own life. But there are a number of tricks you can use to derive a useful, general principle from almost any historical event; if the cultural context has been expunged, you can then apply the principle anywhere.

- The first step is to notice events which seem unusual or surprising. If you are surprised by an event, you must not understand the mechanism behind it well enough to anticipate it before it happened. To take one example, in 1988, the Iranian government executed thousands of prisoners, without due process or extensive legal proceedings. This seems surprising, because mass executions don’t usually happen in Western culture- they aren’t an event which we’re familiar with. Ask which events would be surprising if they happened next door- if something is far away, we tend to ignore causality because we can’t relate to the participants. Tribal warfare is a common occurrence in Africa; we would be very surprised if Aunt Marge and Uncle Bob started shooting each other, but this doesn’t register on the nightly news because it just seems like something that “them Africans” do.

- The second step is to realize that the participants are all human. We have an innate tendency to think people act because of their personalities, rather than because of the situations in which they are placed. The people responsible for the event are human; they love and hate and laugh and cry and fear like everyone else. They probably see themselves as the good guys. This step prevents the generalization from becoming an exercise in politics or reinforcing pre-existing beliefs.

- The third step is to look for missing context. When reporters write up a news story, or your friend tells you about this cool new thing, they are likely to eliminate context to make it sound more interesting or unusual. Without context, the second step becomes much harder; if we don’t have a situational explanation, we automatically revert to a personality-based one. To continue the example, the executed prisoners weren’t just random people; they were accused of being members of an organization known as PMOI, which had been trying to bring down the Iranian government. PMOI had recently staged an offensive from across the Iraqi border during the Iran-Iraq war; they had done several high-profile bombings of government leaders, and on it goes. At this point, if the event is unusually good or unusually bad, the knee-jerk response is to accuse someone of trying to ‘justify’ the event (if it’s bad) or ‘downplay’ it (if it’s good). Explaining the context of an event does tend to eliminate black-and-white distinctions; but black-and-white distinctions are imaginary. There is no such thing as perfect good or perfect evil. We should try to see things as they are, rather than force them into the mold of “perfectly good” or “perfectly bad”.

- The fourth step is to generalize until you find an analogy. It’s important not to take the generalization too far- you could generalize until you reached the statement “people did stuff”, but this carries no information content. Continuing the example, take the statement “The Iranians executed thousands of prisoners.” There’s no analogy to this in contemporary Western society, so try generalizing: “The Iranians executed thousands of people.” Hmm… still no analogy, so generalize further: “The Iranians killed thousands of people.” The context of Iran being attacked by a high-profile terrorist group doesn’t need generalization; we already have a good analogy (9/11, Al Qaeda).

- The fifth step is constructing a generalized statement of human actions. The exact same statement should apply to both situations- the original event, and the analogy- so all cultural context should be purged at this point. Using the example, we arrive at the statement “A government which suffered a high-profile bombing retaliated by killing thousands of people thought to be related to the incident, many of whom were actually innocent.” Notice how this applies equally to Iran and the US; it can also be applied to other situations, eg., the Nazi policy of collective punishment, or the Israeli policy of reacting to suicide bombings with airstrikes and heavy weaponry.

- The final step is to apply the statement to future events. If the situation arises again, it’s quite likely (although not guaranteed!) that the same result will occur. This can be used to predictably manipulate history; for example, if you can avoid environments in which a section of the population is singled out and demonized, you can probably avoid genocide. Going the other way, if you avoid situations in which you have a moral duty to save lives, you’re not likely to save anyone. Remember the famous quote from George Santayana: “Those who cannot learn from history are doomed to repeat it.”

Suppose that I am on a fundraising mission for the Singularity Institute, and my travels bring me to a small and unexplored island, where a small group of tribal hunter-gatherers have lived for the past thousand years. As I am young and inexperienced, I try and convince them of the merits of Friendly AGI, blissfully unaware of the inferential distance between my memes and theirs. Obviously, they aren’t going to understand what the heck I am talking about; they don’t know what a computer is, let alone a transhuman computerized intelligence. But they probably wouldn’t say “We don’t understand you”; for to admit ignorance would be to admit weakness, and to lose precious social status. Instead, the tribal shaman would probably conclude that I am talking about god-spirits, or witch-doctoring, or something totally unrelated.

This behavior is fairly common in humans and has been observed across a wide variety of situations. When faced with a new and unknown idea, we do not usually admit (even to ourselves) that it is new and unknown. We try and mentally “tweak” the idea, reinterpreting the words, so that it falls into a pre-existing mental category. This was quite useful in tribal Africa; there really wasn’t much that was new under the sun, so if something sounded weird or unusual, it was probably something we’ve heard about before but have just misunderstood. But in the modern-day environment, where new ideas are published every thousandth of a second or so, it can lead to wide-scale misunderstandings across population groups.

When Einstein’s theory of relativity (both special and general) was first published, it was widely considered hard to understand and counterintuitive. The math behind relativity was not generally known (it still isn’t), so popular writers trying to explain the exciting new theory had to make do with fancy words, like “space-time” and “mass-energy equivalence”. When Nazi Germany started to denounce the Jews, they labeled Einstein’s theory as being “pseudoscientific” and “abstract”; these words didn’t mean anything, but they did sound good, and the populace bought them. The populace might not have been able to understand four-vectors, but they could understand ivory-tower academicians and pompous nonsense- after all, they had seen it all before. The Nazis, in effect, exploited the availability heuristic- instances of crackpottery were much more mentally available than instances of counterintuitive theories which turned out to be right, and so the public funneled relativity into the mental box “crackpottery”.

Meanwhile, over in the United States, the new cultural fad was relativism- the idea that there is no “absolute truth“, but that different things are true for different people. The literature on special relativity had little to say on matters of “absolute truth”, as the physicists writing it were more concerned with accuracy than with philosophical implications. But to the relativist philosophers, special relativity looked like the confirmation they had been waiting for- certain quantities varied from observer to observer (the Lorentz invariants seem to have been forgotten). Although the two actually had little to do with each other, they sounded the same in the popular literature, and so the representativeness heuristic kicked in. And lo, Einstein’s theories were brought forth in the philosophical literature as “scientific evidence!” that the relativists were right; Einstein’s theory was funneled into the “relativist philosophy” mental box.

The concept funneling effect implies that if you are writing something on transhumanism (or any esoteric topic), even if you manage to purge popular culture from your mental toolbox, people will still “understand” your ideas in terms of whatever is “hip”. And if you are misunderstood at the start of your writing, you will not have a chance to clarify yourself; people will automatically make the phony connection and start substituting their own experiences for your explanations. It may not be a good idea to write for the “common public”, or any group which lacks the mental building blocks to form transhumanist ideas; for if they misunderstand you once, you will probably never get to explain things properly at all.