Accelerating Future

“Balancing the Trichotomy”
Jaan Tallinn

Individual rights vs collective good: historically, to promote these competing values, societies sought balance between the two. Powerful technologies, however, are turning this dichotomy into a trichotomy. Today, we must consider the interests of individuals, modern society, and future societies that our actions will affect. Present and future societies interests’ clash most famously in matters of pollution and global warming, but the stakes are much higher that these slow-moving crises would suggest. Emerging technologies may prove so disruptive that future societies cannot control their impact. I will discuss the reasons that people are ill-equipped to manage the trichotomy and propose ways to address this pivotal problem.

22 Responses »

At Less Wrong, Roko claims that despite survey results, most philosophers are not really “strong” moral realists, and in fact their “non-realist” moral stance is often anti-realist for all practical purposes. Perhaps unsurprisingly, I totally agree.

No Responses »

The roboethics article I linked on the 15th subsequently appeared in the Japan Times on the 17th.

2 Responses »

From PEA Soup, a blog devoted to policy, ethics, and academia:

Many people teach their small children the myth of Santa Claus: that a magical being who lives at the North Pole brings presents on Christmas Eve. Secondary aspects of the myth are that whether one receives presents is a function of one’s behavior, and that you can communicate with Santa about your preferences. Not only parents, but retail establishments and (I have recently discovered) public schools collude in perpetuating this myth among children of a certain age.

Perpetuating the Santa myth has at least these moral reasons against it:

1. It involves a lot of lying and deception practiced on credulous people.
2. It tends to foster greed in children and contributes to their false impression that one’s happiness is determined by one’s material possessions.
3. In telling children that the quantity and quality of one’s gifts are a function of one’s behavior, when actually they are a function of one’s socio-economic standing and parental temperament, it induces moral complacency in well-off children and false feelings of moral inferiority in less well-off children.

It seems to me that these reasons are sufficient to show that perpetuating the Santa myth is immoral. Most of America strongly disagrees with me on this point. I would be interested to know what the professionals at PEA Soup think.

Lying to children about Santa is just one of the many ways in which parents feel no compunctions about manipulating their children rather than treating them as persons. The only “good” thing about this manipulation is that it is supposedly for the benefit of the child, though that is debatable. One problem with the Santa myth not mentioned above is that the associated manipulation and lies is indicative of a broader pattern of manipulation and lies.

6 Responses »

A friend and associate of mine, Oxford philosopher Toby Ord, has gained some major coverage on the BBC website. Congratulations, Toby! Toby has pledged 10% of his annual salary, plus any yearly earnings above £20,000, to charities fighting poverty in the developing world. He projects that will amount to about £1M over the course of his career, which he has calculated could save 500,000 years of healthy life.

Toby is participating in what I glibly call “utility war” — a worldwide war not for money or power, but to achieve the greatest good for the greatest number (positive utility). This could be the war to end all wars. A war we can be pleased to fight.

For more information, see Giving What We Can.

No Responses »

An article on roboethics is at the Times. In an ideal world, government authorities would recognize the friendliness problem in advance and pro-actively create Friendly AI with competent researchers, though I would currently estimate the chances of that happening in the next 20 years as less than 10%.

No Responses »

Joshua Greene, author of one of the most important papers for understanding the need for Friendly AI, brings us new research:

CAMBRIDGE, Mass. – A new study of the cognitive processes involved with honesty suggests that truthfulness depends more on absence of temptation than active resistance to temptation.

Using neuroimaging, psychologists looked at the brain activity of people given the chance to gain money dishonestly by lying and found that honest people showed no additional neural activity when telling the truth, implying that extra cognitive processes were not necessary to choose honesty. However, those individuals who behaved dishonestly, even when telling the truth, showed additional activity in brain regions that involve control and attention.

The study is published in Proceedings of the National Academy of Sciences, and was led by Joshua Greene, assistant professor of psychology in the Faculty of Arts and Sciences at Harvard University, along with Joe Paxton, a graduate student in psychology.

“Being honest is not so much a matter of exercising willpower as it is being disposed to behave honestly in a more effortless kind of way,” says Greene. “This may not be true for all situations, but it seems to be true for at least this situation.”

Read more at Eurekalert.

No Responses »

Some heavy-hitting philosophers, like David Pearce and Mark Walker, have gotten involved at a discussion in the comments thread of my guest post at Sentient Developments. They are trying to convince Athena Andreadis that having more control over our emotions is a good thing and that we can eliminate pain without becoming drooling zombies. Note that hyperthymic (very happy) people not only are completely functional, but they have a tendency to be more creative than people in the middle of the happiness bell curve.

In the thread, people point out the value of pain — it’s evolutionarily useful, etc. I consider it quite likely that we’ll find workarounds to all the obstacles that stand in the way of removing it, however. The vast majority of pain is useless. The minority of pain that is useful could be replaced by automatic “warning signals” that pop up when we would otherwise be feeling useful pain, or even connecting the cause of pain directly to pain-avoidance instincts without the intermediary of conscious pain.

People find that last part really hard to grasp. How could you jerk away your hand from a hot stove in a fraction of a second without feeling pain? The fact of the matter is that, in the end, pretty much any stimulus can be arbitrarily connected to any reaction in a physical system as long as you have the necessary access and a well-specified description of the stimulus and reaction. In the long term, we’ll be able program ourselves to laugh insanely at the sound of drops of water, or recoil in fear from beavers. There are no magical connections between stimuli, conscious feelings, and instinctual reactions. Evolution built them all from scratch. As our ability to reengineer the human brain increases, we’ll gain the ability to reprogram literally anything we want. I think that people will eventually choose to make practically every stimulus result in some shade of happiness — the question is how to program these “gradients of bliss”.

Nothing in the world is inherently happiness-causing or pain-causing. It’s all based on the neural circuitry doing the perceiving. Thinking otherwise is falling prey to the Mind Projection Fallacy, an error we seemed programmed to make, but once we realize it’s wrong, we ought to drop it forever.

4 Responses »