The Industrial Revolution is essentially when the modern world began. For the first time, the production of food and many basic goods was partially automated. The impact was huge: whereas most of history up to that point had been a zero-sum exercise in fighting over a fixed pie, the Industrial Revolution increased the size of the pie itself many times over. This laid the groundwork for positive sum thinking, the notion that we can all have better lives if we just cooperate instead of trying to edge out the next guy.

Today, we have the luxury of pointing out some of the downsides of industrialization, but if we could experience pre-industrial life firsthand (or observe it by visiting those few areas of the planet untouched by industrialization), we would recognize what we are blessed with. Usually we reserve our gratitude for other humans, but the machines that drive industrial civilization deserve our thanks as well. Without them, there would be no mass-produced clothes, or toilets, or plumbing, or medicine, or books, or computers, or pretty much anything except for what we or people we hire could build on a manual, one-by-one basis.

Many of the “superlative” visions of transhumanists, and some other futurists and tech-savvy folk, introduce the notion of a world where the Industrial Revolution is completed, and literally any production we desire can be automated. In the words of the RepRap project, this would give us “wealth without money”. To me, and many others, this is a question of when, not if. I see modern skepticism over the possibility of self-replicating machines to be analogous to pre-Industrial skepticism over the notion of a huge array of products without excessive labor input.

Because I’ve been thinking about self-replicating machines for over a decade now, I’ve already spent quite a bit of time considering many of the positive and exciting implications of the technology. Palaces built by automated legions, cities constructed in weeks or less, solar panel arrays hundreds of square miles in extent covering the desert floor. Call it superabundance, the natural consequence of having machines that self-replicate from raw materials, are powered by the sun, and do all the work for you. It’s the Final Industrial Revolution, because after we develop reprogrammable self-replicating machines, there will be no production left to automate.

I’ve thought about the feasibility and potential of self-replicating machines enough that I’m desensitized to many of the benefits. Maybe I was introduced to the concept young enough in life that I take it for granted. Instead of speculating endlessly about the possible benefits, lately I’ve been very concerned about the risks. My thesis is this: if we can survive the next 50 years or so, we’re home free. I don’t see actively pursuing self-replicating machines as morally necessary as ameliorating the risks, because the eventual invention and deployment of these machines is inevitable. Someone will definitely do it, probably sooner rather than later.

So I consider the risks. Unfortunately, this puts me up as a target for criticism, along with people like Martin Rees and Stephen Hawking. For instance, one “expert in apocalyptic movements and neocatastrophism” called Hawking’s warnings of catastrophic global disaster “regrettable hype”. Honestly, what other intelligent species could build enough bombs to kill everyone on the planet, then go around saying that we have nothing to worry about?

People are terrified of discussing catastrophic risk for a number of reasons, but I think the two primary ones are negative affect and a bias towards optimism. They sound like the same thing, but they’re actually slightly different. Negative affect is the uncomfortable feeling many people get when discussing something they don’t like, such as the annihilation of the human species. The solution is to turn the conversation to a jokey tone as quickly and desperately as possible. Sure, I guess heavy topics aren’t for everyone, but I wish that people could discuss the risk without flinching away at least 1% of the time, but I’m not seeing it. The second reason is a positive bias towards optimism. Humans have a well documented psychological propensity towards unrealistic optimism. Optimism is a badge you can hold up to a community to say, “I’m psychologically normal, and motivated to join in shared projects!” Well, great, but when we’re in a unique historical era where we have weapons that can kill off the human race but lack the wisdom to forgo their use, this attitude can be downright suicidal.

The Final Industrial Revolution will be happening in our lifetimes. How can we steer it down a beneficial path? “Stand back and let it happen” is a recipe for failure.