Evolution crafts organisms with specific goals. Always, they revolve around a variable called inclusive fitness. Subgoals of inclusive fitness include ability to survive, obtain food, mate, and (sometimes) protect offspring and engage in evolution-mediated reciprocal altruism. In humans, the subgoals blossom into a peacock-tail-like phantasmagoria of music, art, ornamentation, intellectual pursuits, yadda yadda. Still, these are all spinoffs of inclusive fitness.

Inclusive fitness according to Earthly life is a tiny, tiny, tiny, tiny, tiny, tiny, tiny, tiny subset of all information-theoretically possible goal systems. The Hamming distance between Earthly life’s goals and goals of biota on another planet may be huge. This phenomenon magnifies itself when you have intelligence that can formulate its own goals and rearrange its evolutionary goals into arbitrary permutations.

Heard of the concept of a limit? When a certain goalset is implemented at the limit, totalistic things happen. For instance, if you were to implement a rabbit genome’s goalset to the limit, most of the terrestrial biomass on planet Earth would be converted into copies of rabbits. Ditto with practically every other animal. Animals are basically just robots manipulated by long DNA molecules, anyway.

When we can produce arbitrary goalsets and back them up with optimization pressure equivalent or exceeding that exerted by Homo sapiens, it’s usually called “bad”. Yet within the next century, we’ll likely create one that we consider “harmless”.