Convergent Subgoal

From the SL4 Lexicon:

Convergent Subgoal:

A subgoal that keeps popping up in generalized Goal Systems; subgoals that proceed logically from large classes of high-level goals. One example might be "acquire greater personal effectiveness", or "increase the intelligence I have available to accomplish goals with". If converting large amounts of local matter into computronium is a Convergent Subgoal (as it seems like it is), then UnFriendly AI would be a serious danger. (It doesn't seem like altruism or Friendliness are Convergent Subgoals; they're too complex structurally.) Examined in more detail in CFAI part 3.2.7.1: Convergent Subgoals, at http://www.singinst.org/CFAI/design/generic.html. See also Ethics in Advanced Artificial Intelligence, Goal System, paperclip AI, thermostat AI.