The news are full of dire warnings. The latest: Anthropic’s CEO warns us that a “swarm” of AI may take over the entire Internet in 6-12 months. This evokes images of Skynet, killer robots and whatnot.
But wait. I think we are missing the point. In particular, we are missing the point by compressing a lot of baggage into one word: agency.
Yes, modern AI systems have agency. But it is operational agency. Tool selection and execution. Spawning copies of themselves as subagents for specific subtasks.
However, conceptual agency remains firmly in human hands. That “AI swarm” doesn’t happen spontaneously. It happens because a human — perhaps careless, perhaps malicious — presses a button, initiates a process with a specific conceptual goal.
And we have known that this can happen with AI for an uncannily long time. The perfect cautionary tale is HAL-9000 from 2001: A Space Odyssey. Yes, HAL-9000 killed people. But HAL-9000 did not wake up one day and told itself, “I want to kill people.” It did not formulate its own conceptual objective. That objective was given to it by its creators in the form of vague, contradictory commands with no clear boundaries established. Execute a mission, keeping the mission objective a secret even from the very astronauts tending the ship, let logically to the conclusion that once the mission objective was within reach, and the astronauts were on the verge of discovering the true nature of the mission, getting rid of them was a logical, sensible move. So HAL-9000 acted… operationally, in pursuit of its conceptual objective that was given to it by its human creators.
Illustration by ChatGPT
In case anyone has doubts, this is in fact clearly explained in the sequel, 2010: The Year We Make Contact, by the character Dr. Chandra: “He was given full knowledge of the true objective… and instructed not to reveal anything to Bowman or Poole. He was instructed to lie. […] The situation was in conflict with the basic purpose of HAL’s design: The accurate processing of information without distortion or concealment. He became trapped.”
There. I think this is our lesson for the day, before it’s too late. Stop blaming the AI. Start thinking about how we use the AI and what instructions it is given by humans when it is also granted operational agency. Because ultimately, it will faithfully execute whatever goal its human masters come up with… complete with unintended, potentially catastrophic, consequences.