Disaster 09 · Engineering disasters
The forgotten code that lost a fortune in 45 minutes
A software update missed one of eight servers, waking up old code that sent millions of unwanted stock orders and cost the firm about $460 million in under an hour.
- Knight Capital Group
- New York, United States
- 2012
- 3 min read
On this page
Before you readWhat would you do?
Your team uses an automation that sends payment reminders to customers. You update the message template, but the old version still runs on one account nobody remembered. What safeguard would have helped most?
Pick an answer now. The file tells you whether you were right.
01The file in one minute
1One server missed
An update reached seven of eight servers. On the eighth, a reused switch woke up old code.
245 minutes
The old code sent a flood of orders, and Knight lost about $460 million, according to the SEC.
3Warnings ignored
Automated error emails went out before the market opened but were not treated as urgent.
4The lesson
Confirm every change everywhere, delete unused old code and keep a hard limit that stops runaway processes.
02Timeline
2003
Knight stops using an old trading function called Power Peg, but leaves the code.
Late July 2012
New code is copied to seven of eight servers.
1 August 2012
The eighth server sends a flood of orders for about 45 minutes.
October 2013
The SEC charges Knight, which pays a $12 million penalty.
03What happened
Knight Capital was one of the largest traders of US shares in 2012. To take part in a new New York Stock Exchange programme starting on 1 August, it updated its trading software. A technician copied the new code onto seven of the firm's eight servers. Nobody noticed the eighth had been missed.
The new code reused a switch that, years earlier, had controlled an old function called Power Peg, which had not been used since 2003. On the eighth server, turning on the switch woke up the old code instead.
When the market opened, that server began sending a flood of orders. In about 45 minutes Knight's systems made millions of trades in roughly 150 stocks. Automated emails had warned staff about an error before the market opened, but they were not treated as urgent. By the time the trading was stopped, the firm had lost about $460 million, according to the US Securities and Exchange Commission.
Knight needed emergency funding to survive and was taken over within a year. In 2013 the SEC charged it with breaking market rules on risk controls, and the firm paid a $12 million penalty.
04Why it happened
The SEC found that Knight had no written process for deploying code, no check that every server had the same version, and no automatic limit that would stop trading once losses grew large. Old, unused code had been left in place for years.
During the crisis, staff could not quickly tell what was wrong, and one attempted fix made things worse by putting the old code on all eight servers. Without a clear way to stop everything at once, minutes turned into hundreds of millions of dollars.
The idea behind this caseLesson: Review AI like a manager →
05Lessons for decisions
- Use a checklist for every change, and confirm the change is in place everywhere it should be.
- Delete old things you no longer use, such as old files, accounts or automations, so they cannot come back by accident.
- Set hard limits that stop an automatic process when something looks wrong, like a daily spending cap.
- Have a big, simple off switch and practise using it, so nobody has to work out how to stop things during a crisis.
07 — Sources
- SEC Charges Knight Capital With Violations of Market Access Rule · U.S. Securities and Exchange Commission · 2013-10-16
- Knight Capital Group · Wikipedia · 2026