The photo contest that woke up AI

ImageNet was a huge collection of photos, each labelled with what it showed. Every year a contest asked teams to build software that could name the object in photos it had never seen, choosing from 1,000 categories. Most entries used methods where people hand-designed which visual features to look for.

In 2012 a team from the University of Toronto entered a deep neural network: many layers of artificial neurons, trained with backpropagation on more than a million labelled photos. Alex Krizhevsky wrote the code, Ilya Sutskever pushed for the project, and Geoffrey Hinton supervised. Krizhevsky trained it on a computer with two NVIDIA graphics cards in his bedroom at his parents' house.

The contest scores how often the right answer is missing from a system's top five guesses. The Toronto team, entered as SuperVision, got that wrong about 15 to 16 percent of the time. The next-best team was wrong about 26 percent of the time. The paper describing the network, now known as AlexNet, was presented at the NIPS conference that December.

The core ideas were decades old: the perceptron's learning from examples and backpropagation's way of training many layers. What had changed was data and computing power. ImageNet gave the network enough examples, and graphics cards, built for video games, could do the huge number of small calculations fast. The result convinced much of the field almost overnight. Google soon bought the team's small company, and Sutskever later co-founded OpenAI, which built ChatGPT.

For a chatbot user, AlexNet explains why today's AI is mostly about scale. Chatbots are built on the same recipe: a big network, a vast amount of examples, and a lot of computing. That is why they are expensive to build, why a handful of large companies make the leading models, and why a chatbot's knowledge reflects what was in its training data, including its gaps and biases.

The idea behind this caseLesson: How machines learn →

1. 1Ask what data an AI tool learned from, because that shapes what it is good and bad at.
2. 2Expect weaker results on subjects that are rare in public data, such as local laws, small languages or niche products.
3. 3Remember that a model's knowledge stops where its training data stops, unless it can search or read your documents.
4. 4Do not assume a bigger or newer model is always better for your task. Try it on real examples.