The house-flipping algorithm that could not keep up

Zillow is best known as a website for browsing homes and their estimated prices. In April 2018 it announced it would start buying and selling homes itself, beginning in Phoenix and then Las Vegas. The service, later called Zillow Offers, made quick cash offers, fixed the homes up and resold them.

By 2021 it was buying at a huge pace. In the third quarter of that year it bought 9,680 homes and sold 3,036, according to its results as reported by HousingWire. In October it paused new purchases, citing limits on how many homes it could handle.

On 2 November 2021, Zillow said it would close Zillow Offers. It wrote down the value of its homes by about $304 million, because it had paid more than it now expected to sell them for, and warned of a further $240 million to $265 million in losses. About a quarter of its staff would lose their jobs.

Chief executive Rich Barton said the company had found the "unpredictability in forecasting home prices far exceeds what we anticipated".

Zillow's offers relied on forecasts of what each home would sell for a few months later. Any forecast like this learns from past sales. When the market shifts quickly, the past becomes a weaker guide, and the forecast can be wrong in the same direction for many homes at once. Zillow's own explanation was simply that prices were far harder to predict than it had expected.

Scale made it worse. A small pricing error on one house is a nuisance. The same error repeated across thousands of purchases, with real money paid up front, becomes a loss of hundreds of millions.

AI models work the same way. They learn patterns from historical data and assume tomorrow looks like yesterday. When customers, prices or rules change, accuracy quietly drops, and nothing inside the model raises an alarm. The danger grows when its predictions trigger real spending automatically.

The idea behind this caseLesson: How machines learn →

1. 1Ask what period of data the model learned from, and whether the world still looks like that.
2. 2Track live accuracy against real outcomes every week, not just the test score from launch day.
3. 3Set limits on how much money or how many decisions a model can commit before a person reviews it.
4. 4Grow slowly in a new market or season, and pause automatically when errors start to lean one way.
5. 5Plan in advance what you will do if the model is wrong for a whole quarter.