Disaster 08 · Engineering disasters

The hidden system that trusted one sensor

Two new Boeing 737 MAX jets crashed within five months because a flight-control system pushed the nose down on bad data from one sensor, a system many pilots had not been told about.

  • Boeing and the US Federal Aviation Administration
  • Indonesia and Ethiopia
  • 2019
  • 3 min read
On this page
Before you readWhat would you do?

Your online shop sets up an automatic tool that cuts prices when a competitor's price drops. It reads prices from one website only. What is the most important safeguard?

Pick an answer now. The file tells you whether you were right.

01The file in one minute

  1. 1Hidden software

    MCAS could push the nose down automatically and was not described in the pilots' manuals.

  2. 2One sensor

    It relied on a single angle-of-attack sensor, so one fault could trigger it again and again.

  3. 3Two crashes

    Lion Air 610 in October 2018 and Ethiopian 302 in March 2019 killed 346 people in total.

  4. 4The lesson

    Automation needs more than one input, firm limits and people who know how to stop it.

02Timeline

  1. 29 October 2018

    Lion Air flight 610 crashes after take-off from Jakarta. 189 people die.

  2. 10 March 2019

    Ethiopian Airlines flight 302 crashes near Addis Ababa. 157 people die.

  3. March 2019

    Regulators around the world ground the 737 MAX.

  4. September 2020

    A US House committee report criticises Boeing and the FAA's certification process.

  5. November 2020

    The FAA clears the plane to fly again after changes to MCAS.

03What happened

Boeing launched the 737 MAX as a more fuel-efficient version of its best-selling jet. The new, larger engines changed how the plane handled in some conditions, so Boeing added software called MCAS that could push the nose down automatically.

MCAS took its reading from a single angle-of-attack sensor, even though the plane had two. If that sensor failed, MCAS could push the nose down again and again. The system was not described in the pilots' manuals, partly because Boeing wanted to avoid extra simulator training for pilots already flying older 737s.

On 29 October 2018, Lion Air flight 610 crashed into the sea shortly after taking off from Jakarta, killing 189 people. On 10 March 2019, Ethiopian Airlines flight 302 crashed near Addis Ababa, killing 157. In both cases a faulty sensor triggered MCAS and the pilots fought it until they lost control.

Regulators around the world grounded the 737 MAX in March 2019. It was cleared to fly again in the United States in November 2020, after changes that made MCAS use both sensors and act only once.

04Why it happened

A US House committee report in 2020 described pressure to compete with Airbus on cost and schedule, and a certification process in which the regulator relied heavily on Boeing's own staff. Each decision, such as using one sensor or leaving MCAS out of the manuals, made sense locally. Together they created a hidden single point of failure.

Pilots could not respond well to a system they did not know existed. Automation that acts on its own needs the people supervising it to understand what it does and how to stop it.

The idea behind this caseLesson: AI agents: AI that takes action →

Kit the fox, ManuAI's guide, thinking it over

05Lessons for decisions

  1. When a system acts on its own, make sure the people responsible for it know what it does and how to switch it off.
  2. Never let one input, one sensor or one person's check be the only thing standing between you and a serious mistake.
  3. Be wary when the people who build something are also the main people checking it.
  4. When a first serious failure happens, act quickly on the cause before a second one.
Your online shop sets up an automatic tool that cuts prices when a competitor's price drops. It reads prices from one website only. What is the most important safeguard?

06 — Would you have caught it?

Your online shop sets up an automatic tool that cuts prices when a competitor's price drops. It reads prices from one website only. What is the most important safeguard?

07 — Sources

  1. Final Committee Report: The Design, Development and Certification of the Boeing 737 MAX · U.S. House Committee on Transportation and Infrastructure · 2020-09
  2. Boeing 737 MAX groundings · Wikipedia · 2026