ManuAI
VI
All 12 lessons

Lesson 4 of 12 · Data

When the data is lopsided

Students study a drawn training set for a shoe detector that is mostly sneakers, find why it fails on sandals and school shoes, and fix the set.

Or print one part

In the print dialog choose A4, scale 100% (or Default) and turn on Background graphics for the coloured parts. Choose Save as PDF to keep a file.

Lesson 4 of 12 · Data · Teacher's plan

When the data is lopsided

Students study a drawn training set for a shoe detector that is mostly sneakers, find why it fails on sandals and school shoes, and fix the set.

Students will be able to

  • explain that an AI can only learn from the examples it is given
  • spot what is missing or over-represented in a set of examples
  • describe bias as a system working well for some cases and badly for others because of its data
  • suggest how to make a set of examples more balanced

Prepare

  • A training set of 12 simple shoe drawings on the board or a printed sheet: 10 sneakers, 1 boot, 1 school shoe
  • A test set of 6 drawings: sneakers, sandals, school shoes, a flip-flop, a high heel, a boot
  • Blank paper for students to draw extra examples
  • One worksheet per student

Words to know

Training set:
The examples a machine learns from.
Lopsided data:
A set of examples with far too much of one kind and too little of others.
Bias:
When a system keeps getting things wrong for some cases more than others, often because of its data.
Balanced data:
A set of examples that covers all the kinds the system will meet, in fair amounts.

Main activity · Unplugged: paper and pens

Fix the shoe detector

Students learn the idea of a shoe from a training set that is almost all sneakers, then try to label new pictures using only what that set shows. They see the detector would miss sandals and school shoes, find the gap, and design a better set.

ManuAI pages for this lesson

The 45-minute plan · 45 minutes

Warm-up6 min

An alien learns about shoes

  • Tell a story: an alien has never seen a shoe. We show it 12 pictures to teach it.
  • Show the training set. Ask: "What will the alien think a shoe looks like?"
  • Collect ideas: laces, thick soles, covers the whole foot.
Activity18 min

Test the shoe detector

  • In groups, students write three rules the alien might learn from the training set, such as "a shoe has laces".
  • Show the test set one picture at a time. Groups use only their rules to say shoe or not shoe.
  • Record results on the board. Sandals, flip-flops and school shoes without laces are often marked as not shoes.
  • Ask: "Is the alien stupid? Or did we give it lopsided examples?" Agree that the data was the problem.
  • Groups count the kinds of shoe in the training set and list what is missing.
Practice16 min

Design a fairer training set

  • Each group draws or lists four new examples to add to the training set to fill the gaps.
  • Students complete the worksheet, including the real-life examples task.
  • Share: why does it matter if a real AI, such as a face or voice system, is trained on lopsided data? Keep the talk on the data and away from any group of people.
Wrap-up5 min

The machine learns what it sees

  • Write on the board: an AI is only as fair as its examples.
  • Ask one student to explain the word bias in their own words.
  • Hand out the exit ticket.

Teacher tips

  • Keep the classroom examples about objects such as shoes, fruit or leaves. When students mention real AI systems that worked worse for some people, talk about missing data. Do not invite jokes or generalisations about any group.
  • Misconception: "the computer is biased because it has opinions". It has no opinions. It copies the patterns in the data people gave it, which is why the people choosing the data carry the responsibility.
  • Large classes: draw the training set once on the board instead of printing it. Fast groups can design a second detector (for example a bird detector) and say what a lopsided set for it would look like.

Exit ticket answers

  1. A fruit detector was trained on 100 photos of red apples and 2 photos of green apples. What is most likely? A. It will be good at red apples and worse at green apples
  2. In your own words, what is bias in an AI system? A good answer: When the system works well for some cases and badly for others, usually because its training examples had too much of some kinds and too little of others.
  3. What is the best first fix for a lopsided training set? B. Add more examples of the kinds that were missing
Free to print and share in classrooms and at home.ManuAI · free AI lessons for everyone · ai.manusignal.com

Worksheet · Lesson 4 of 12 · Data

The shoe detector

NameClassDate

Use the training set on the board. Think about what the machine learned and what it missed.

  1. 1Count the kinds of shoe in the training set.

    Kind of shoeHow many in the training set
    Sneakers
    Sandals
    School shoes
    Boots

  2. 2Write one rule the shoe detector might have learned that is wrong for real shoes.

  3. 3Why did the detector fail on sandals?

    • Sandals are not really shoes
    • Computers cannot see sandals
    • The training set had almost no sandals in it
    • The detector was switched off

  4. 4Draw or list four examples you would add to make the training set more balanced.

  5. 5Think of a real AI, such as a voice assistant or a photo app. What could go wrong if its training data was lopsided?

Free to print and share in classrooms and at home.ManuAI · free AI lessons for everyone · ai.manusignal.com

Cut along the dotted lines: two tickets per page.

Exit ticket · Lesson 4ManuAI

When the data is lopsided

NameClassDate

  1. 1A fruit detector was trained on 100 photos of red apples and 2 photos of green apples. What is most likely?

    • AIt will be good at red apples and worse at green apples
    • BIt will be equally good at both
    • CIt will only recognise bananas
    • DIt will refuse to work

  2. 2In your own words, what is bias in an AI system?

  3. 3What is the best first fix for a lopsided training set?

    • ADelete all the data
    • BAdd more examples of the kinds that were missing
    • CMake the computer faster
    • DTell users not to use it