The scowl that does not always mean anger

The common view is simple: a smile means happy, a scowl means angry, a frown means sad, and these faces are the same everywhere. Influential studies in the 1960s and 1970s, including Paul Ekman's work with the Fore people of Papua New Guinea, were widely taken as proof that a handful of emotional expressions are universal.

That view moved into technology. Companies built software that claims to detect emotions from a face in a photo or video, for uses such as judging job candidates in recorded interviews, measuring how engaged students are, and testing reactions to adverts.

In 2019 five scientists, Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix Martinez and Seth Pollak, published a review of more than 1,000 published findings in Psychological Science in the Public Interest. They asked two main questions: do people reliably make the expected face when they feel an emotion, and does that face reliably mean that emotion and not something else?

The answer to both was: not reliably. People do smile when happy or scowl when angry more often than chance, but much less often than the common view assumes. The same face can mean different things: people scowl when they are concentrating or confused, and smile when they are nervous or being polite. How emotions show on the face varies across cultures and situations, and studies in some small, remote communities found much less agreement about what the classic faces mean. Not every researcher accepts every conclusion of the review, but it showed that a face alone is a poor basis for judging what someone feels.

The AI angle: an emotion AI system learns to label faces from examples that people have already labelled, so it learns the common view, not the inner feeling. It can spot a smile very accurately and still be wrong about happiness. Lawmakers have noticed. Article 5 of the EU AI Act, which applies from 2 February 2025, bans AI systems that infer people's emotions in the workplace and in education, except for medical or safety reasons. And in 2021 the hiring-software company HireVue said it had dropped facial analysis from its video interview assessments, saying it no longer added significant value.

The idea behind this caseLesson: AI and your kids →

1. 1Separate what a tool can see, such as a smile or a frown, from what it claims to know, such as happiness, boredom or honesty.
2. 2Ask whether the tool was tested against what people actually felt, across different cultures, ages and disabilities, not just against labels on photos.
3. 3Be cautious with any tool that judges people at work or school from their face or voice, and check whether it is even legal where you are.
4. 4When a chatbot says facial expressions are universal, ask it what the 2019 Barrett review found, and read the review's summary yourself.