ManuAIVI

L065 · What's next · 2 min

The black box problem

Even the people who build large AI models cannot fully explain why a model gave a particular answer.

01 · The idea

A neural network is billions of numbers adjusted during training. Nobody writes the rules by hand. The behaviour comes out of all those numbers working together.

So when a model gives an answer, there is no simple list of reasons to read. You can ask it to explain itself, but its explanation is just more generated text, and it may not match what really happened inside.

Researchers are working on this, in a field called interpretability. They have found ways to spot some ideas inside models and see how certain answers form. It is real progress, and still far from a full picture.

For now, this is why decisions about loans, jobs, health or the law should not rest on an AI answer alone. People need to be able to question a decision and get a reason a human can stand behind.

02 · Think of it like this

It is like a very experienced chef who can taste a dish and say "more salt" without being able to explain how they know. Useful, but you would not let them run a hospital kitchen without checks.

03 · Do it now

04 · Quick check

Why is a large AI model called a black box?

Words in this lesson

See also