Spot the Bias — Case Files

Three AI systems. Three problems. Your job: find the bias, explain it, fix it.

🔍 Detective Briefing: Each case below describes a real AI system that made unfair decisions. The AI wasn't mean — it just learned from unfair examples. For each case: (1) identify WHO was treated unfairly, (2) explain WHY the AI got it wrong, and (3) suggest a FIX.

Case 01 📷

The Face Recognition Door

Bias type: Representation gap in training data

A school installed an AI-powered door that unlocks when it recognises a student's face. The AI was trained on 500 face photos — mostly students with lighter skin tones. When tested, the door opened quickly for most students — but repeatedly failed to recognise students with darker skin tones, leaving them waiting outside in the cold while others walked straight in.

🔍 Who was affected?

⚠️ Why did the AI fail?

🔧 How would you fix it?

Verdict This is called representation bias — when training data doesn't include enough examples from all groups. Even though no one programmed discrimination, the AI discriminated anyway.

Case 02 📝

The Homework Helper That Plays Favourites

Bias type: Socioeconomic gap in training data

A city built a free AI homework helper for all students. It was trained on textbooks and lesson materials from private and well-funded schools. Teachers noticed that students from well-funded schools got detailed, helpful answers, while students from under-funded public schools got short, confusing responses — even when asking the exact same question.

🔍 Who was affected?

⚠️ Why did the AI fail?

🔧 How would you fix it?

Verdict This is called historical bias — the training data reflected existing inequalities. The AI learned to give better service to already-advantaged groups, making the gap bigger.

HiKIDAI Activity 04 · Spot the Bias · Case 3 & Summary

Case 03 💼

The Job Sorting Machine

Bias type: Historical inequality in training data

A company built an AI to review job applications for technology jobs. It was trained on 10 years of successful hires from the company's own records. When tested, the AI consistently ranked men higher than women — even when women had identical qualifications. When the company investigated, they found that 90% of past hires in the training data were men, because fewer women had applied and been hired in the past.

🔍 Who was affected?

⚠️ Why did the AI fail?

🔧 How would you fix it?

Verdict The AI learned that "successful tech employee = man" — not because that's true, but because the training data reflected the company's own past unfairness. AI can learn and repeat human mistakes at scale.

🔍 Your Detective Summary

Looking at all 3 cases — what do they have in common?

What is the most important thing a team can do BEFORE releasing an AI?

🌍 Real-World Connection

All 3 cases in this activity are based on real AI systems that caused real harm. Face recognition bias has been documented by researchers at MIT. Hiring AI bias was found at a major tech company in 2018. Education AI inequity has been studied across multiple school districts.

The good news: when people like you spot the bias, companies fix it. AI Detectives make the world fairer. 🕵️

💡 Module B Learning Outcome "If the examples we use to train AI are unfair, the AI will make unfair decisions — even if nobody programmed it to be unfair. We need diverse data and ongoing audits to keep AI fair."

🤖 Kai says: "I can only be as fair as the examples I learned from. If those examples were unfair, I will be unfair — without even knowing it. That is why I need humans to check my work. You are my fairness inspector."