QUESTION · FOR EDUCATORS

Why does AI make things up, and how do students catch it?

AI tools can state false things fluently, often called hallucinations, because they generate likely-sounding text rather than verified facts. Students catch errors by picking out every checkable claim, checking each against a real non-AI source, recording the result, and fixing or deleting what is wrong. Confident does not mean correct.

Written by David Fineberg · Last updated

Why do AI tools make things up?

A generative AI tool predicts what text is likely to come next based on patterns in its training data. On its own it is not looking facts up and has no built-in way to tell what is true. When a pattern points toward a plausible-sounding answer, it produces one, even if the answer is false. The tone stays smooth and sure either way. For the fuller picture, see What is generative AI?

What kinds of errors should students look for?

  • Invented facts. Events, details or explanations that sound right but are not.
  • Fake sources or citations. Book titles, authors, articles or web addresses that look real but do not exist, or that exist but do not say what is claimed.
  • Wrong numbers. Dates, quantities and calculations that are off.
  • Outdated information. Answers that miss recent changes.
  • A confident tone. Wrong answers are often worded as smoothly as right ones, so a mistake can look like a fact.

What routine helps students catch errors?

This is the Scrutinize It pillar of the WISE framework: confident doesn't mean correct, so check AI's answers. A four-step routine works across subjects:

  1. Pick every checkable claim. Underline each name, date, number, quote and statement of fact.
  2. Check each against a real, non-AI source that you can name and find again, such as a textbook, a library resource or an official website.
  3. Record the result. Write what you checked, where, and whether the claim held up.
  4. Fix or delete. Correct what is wrong, and remove anything you cannot verify.

The record from step 3 doubles as part of a use note; see How should students cite or acknowledge AI use? and Can students use AI for schoolwork?

What classroom exercise helps?

Error hunt

The teacher asks an AI tool a question on a topic the class is studying, then shows the answer on a shared screen. In pairs, students mark every checkable claim and look each one up in class materials or approved sources. Pairs report back: which claims were right, which were wrong, and which could not be confirmed.

Follow with discussion questions:

  • Was the wrong part easy to spot from the tone alone?
  • Which claims took the longest to check?
  • What would you do if you could not verify something?

Run this with an answer on a topic you know well, so you can guide the check. Do not use student work or personal information in the demonstration.

Where can educators look up terms?

Words like prompt, training data and hallucination are defined in plain language in the glossary.

Quick answers

Can a better prompt stop AI from making things up?

A clearer prompt can help, but it cannot guarantee accuracy. Every factual claim still needs to be checked against a real source.

Is it a hallucination if the AI cites a real source?

It can still be one. A real source may be described wrongly, so students should open the source and confirm it says what the AI claims.

Free classroom-ready resources

Handouts, a sample school AI policy, a prompt-portfolio set and a printable WISE Compass poster. Enter your email to get them, plus new resources as they are added.

Get the free resources
Free · unsubscribe anytime
ABOUT THE AUTHOR

David Fineberg is an educator and AI-in-education consultant based in Montreal, working with English and French school communities. He is the author of Teaching in the Age of AI: The WISE Canadian Classroom. He offers staff workshops, consultations and talks through Edify AI.