Verifying AI & Hallucinations

Share / assign

Why it works

AI language tools generate fluent, confident-sounding text that can be factually wrong, they can invent facts, fabricate sources and citations, state falsehoods with total confidence, and make reasoning errors, a phenomenon often called hallucination. Because the output sounds authoritative, users (students and teachers alike) can easily accept false information as true. Using AI responsibly therefore requires the habit and skill of verification: treating AI output as a draft or suggestion to be checked, not a trusted source; verifying facts, figures, and especially sources and citations against reliable references; recognizing the kinds of errors AI makes; and understanding that confidence is not accuracy. Teaching this skill, to students and in one's own practice, is essential for using AI without being misled.

The research: AI literacy; research on AI reliability.

The run-of-show

Choose your slot. The agenda, timings, and length update to match.

The core activity: verify, don't trust, AI output

Teachers leave with a verification habit and a way to teach it.

Understand that AI can produce confident, fluent, but false output (hallucination), including invented sources.
Treat AI output as a draft or suggestion to verify, not a trusted source.
Verify facts, figures, and especially sources and citations against reliable references.
Teach students this habit and skill explicitly.

Confidence is not accuracyAI sounds authoritative even when inventing facts and sources. Treat its output as a draft to verify, check facts and especially citations, and teach students to do the same.

Facilitator notes

Know AI hallucinates confidently, and treat output as a draft to verify.
Check facts and especially sources.
Teach students the verification habit.

Trusting confident AI output gets people believing false thingsAI states falsehoods, and invents sources, with total confidence. Verify facts and citations against reliable references, and teach students that confidence is not accuracy.

Adapt it

Elementary: AI can be wrong, check it, ask an adult.
Secondary: verify facts and sources, recognize hallucinations.
All settings: verify output, check sources, confidence is not accuracy.

Participant handout

One page for every teacher. Print it, or save it as a PDF.

K12 Academics · Professional Learning

Verifying AI: confidence is not accuracy

Check, don't trust: treat AI output as a draft, verify facts and especially sources, and teach students the habit.

How I understand AI hallucination (confident but false, invented sources):
How I treat AI output as a draft to verify:
How I verify facts and especially sources against references:
How I teach students the verification habit:
A verification routine to use:

Make it stick

Confidence is not accuracy:

Verify, don't trust, AI output.
Check sources.
Bring a verification routine to the next PLC.

Common questions

Does AI really make things up?

Yes. AI can invent facts, fabricate sources and citations, and state falsehoods confidently, often called hallucination, because it generates fluent text, not verified truth. Confidence is not accuracy.

How do I verify AI output?

Treat it as a draft to check, not a trusted source: verify facts, figures, and especially sources and citations against reliable references, and watch for confident-sounding errors. Never accept it uncritically.

Why teach this to students?

Because AI output sounds authoritative and students can easily accept false information as true. Verification and recognizing hallucinations are essential skills for using AI responsibly and not being misled.