The useful mental model is not a knowledgeable assistant. It is an extremely fluent writer who has read a great deal, remembers imperfectly, and will never tell you which parts it is unsure about.
Check anything you would be embarrassed to be wrong about, and specifically: numbers, dates, names, citations and quotations. Confidence in the wording tells you nothing — fluency and accuracy come from the same process.
Why the errors look so convincing
A language model predicts text that fits. A correct answer and a plausible-sounding wrong answer are equally good fits, so nothing in the writing style distinguishes them.
That is why the failures are unsettling: they arrive with the same steady tone as the true parts. There is no wobble to notice.
The four categories that go wrong most
Specific numbers. Statistics, prices, percentages, measurements. Precision reads as authority and is easy to generate without a source.
Dates and sequences. What happened when, and in what order.
Names and attributions. Who said something, who wrote what, which company did which thing.
Citations. References to papers, laws, cases and pages that do not exist, or exist and say something else. Open every one.
A routine that takes two minutes
Ask where it came from. Not for a citation to be produced — for the source to be named so you can go and look.
Check one hard fact against a primary source. A government page, the original paper, the company's own documentation. If that one fails, distrust the rest.
Watch for confident specificity you did not ask for. Unrequested precision is often invention.
Ask the same question differently, in a fresh conversation. An answer that changes shape under rephrasing was never grounded.
Medical, legal, tax and financial specifics about your situation. A model does not know your circumstances, cannot be accountable, and is confidently wrong at exactly the rate it is confidently right. Use it to understand a topic, then take the decision to someone who can be held responsible.
How much checking a claim deserves
Verifying everything is impractical and verifying nothing is how people get caught. The useful question is what a wrong answer would cost you.
| If the answer is wrong… | Checking effort |
|---|---|
| You are mildly misinformed at a dinner party | None needed |
| You make a small purchase you regret | Skim one source |
| You act on it medically, legally or financially | Verify against a primary source, every time |
| You publish it under your name | Verify, and cite what you verified against |
| Someone else relies on it | Verify, and say what you checked |
The middle rows are where judgement lives. The bottom rows are not judgement calls — money, health and law are exactly where a confident wrong answer does real damage, which is the subject of should you trust AI with money or health questions.
What "give me a source" does and does not prove
Asking for sources is worth doing and is not sufficient on its own.
A model can produce a citation that looks entirely plausible and does not exist — right journal, plausible authors, invented title. It can also cite a real paper that does not say what the answer claims. Both failures survive a quick glance, which is precisely why they are dangerous.
So the check is not "did it give me a source". It is:
- Does the source exist? Search the exact title.
- Does it say what was claimed? Open it and look.
- Is it primary? An official regulator or the original study beats a summary of a summary.
- Is it current? Rates, laws and guidance change; a correct 2021 answer can be wrong now.
A model with live web search attached is in a different position — it can retrieve and quote a real page. That removes the fabricated-citation problem but not the misreading one, and a retrieved source can still be low quality.
Where it is genuinely strong
Explaining a concept you can then verify. Drafting something you will edit. Summarising a document you supply. Suggesting angles you had not considered.
The pattern: it is reliable when you bring the facts and it brings the structure — and least reliable when you ask it to supply facts from memory.
Related reading
This is general information — see our disclaimer.
Frequently asked questions
Why do AI models state wrong things confidently?
They generate text that fits the pattern of a good answer. Fluency and accuracy are produced by the same process, so confidence carries no information about correctness.
Which answers need checking most?
Numbers, dates, names, citations, quotations, and anything legal, medical or financial. Those are the categories where errors are both most likely and most costly.
Does asking the model to check itself work?
Only partly. It can catch some errors, but it can also restate a wrong answer more convincingly. Verification has to come from outside the model.
Are citations from AI reliable?
Treat every citation as unverified until you open it. Plausible-looking references to papers, cases and pages that do not exist are a well-documented failure.
Sources
Found an error? Email us and we will fix it and note the change at the bottom of this article. Hello@daily-atlas.com
