An AI answer can be well written, sound reasonable, and still be false.
That problem did not disappear with newer models. Factuality has improved, sometimes substantially, but OpenAI itself still describes hallucinations as an unresolved challenge: plausible but false statements that can be produced with unwarranted confidence.
Source: OpenAI, “Why language models hallucinate”.
For a business, the cost does not appear only when AI is wrong. It also appears in something less visible: the work required to decide which outputs can be trusted and which must be checked.
A cheap error becomes expensive when it leaves the screen
If you ask AI for ten headline ideas and three are bad, almost nothing happens. You discard them.
If you ask about a contract clause, refund policy, financial figure, or regulation and the wrong answer becomes an action, the economics change.
In 2024, the Civil Resolution Tribunal of British Columbia decided a useful example involving Air Canada. The airline’s chatbot had provided incorrect information about bereavement fares. Air Canada argued, among other things, that it should not be liable for information supplied by the chatbot. The tribunal rejected that reasoning and found that the company remained responsible for information on its website and had not taken reasonable care to ensure the chatbot was accurate.
Reference: Moffatt v. Air Canada, 2024 BCCRT 149.
The lesson is not “never use a chatbot.” It is more operational: when an answer can create an obligation, verification stops being optional.
The bottleneck moves from drafting to checking
A specialist might once have spent twenty minutes drafting an answer and five minutes reviewing it. AI can reduce the draft to thirty seconds while the five-minute review remains.
The expensive part of the workflow has moved. Generation became extremely cheap; verification did not necessarily become cheap with it.
That creates a strange incentive. Producing five alternatives costs almost the same as producing one. Verifying five factual claims still costs more than verifying one.
This is how AI can increase workload even when every individual response arrives faster.
Not every output deserves the same scrutiny
The sensible answer is not to fact-check every generated sentence like a forensic audit. That would destroy much of the advantage.
Match review effort to the downstream cost of being wrong.
| Use | If it is wrong… | Reasonable control |
|---|---|---|
| Internal brainstorming | A weak idea is discarded | Quick review; facts that will not be used do not need checking. |
| Email draft | A wrong fact or tone may be sent | Human review before sending. |
| Source-document summary | A conclusion can be distorted | Check important claims against the document. |
| Figure in a proposal or report | It may drive a decision | Named, current source. |
| Law, contract, or policy | It may create legal or operational consequences | Verify against the primary source. |
| Agent acting without pre-approval | The error can reach third parties directly | Testing, action limits, monitoring, and an escalation path. |
“It came from the document” is not enough
Connecting a model to a PDF, knowledge base, or RAG system reduces one problem: the model no longer has to rely only on information encoded during training.
It can still retrieve the wrong passage, combine two sections, miss an exception, or misread a table. Having access to a source and interpreting that source correctly are different things.
For high-impact work, preserve traceability: page, paragraph, URL, document, or record behind the claim.
A citation does not automatically prove the interpretation is correct, but it makes verification much cheaper.
A better test than asking “How good is the model?”
Take twenty real tasks from the workflow—not demo prompts. For each one measure:
- time to a usable AI draft;
- human review time;
- number of important claims that required a source;
- meaningful corrections made before publishing or acting.
If drafting takes one minute and review takes twelve, you have found the real cost center. If another workflow takes two minutes to generate and thirty seconds to review, that may be a much stronger automation candidate.
Uncertainty is useful information
One interesting point in OpenAI’s research on hallucinations is that evaluation systems can reward guessing when they focus on raw accuracy and penalize abstention too heavily.
There is a practical lesson for users: do not force the model to manufacture certainty when the evidence is insufficient.
Asking it to separate what it knows, what it infers, and what it cannot verify does not make the system infallible. It does make the output easier to use responsibly. When an important claim cannot provide a source or indicate where it came from, that absence is a reason to stop.
The goal is not zero errors. It is stopping expensive errors early.
Mature workflows do not assume a model will never be wrong. They are designed so cheap mistakes die early and costly ones encounter review before becoming an email, payment, contract, decision, or publication.
That is the hallucination “tax” worth measuring: not an unsupported claim about millions of dollars, but verification time plus the cost of errors the process allowed to escape.
If the use case already includes autonomous action, ScalarPivot’s AI Risk Calculator can serve as a first-pass screen for which workflows deserve stronger controls.
