> >
Technical Guide · LLM Reliability

LLM Hallucination Mitigation: Four Mechanisms, Eight Techniques

Most write-ups on hallucination hand you a checklist. This one asks a different question: what is each technique actually doing to the system — and it turns out there are only four real answers.

Every technique for reducing LLM hallucination does exactly one of four things to the system: it changes what the model has access to, it changes what shape the model is allowed to answer in, it changes the model's own parameters, or it checks the model's output after the fact and intervenes if something looks wrong. Once you sort by what a mechanism changes rather than when you'd reach for it, eight seemingly separate techniques collapse into four families — and the family a technique belongs to tells you its real tradeoffs far better than its name does.

This is a survey of that taxonomy: four mechanism classes, what each one costs, and which of eight common techniques falls into which.

Grounding

Gives the model real, external information to work from.

Constraint

Narrows what shape or content the model's output is allowed to take.

Verification

Checks a draft answer after it's written, before it reaches the user.

Adaptation

Changes the model's own parameters, offline, ahead of any request.

In this article
  1. Retrieval-Augmented Generation — ground responses in trusted documents instead of model memory alone.
  2. Prompt Grounding — use clear instructions and constraints to close off room for invention.
  3. Chunking + Reranking — retrieve wide, then prioritize the passages that actually answer the question.
  4. Structured Output — use schemas and validators to make claims mechanically checkable.
  5. Tool Calling — hand off arithmetic and live facts to a calculator, API, or database.
  6. Advanced RAG — verify a draft answer against its sources before it reaches the user.
  7. Fine-Tuning on Trusted Data — shape behavior and terminology, not fast-changing facts.
  8. Confidence Gates + Human Review — escalate the uncertain and high-stakes cases instead of shipping everything.
TRAINING TIME PRE-GENERATION DURING GENERATION POST-GENERATION Fine-Tuning on Trusted Data Retrieval-Augmented Generation Chunking + Reranking Prompt Grounding Structured Output Tool Calling Advanced RAG (Verification) Confidence Gates + Human Review
The same eight techniques, plotted twice: horizontal position is when a mechanism acts relative to a single request; color is which of the four families it belongs to. Notice the families don't line up neatly with timing — grounding mechanisms show up both before and during generation, which is exactly why "pipeline order" and "mechanism type" are two different, both-useful ways to think about this.
Mechanism 01

Grounding — feeding the model real information

A model hallucinates most confidently in the gap between what it memorized during training and what a specific question actually needs. Grounding mechanisms close that gap by handing the model verifiable, external material at the moment it's needed — documents, passages, or live tool results — so the model is reporting on evidence in front of it rather than reconstructing an answer from parametric memory.

01

Retrieval-Augmented Generation

Ground responses in trusted documents, databases, and enterprise knowledge instead of model memory alone.

A model hallucinates most confidently in exactly the gap between what it memorized during training and what the question actually needs — a policy that changed last quarter, a product that shipped after the training cutoff, an internal document it never saw at all. Retrieval-augmented generation closes that gap by fetching relevant passages from a trusted corpus at query time — typically via embedding similarity search over a vector database — and inserting them directly into the prompt as context.

This doesn't make hallucination impossible; it changes its shape. Instead of the model reconstructing a fact from parametric memory, it's now summarizing or reasoning over a passage sitting right in front of it, which is a task LLMs are considerably more reliable at. The failure mode shifts from "the model doesn't know" to "the model wasn't given the right passage" — still a real problem, but a far more debuggable one, since you can inspect exactly what the model saw.

Where it can still go wrong: RAG only helps if retrieval actually surfaces the relevant passage. A corpus with gaps, stale documents, or poor chunking will confidently ground an answer in the wrong context — which is exactly why reranking (#3) exists as its own step.

Mechanism of action: substitutes retrieval for recall, so an answer is reconstructed from a passage rather than pattern-matched from training data.

03

Chunking + Reranking

Retrieve smaller passages, then prioritize the context most relevant to the actual question.

Embedding search is fast but coarse: it ranks passages by how similar their meaning is to the query, which isn't quite the same as how well they answer it. A passage about "return windows" and a passage about "return shipping costs" can sit close together in embedding space while only one of them actually answers a specific question about international returns. Retrieve wide — the top 20 to 50 candidates — then run a second-stage reranker, typically a cross-encoder that scores the query and each passage jointly, and keep only the top 3 to 5 for the model's context window.

Chunk boundaries matter just as much as ranking. A fact split across two chunks (a number in one, its unit or condition in the next) can silently disappear from whichever chunk the reranker doesn't surface. Overlapping chunks, and chunking along natural document boundaries (sections, list items) rather than fixed character counts, catch a surprising share of otherwise invisible retrieval failures.

Mechanism of action: raises the quality of what grounding actually supplies, since retrieval alone tends to surface passages that are topically similar but not always the ones that answer the specific question.

05

Tool Calling

Let the model query search engines, databases, calculators, or APIs when facts need external validation.

Arithmetic, unit conversion, current prices, live system status — these are precisely the categories where a language model will produce a confident, specific-looking number that's simply wrong, because it's pattern-matching against training data rather than computing an answer. Tool calling routes these sub-tasks to something that's actually authoritative: a calculator for math, a live API for a stock price, a database query for an account balance. The model's job narrows to deciding which tool to call and how to phrase the final answer around the result — not to generating the fact itself.

user_query = "What's 18% VAT on a £2,450 invoice, in USD?"

# the model should call two tools, not compute either number itself
vat_amount = calculator.evaluate("2450 * 0.18")      # -> 441.00 GBP
fx_rate    = fx_api.get_rate("GBP", "USD")           # -> live rate
final      = calculator.evaluate(f"{vat_amount} * {fx_rate}")

Mechanism of action: grounding extended into the generation step itself, for facts that don't exist in any retrievable document because they're computed or live.

Mechanism 02

Constraint — narrowing what the model is allowed to say

Grounding gives a model better material; constraint mechanisms restrict what it's permitted to do with any material at all. An ambiguous instruction, or an unconstrained output format, leaves room the model will fill with something plausible rather than something true. These mechanisms close that room off before generation even completes.

02

Prompt Grounding

Use clear instructions, constraints, and few-shot examples to reduce ambiguity and unsupported claims.

An ambiguous instruction leaves room the model will fill — usually with something plausible-sounding rather than something true. "Summarize this contract" invites more liberty than "summarize this contract using only clauses present in the text below, and state 'not specified' for anything the text doesn't cover." The second version removes an entire category of hallucination simply by naming the boundary explicitly.

Three moves do most of the work: give the model explicit permission to say it doesn't know (models rarely volunteer uncertainty unless the prompt tells them that's an acceptable answer); constrain the source of truth ("answer only from the provided context"); and show, not just tell, via one or two few-shot examples of the exact answer shape you want, including an example of a correct "not found" response.

# vague — invites invention
"Answer the user's question about our refund policy."

# grounded — names the boundary and the escape hatch
"Answer using ONLY the refund-policy excerpt below.
 If the excerpt doesn't cover the question, respond
 exactly: 'Not covered in the current policy document.'
 Do not use outside knowledge."

Mechanism of action: removes the invitation to guess, by making silence or refusal an explicitly acceptable output rather than an implicit failure.

04

Structured Output

Use JSON schemas, templates, and validators to make responses predictable and easier to verify.

Free-form prose is hard to check programmatically; a schema turns an answer into fields you can validate mechanically. Requiring a source_id field that must reference an actual retrieved chunk ID doesn't stop the model from wanting to hallucinate a citation — but it does mean an invented one fails validation instead of quietly reaching the user. Constrained decoding (grammar-based generation that only allows schema-valid tokens) or a validate-and-retry loop both work; the point is that the output's shape becomes a checkpoint, not just a formatting preference.

{
  "type": "object",
  "properties": {
    "answer":     { "type": "string" },
    "source_ids": {
      "type": "array",
      "items": { "type": "string" },
      "minItems": 1
    },
    "confidence": { "type": "string", "enum": ["high", "low"] }
  },
  "required": ["answer", "source_ids", "confidence"]
}
// a source_id that doesn't match a retrieved chunk
// fails validation before the answer ever ships

Mechanism of action: turns a claim's shape into a checkpoint, so an invented or malformed answer is mechanically rejectable rather than only stylistically odd.

Mechanism 03

Verification — catching mistakes after they're written

Grounding and constraint both act before or during generation; verification mechanisms accept that some hallucinations will still get through and catch them afterward, before a person ever sees the result. They cost real latency and inference budget, which is why they tend to be applied selectively rather than to every request.

06

Advanced RAG

Add fact-checking, verification, or correction steps before the final answer reaches the user.

Even with good retrieval, generation can drift: a model summarizing three retrieved passages will occasionally merge details across them, soften a qualifier, or state a number slightly differently than the source did. A verification pass — sometimes called chain-of-verification or self-critique — runs a second model call (or the same model with a different prompt) that checks each claim in the draft answer against the specific source passages it cited, sentence by sentence, and flags or rewrites anything unsupported.

This roughly doubles inference cost and latency for the queries it runs on, so it's usually applied selectively: to answers heading to end users in regulated or high-stakes domains, to anything the confidence gate (#8) already flagged as uncertain, or to a sampled percentage of traffic for ongoing quality monitoring rather than every single request.

Mechanism of action: catches drift introduced during generation itself — a merged detail, a softened qualifier — that grounding alone doesn't prevent, because the model had the right source and still stated it slightly wrong.

08

Confidence Gates + Human Review

Escalate uncertain, sensitive, or high-impact outputs for human approval instead of shipping every answer straight through.

Not every answer deserves the same amount of scrutiny, and not every system can afford to verify every answer at the depth described in #6. A confidence signal — self-consistency across several sampled generations, the token-level log-probability of the response, or a lightweight classifier trained on a labeled set of past correct and incorrect answers — gives you a cheap way to sort the easy cases from the ones that need a closer look.

Route low-confidence outputs, and anything touching money, medical, legal, or safety-sensitive territory regardless of confidence, to a human review queue before it reaches the end user. Everything else ships straight through. This is the step that turns "the model is usually right" into a system with a bounded worst case — which is the actual reliability guarantee most production deployments need.

samples = [llm.generate(prompt) for _ in range(5)]
agreement = fraction_matching(samples)  # self-consistency

if agreement < 0.6 or answer.touches_sensitive_domain():
    queue_for_human_review(answer)
else:
    return answer  # ships straight to the user

Mechanism of action: bounds the worst case rather than preventing every error, by routing the outputs most likely to be wrong to a reviewer instead of the end user.

Mechanism 04

Adaptation — changing the model itself

Every mechanism above operates around a fixed model. Adaptation is the one mechanism that changes the model's own parameters — which makes it powerful for some problems and a poor fit for others.

07

Fine-Tuning on Trusted Data

Improve domain behavior and terminology using curated data, while still grounding changing facts externally.

Fine-tuning is easy to reach for as a fix-all and a poor tool for exactly the problem people usually want it to solve. It's genuinely effective at teaching a model your domain's vocabulary, your preferred answer format, and reasoning patterns specific to your task — things that are stable over time. It's a poor way to keep a model current on facts that change, because every update requires a retraining cycle, and the model has no mechanism to "forget" a stale fact it was fine-tuned on before a newer one is trained in.

The practical split: fine-tune for how the model should behave — tone, structure, domain reasoning — and retrieve for what the model needs to know. Treating fine-tuning as a substitute for retrieval, rather than a complement to it, is one of the more common ways teams reintroduce hallucination after believing they'd fixed it.

Mechanism of action: shifts the model's default behavior for a stable class of inputs; it does not substitute for grounding on facts that change faster than the retraining cycle.

Where this mechanism is often misapplied: fine-tuning is the one family that can't be added or removed per-request the way the other three can. Treating it as a substitute for grounding — rather than a complement, for the parts of behavior that are genuinely stable — is one of the more common ways teams reintroduce hallucination after believing they'd fixed it.
Putting it together

Stacking the techniques

These eight don't compete for the same job; each addresses a different failure mode, at a different cost. A practical starting stack for most production RAG systems is retrieval + reranking + prompt grounding + structured output + a confidence gate — that combination is comparatively cheap and catches the majority of ungrounded answers. Tool calling and the verification pass in Advanced RAG add real latency and cost, so they earn their place on specific query types (numeric, high-stakes) rather than every request. Fine-tuning sits outside the per-request path entirely — it's a periodic, offline investment in how the model behaves, not a substitute for anything above.

Where each technique intervenes, and what it costs
TechniqueIntervenes atFailure mode addressedEffortLatency cost
1 · RAGRetrievalMissing or outdated knowledgeMediumLow–Med
2 · Prompt GroundingPrompt constructionAmbiguous instructions inviting inventionLowNone
3 · RerankRetrievalIrrelevant or split contextMediumLow
4 · Structured OutputGenerationUnverifiable or malformed claimsLow–MedLow
5 · Tool CallingGenerationFabricated numbers or live factsMediumMedium
6 · Advanced RAGPost-generationClaims drifting from the sourceMed–HighMed–High
7 · Fine-TuningTraining (offline)Domain style and terminology — not factsHighNone
8 · Confidence GatePost-generationHigh-stakes wrong answers reaching usersMediumVaries

None of this makes hallucination zero. It makes it bounded, visible, and — on the cases that matter most — caught before a person ever sees the wrong answer.

User Query Prompt Grounding • 2 Retrieve + Rerank 1 • RAG  ·  3 • Rerank Generation 4 • schema-constrained Verification 6 • Advanced RAG Confidence Gate • 8 high conf. Answer to user Tool Calling • 5 calculator, search, APIs external facts Fine-Tuning • 7 style & terminology offline, periodic low conf. Human Review • 8 escalation queue approved
Where each technique intervenes along a request's path: prompt grounding and retrieval shape what the model sees before it writes anything; tool calling and fine-tuning feed generation from the side (one per-request, one offline); structured output constrains the draft; verification checks it against sources; and the confidence gate decides whether it ships straight to the user or stops for a human first.
Summary

Choosing mechanisms, not techniques

Once a technique is filed under its real mechanism, the tradeoff question becomes simple: which of the four things do you need to change, and what does changing it cost you?

MechanismChangesPrimary costFails when…
GroundingWhat the model has access toCorpus & retrieval infrastructurethe corpus is stale, wrong, or the right passage isn't retrieved
ConstraintWhat the model is allowed to outputLow — mostly prompt/schema designthe constraint doesn't cover the failure mode it wasn't designed for
VerificationWhether a draft answer ships as-isLatency & inference cost, applied per-requestit's skipped on cost grounds for the request that needed it most
AdaptationThe model's own parametersTraining time, data curation, redeploy cyclesit's asked to keep up with facts that change faster than retraining

None of the four mechanisms is sufficient alone, and none of them is free. A production system that's serious about hallucination usually ends up running at least one mechanism from each family — not because more is always better, but because each family catches a category of failure the other three structurally cannot.

Frequently Asked Questions

What is LLM hallucination?

When a language model produces fluent output that is false or unsupported by its sources.

How can hallucination be reduced?

The guide organises mitigation into four mechanism classes, including grounding in retrieved evidence and constraints on output, and describes eight mechanisms.

Is hallucination fully solvable?

No. Mitigations reduce it, so production systems combine several mechanisms and evaluation.

Where can I read about RAG?

See the RAG Architectures Elucidated page on this site.

What does this page cover?

It is a guide to reducing LLM hallucination in production: four mechanism classes, including grounding and constraints, with eight mechanisms and how to apply them.