How to Debug an AI Agent That's "Hallucinating" in Production

By Carlos Montiel | Enterprise AI Specialist
Leer en español →
Published: 2026-07-28 | By: Carlos Montiel | Reading time: ~5 minutes

"The model hallucinated" isn't a diagnosis, it's a complaint. The real work starts when you turn that vague report into a reproducible root cause -- and most of the time the cause isn't in the model, it's in your pipeline.

Before debugging, define what type of hallucination it is

Not every hallucination has the same root cause. Classify the incident before touching code:

- **Pure factual invention** -- the model asserts something false with no related context in the prompt. Usually a case of misapplied parametric knowledge. - **Contradiction of the provided context** -- the model had the correct information in the prompt (retrieved document, tool result) and still answered something different. This is the most serious, because it means the model is ignoring the grounding you gave it. - **Overgeneralization of a pattern** -- confidently answers something that "sounds" right for that type of question but doesn't apply to the specific case.

The second type almost always points to a pipeline problem (context engineering, not the model). The first and third usually need prompt or model adjustments.

Step one: reproduce with the exact `request_id`

Everything you do depends on being able to reproduce the failure. Log the `request_id` of every call from day one -- it's free and solves 80% of debugging friction later.

response = client.messages.create( model="claude-opus-4-8", max_tokens=1024, messages=messages, ) logger.info("llm_call", extra={ "request_id": response._request_id, "model": response.model, "stop_reason": response.stop_reason, "input_tokens": response.usage.input_tokens, "cache_read_tokens": response.usage.cache_read_input_tokens, })

Without the exact prompt and `request_id` saved, "reproducing" the bug means guessing what was sent to the model three days ago. Save the full payload (system, messages, tools) associated with every response that reaches production -- not just the output.

Step two: audit the context the model actually received

The most common mistake: you assume the model had certain information because "it's in the database," but you never verify it actually made it into the prompt. Before blaming the model, print the exact prompt that was sent and check:

- Was the relevant document retrieved by your RAG step, or did retrieval silently fail? - Did the context arrive complete or was it truncated by a token limit? - Did the tool result the model needed arrive with `is_error: true`, and did the model improvise a response instead of reporting the error?

# Minimal audit: reconstruct exactly what the model saw def audit_context(messages): for m in messages: if isinstance(m["content"], list): for block in m["content"]: if block.get("type") == "tool_result" and block.get("is_error"): print(f"WARNING: tool_result with unhandled error: {block}")

If the model confidently answered something that wasn't in its context, it didn't hallucinate out of caprice -- it probably filled a gap your pipeline left it.

Step three: check if the problem is temperature/effort, not knowledge

On models that support `effort`, a low level can make the model respond quickly without internally verifying its own response against the provided context. If the use case is precision-sensitive (legal extraction, financial data, medical answers), raise the effort level before redesigning the whole prompt.

# If you see hallucinations on tasks where precision > speed response = client.messages.create( model="claude-opus-4-8", max_tokens=2048, thinking={"type": "adaptive"}, output_config={"effort": "high"}, # raise from "low"/"medium" messages=messages, )

Step four: force the model to admit when it doesn't know

A pattern that measurably reduces hallucinations: explicitly instructing that saying "I don't have that information" is a valid and preferred response over inventing something plausible.

system_prompt = """ Answer only with information present in the provided context. If the answer isn't in the context, say exactly: "I don't have that information in the available context" -- do not fill in with general knowledge or assume. If you have to cite a fact, cite the exact context fragment that backs it up. """

This single adjustment usually significantly reduces context-contradiction hallucinations -- but it requires your context to actually contain the information when you need it, or the model will say "I don't know" too often (a different, but more manageable, problem).

Step five: build a regression set, not a one-off fix

When you find a reproducible hallucination case, don't fix it ad hoc and move on -- turn it into a permanent test case. A set of 30-50 real (not synthetic) cases that you run every time you change the prompt or the model is the only real defense against silent regressions.

regression_cases = [ { "input": "...", "context": "...", "must_contain": ["expected_correct_fact"], "must_not_contain": ["fact_it_used_to_hallucinate"], }, # ... 30-50 real cases captured from production ] def run_regression(cases, prompt_version): failures = [] for case in cases: response = execute(case, prompt_version) if any(bad in response for bad in case["must_not_contain"]): failures.append(case) return failures

When the model itself can help you diagnose

An underused technique: ask the model to audit its own response against the context, in a separate second call.

audit_prompt = f""" Here is the context given to a model and the response it generated. Verify, sentence by sentence, whether each claim is backed by the context or not. Context: {context} Response to audit: {original_response} For each sentence in the response, mark: SUPPORTED | NOT SUPPORTED | PARTIALLY SUPPORTED, with the exact quote if applicable. """

This second call is cheap (you can use a smaller model) and detects context contradictions much more reliably than eyeballing the output. It's, in essence, the same principle as the reflection pattern I cover later -- just applied as a debugging tool instead of as part of the production flow.

The real discipline here isn't a single technique, it's the order: reproduce, audit context, adjust effort, force admission of uncertainty, and only at the end touch the underlying prompt -- with a regression set that prevents the next deployment from breaking what you just fixed.

Carlos Montiel
Enterprise AI Solutions Architect
Specialist in LLMs, Agents, and Orchestration
guatemalia.com/en/#contact · info@guatemalia.com

Need to implement AI at your company?

Carlos Montiel is an enterprise AI solutions architect. He implements LLMs, Agents, RAG, and orchestrators for companies across Guatemala and Latin America. Reach out for a consultation.

Contact Carlos Montiel

info@guatemalia.com