Language models are trained to complete text plausibly, not to measure their own certainty. Teaching them to say "I don't know" means going against that default bias -- and there are concrete techniques that pull it off consistently.
Language models' training bias favors answers that sound complete and confident over honest ones that admit a gap. This isn't an isolated bug -- it's a direct consequence of how these systems are trained and evaluated: an incomplete or hesitant answer gets penalized more in many benchmarks than an incorrect but fluent one. The result is that, with no explicit instruction, the model almost always prefers inventing something plausible over admitting an information gap.
The solution isn't a single prompt trick -- it's a combination of explicit instruction, structural constraint, and downstream verification.
The most effective instruction isn't "be honest" (too vague to change the model's behavior) -- it's giving explicit, unambiguous permission to respond with uncertainty, and specifying exactly which phrase to use.
The phrase "you will not be penalized for..." looks unnecessary but is effective: it directly counteracts the training bias toward answers that sound complete.
Asking the model to "say if you're not sure" inside a prose response is fragile -- easy to skip under pressure to complete the task. Forcing an explicit confidence field into a structured output turns uncertainty into a data point, not an opinion the model can omit.
With this schema, your application can treat `confidence: "insufficient_information"` as an explicit code branch -- show a different message, escalate to a human, or ask for more context -- instead of relying on the model spontaneously mentioning it in the text.
Before assuming you need more prompting, check whether the model simply doesn't have the information in its context. A model well-instructed to say "I don't know" will still invent things if its context doesn't include the correct fact -- because from its perspective, there's no difference between "this isn't in my context" and "this doesn't exist." If your system is RAG, check retrieval recall first (see the RAG context engineering article) before assuming the problem is model calibration.
For high-risk cases, a second call auditing the first response against the context detects unsupported claims more reliably than asking the same model, in the same call, to self-evaluate while generating.
Using a cheaper model for this verification step is reasonable -- the task of "comparing a claim against the source text" is simpler than the original generation, and doesn't need the bigger model.
Some models, faced with certain risk categories, outright refuse to answer instead of trying to respond with low confidence. This is a different case from "I don't know" but your code must handle it -- never assume `response.content` always has useful content without checking `stop_reason` first.
A well-calibrated model isn't one that never makes mistakes -- it's one that, when it's wrong, expressed low confidence; and when it's right, expressed high confidence. To measure this on an evaluation set:
If your evaluation shows the model declares "high confidence" at the same accuracy rate as "low confidence," the problem isn't that the model doesn't know when to doubt -- it's that your prompt isn't giving it a real reason to differentiate, and you need to revisit the calibration instruction from scratch.
Carlos Montiel is an enterprise AI solutions architect. He implements LLMs, Agents, RAG, and orchestrators for companies across Guatemala and Latin America. Reach out for a consultation.
Contact Carlos Montiel