◆Governance
Pointing Claude at your own policies and procedures feels like the safe first project. The documents are yours, the answers are checkable, nothing leaves the building. That is exactly why the errors that show up are quiet ones, and why people learn to trust them before anyone thinks to look.
Ask a retrieval system about your travel policy and it will return the passage most similar to your question. It will not ask whether that passage is still in force. If the 2019 version and the 2024 revision both live in the index, the older one often wins, because five years of edits tend to make language more precise and less like the way people actually ask questions.
The answer comes back confident, cited, and wrong. Nobody catches it because the citation checks out. The document is real. It is just retired.
The control is unglamorous: effective-date and supersession metadata attached at ingestion, retrieval filters that exclude anything superseded, and a deprecation workflow wired to whatever document management system your policy owners already use. Then write eval questions specifically against retired versions and run them before every index update. If the old policy can still surface, you want a failing test to tell you, not a reimbursement dispute.
Your source systems already know who can see what. SharePoint knows. The HR system knows. A vector index, by default, knows nothing. Embed everything into one shared index and you have flattened years of carefully built entitlements into a single searchable pool. A manager asks a routine question and gets a passage from an investigation file they were never cleared to open.
The fix has to happen before retrieval, not after generation. Filtering the model's answer is too late, because the restricted passage already shaped it. Carry the access-control map into the index as document-level metadata, enforce it at query time so restricted content is never retrieved for that user, and log who retrieved what so the audit trail exists when someone asks.
Then probe it. Log in as the most restricted role you have and spend an afternoon asking for things that role should never see. Entitlement testing is the retrieval equivalent of a penetration test, and almost nobody does it until after an incident.
Documents get split into chunks before they are indexed, and splitters do not read for meaning. The rule lands in one chunk: expenses under a threshold are approved automatically. The exception lands in the next: unless the vendor is a contractor, see appendix C. The model retrieves the first chunk, answers with the rule, and cites the correct document. The carve-out never entered the conversation.
This is the failure a spot check is least likely to find, because the answer is true as far as it goes. Structure-aware chunking helps, keeping clauses attached to their exceptions and headings attached to their sections. Parent-document retrieval helps more, pulling the surrounding context back in once a chunk matches. But the real safeguard is an eval set seeded with exception-bearing questions, written by the people who know where the bodies are buried in your policy manual. Ask about the contractor. Ask about the edge case. If the system returns the rule without the exception, it fails, and you know before your users do.
Two failure modes travel together here. The first: your documents disagree, as any large corpus eventually does, and the model blends the contradiction into one fluent answer that matches neither source. Averaging two policies produces a third policy that does not exist. The second: the footnote names the right document, but the specific sentence the model asserted is not actually in it. The citation is real, the claim is invented, and the pairing passes every casual glance.
Both need controls the model cannot provide on its own. Instruct the system to surface disagreement rather than resolve it: when retrieved sources conflict, the answer should say so and show both. For citations, check at the claim level, tracing each assertion back to a specific passage rather than trusting a document-level footnote. And keep a human review step on anything that leaves the building. In our Governed-by-Design work, cited-and-logged is the floor, not the ceiling. A citation that has not been verified against its claim is a formatting choice, not a control.
A citation that names the right document for the wrong claim is more dangerous than no citation at all. It borrows trust it has not earned.
None of these five failures is a model problem. They are pipeline and governance problems, which is good news, because pipelines and governance are things a CIO can inspect, test, and sign off on. Every control above is checkable before go-live: the supersession filter either excludes the retired policy or it does not, the entitlement probe either leaks or it does not, the exception eval either passes or it fails.
This is why we scope retrieval projects as a Governed Pilot with a fixed success criterion agreed up front. If the criterion is not met, the client does not pay. The criterion usually lives in exactly this territory: an eval set the client's own experts wrote, run against the live pipeline, with the audit log open.
| Failure mode | What it looks like | The control that catches it |
|---|---|---|
| Superseded versions | Retired policy retrieved confidently | Effective-date metadata and supersession filters |
| Permission bleed | Restricted files surface from a shared index | Entitlements enforced at query time, logged |
| Severed exceptions | The rule returns without its carve-out | Structure-aware chunking plus exception evals |
| Blended contradictions | Fluent answer matching no source | Conflicts flagged and shown, not averaged |
| Citation drift | Right document, wrong claim | Claim-level citation checks and human review |
Retrieval on your own documents is still the right first project for most regulated teams. Build it as if these five errors are already in the pipeline, because they are, and the free Claude Readiness Assessment is a reasonable place to find out which ones.