The AI vendor questions an examiner expects you to have asked before signing.

Your next exam will not test whether the vendor's AI is good. It will test whether you can show what you asked, what you got back, what you verified independently, and who accepted the residual risk. If the answer lives in a sales deck and a signed contract, you have a gap.

Buying the model does not move the risk

A community bank that deploys a vendor's AI tool owns the outcomes of that tool. That is not a new principle. Third-party risk management expectations have said for years that you can outsource the activity but not the accountability, and AI vendors fall squarely inside that perimeter. The novelty of the technology does not create a carve-out. If anything, it raises the bar, because the tool's behavior can change after you sign.

So the deliverable of your vendor review is not a decision to buy. It is a file. A dated, organized record of the questions you asked, the answers you received in writing, the checks you ran yourself, the risk rating you assigned, and the name of the person who approved it. An examiner who opens that file and finds a coherent story will move on. An examiner who finds a brochure will start digging.

Get these six things in writing before you sign

Verbal assurances from a sales engineer do not survive an exam. Neither does a marketing page that can be edited after you buy. Every material claim needs to land in the contract, an addendum, or a signed vendor response you can print. Here is the minimum set for an AI purchase.

• Training-use terms. A contractual statement that your data, your members' data, and your prompts are not used to train or improve the vendor's models, and what happens to data in transit and at rest.
• Subprocessor disclosure. Every downstream party that touches your data, including the underlying model provider, with a commitment to notify you before that list changes.
• Model change notification. Advance notice when the vendor swaps, retrains, or materially updates the model behind the product, so your own testing can keep pace with what you actually run.
• Evaluation results on your data. Not benchmark scores on public datasets. Performance on a sample that looks like your loan files, your member communications, your document formats, produced during a trial you control.
• Incident and escalation path. Who calls whom when the tool produces a harmful output or a data exposure, in what timeframe, with what notification obligations to you.
• Exit and data return. How you get your data, your configurations, and your logs back at termination, in a usable format, on a defined schedule, with confirmation of deletion on the vendor side.

An attestation is not evidence

There is a difference between a vendor telling you something is true and you being able to show it is true. Both matter, but they sit in different parts of your file, and examiners weigh them differently. A signed statement that your data is excluded from training is an attestation. A retention schedule in the contract plus your own test confirming records are inaccessible after the window closes is evidence.

You will not be able to independently verify everything, and no one expects you to. The discipline is to sort each vendor claim into one of two buckets, then close the gap on the claims that matter most. For an AI tool, the claims that matter most are the ones about accuracy on your work, data handling, and change control.

Attestation

• Vendor says data is not used for training
• Marketing page cites accuracy figures
• Sales call describes the escalation process
• Security summary lists subprocessors

Evidence you can show

• Contract clause plus your own data-flow review
• Eval results on your sampled documents, dated
• Named contacts and timeframes in the agreement
• Subprocessor list attached as a contract exhibit

Build the file like a stranger will read it

Assume the person reading your diligence file has never spoken to you, does not know the vendor, and is professionally skeptical. That is roughly the situation in an exam. The file should let them reconstruct your reasoning without a meeting.

In practice that means a short decision memo up front: what the tool does, what data it touches, what could go wrong, what you checked, what you accepted and why. Behind it, the signed vendor responses, the eval results with the date and the sample description, the contract clauses that carry the key terms, and a list of open items with owners and due dates. Open items are fine. Undocumented open items are not.

Then keep the file alive. AI vendor risk does not end at signing, because the model behind the product can change. Set a review cadence, log each model change notification you receive, and rerun your evaluation set when the vendor tells you something material moved. A file that stops on the contract date reads as a checkbox exercise. A file with entries from last quarter reads as a program.

Scope the use→Written responses→Independent checks→Decision memo→Sign-off→Ongoing review

Where a specialist partner fits

Most community banks and credit unions do not have a team whose job is evaluating AI vendors, and building one for a single purchase rarely makes sense. This is the work we do. AI Definitive is a Claude specialist and Claude Partner Network member serving regulated industries, and our Governed-by-Design framework builds the artifacts this article describes into the engagement itself: client data is never used to train models, outputs are cited and logged, human oversight is explicit, and access controls are mapped. Engagements are designed to support frameworks like SR 11-7 rather than to substitute for your own compliance judgment.

If you are earlier in the process, the free Claude Readiness Assessment is a structured way to scope what you would deploy and what your file would need to contain. If you are ready to test on your own documents, the Governed Pilot runs at fixed scope with a success guarantee: if the agreed criterion is not met, you do not pay. The Model Governance and Audit Pack turns the resulting evidence into documentation your examiner can actually read.

The vendor will not sit in the exam room with you. Your file will. Build it before you sign, keep it current after you deploy, and the AI conversation with your examiner becomes a review of evidence instead of a test of memory.