SR 11-7 Model Risk Management for LLMs: A Practical Reading for AI Vendors and Buyers
SR 11-7 is 2011 guidance for statistical models, but examiners are already applying it to LLMs. Here's how the three pillars translate.
- Anton ShemereyField CTO
In this article
SR 11-7 was issued by the Federal Reserve and the OCC in 2011, back when a "model" at a bank meant a credit scoring engine, a Basel capital calculation, or a statistical fraud filter. Nobody writing it was thinking about large language models. Yet SR 11-7 remains the baseline standard examiners reach for today, and they are applying it to LLMs and LLM-based agents without a new rulebook to guide them. That leaves banks and the vendors selling into them in an uncomfortable but workable position: there is no SR-11-7-for-AI addendum, so the only defensible path is reasoning forward from the guidance's original principles to what they actually require of a language model in production. This article does that translation, pillar by pillar.
What SR 11-7 actually says
Strip away the examiner-speak and SR 11-7 rests on three pillars.
Robust model development, implementation, and use. The bank needs to understand the conceptual soundness of a model — why it works, what data it was built on, what assumptions it embeds — and needs controls around how it's actually implemented and used in production, not just how it performed in a lab.
Effective validation, independent of model development. Someone who didn't build the model has to test it — checking that it does what it's supposed to do, understanding its limitations, and continuing to monitor it after deployment. Independence is not a nice-to-have here; it's the point. A team validating its own work is not validation under SR 11-7.
Sound governance, policies, and controls. There has to be a model inventory, clear ownership, policies that dictate how models get approved and retired, and a governance structure — often a model risk management committee — that can actually enforce those policies rather than rubber-stamp them.
None of this is exotic. It's what any mature risk function should want regardless of regulation. The friction shows up when you try to map these three pillars onto a system that generates free-text output from a prompt rather than a numeric score from a formula.
What counts as a "model" under SR 11-7
SR 11-7 defines a model in deliberately broad terms: any system, approach, or quantitative method that applies statistical, economic, financial, or mathematical theories and assumptions to process input data into quantitative estimates that inform a business decision. It says nothing about the technology underneath. It doesn't require a regression coefficient or a neural network weight matrix — it requires input data, a processing method, and an output that feeds a decision.
An LLM that summarizes a loan file and flags risk factors for an underwriter, screens transactions for potential fraud, drafts adverse action notices, or converses with a customer about their account is doing exactly that: taking input, applying a method, and producing output that informs — or in agentic setups, directly makes — a decision. "It's just a chatbot" is not a defense once its output touches a covered use case. The guidance was never about the mechanism generating the estimate; it was about the fact that a decision is being informed by something other than direct human judgment, and that something needs to be understood, tested, and governed before it's trusted.
This is also where the accountability question raised in AI Agent Governance in Banking starts to bite: if the LLM is in scope as a model, someone at the bank has to own it the way they'd own any other model — inventory entry, named owner, defined risk tier — well before an agent is given the authority to act on its output.
Scope under SR 11-7 is determined by use case, not by what you call the technology. The same LLM can be in scope for one application and out of scope for another inside the same bank — a model powering credit adverse-action letters is squarely covered; the same model answering employee IT-help-desk questions likely isn't. Vendors and buyers who argue scope at the model level, rather than the use-case level, are having the wrong conversation.
Applying the three pillars to an LLM
The pillars don't change. What satisfies them does.
| SR 11-7 pillar | What it traditionally meant | What it means for an LLM |
|---|---|---|
| Development, implementation, and use | Documented methodology, data lineage, and assumptions behind a statistical or credit model; conceptual soundness review before deployment. | Documented base model choice and version, fine-tuning or RAG data sources, system prompt logic, and known failure modes (hallucination, prompt injection, bias in training data). "Development" now includes the retrieval corpus and prompt chain, not just a trained weight set. |
| Independent validation | Backtesting against historical outcomes, statistical performance metrics (AUC, KS, calibration), sensitivity analysis, benchmark comparison. | Evaluation harnesses run against golden question sets by a team independent of the build team; hallucination-rate and factual-accuracy measurement; adversarial and red-team testing for prompt injection and jailbreaks; ongoing re-evaluation whenever the base model, prompt, or retrieval corpus changes — not a one-time sign-off. |
| Governance, policies, and controls | A model inventory entry, a named model owner, a risk tier, and a change-approval process before a new model version goes live. | All of the above, plus version control over the prompt, the retrieval corpus, and the underlying model as three separately tracked, separately approved artifacts. A prompt tweak or a corpus refresh is a model change under this framework, not an invisible tuning pass, and it needs the same change-control trail as swapping the model version itself. |
What examiners actually ask for
In practice, an examiner reviewing an LLM deployment is going to ask for artifacts, not assurances. The banks that fare best in these reviews have already assembled:
- A model inventory entry naming the LLM, its use case, its risk tier, and an accountable owner — not just "the AI platform" as a single line item covering five different applications.
- A validation report produced by a function independent of the team that built or deployed the system, covering both pre-deployment testing and the methodology for ongoing monitoring.
- Ongoing monitoring evidence — hallucination rate, drift in output quality, and performance against the golden set over time, not just a report generated once at launch.
- A documented human-oversight boundary: exactly which decisions the model can make unassisted, which require human review before acting, and how that line is enforced technically, not just in a policy document.
- Change-control records for every prompt edit, model version upgrade, and retrieval corpus update, each tied to a re-validation event.
Notice what's missing from that list: nothing here demands a specific architecture. A bank running a private, on-premise deployment and a bank running a hosted API can both produce these artifacts — but only if the underlying platform was built to make that documentation possible in the first place, which is exactly the architectural case laid out in Enterprise RAG Architecture: The 6-Layer Framework.
What this means for AI vendors
None of the artifacts above are things a bank can produce alone if the vendor's platform doesn't support producing them. That reframes the buying decision: a vendor gap here isn't a missing feature, it's MRM exposure the buyer inherits on day one.
Concretely, three vendor behaviors should raise flags in procurement. First, a vendor that can't produce validation-style documentation — evaluation methodology, benchmark results, known failure modes — because the model is a black box even to them, which pushes the entire validation burden onto the bank's second line with none of the underlying detail. Second, a vendor whose architecture doesn't support independent monitoring — no API for pulling logs, no way for the bank's own model risk team to run its own golden set against production traffic without going through the vendor. Third, and most common, a vendor that ships prompt and model changes silently as "continuous improvements" rather than versioned, disclosed, re-validatable events — which as the table above shows, is precisely the kind of change SR 11-7 expects to be governed.
The practical version of this for a buyer: ask the vendor to show you their change log for the last six months, ask whether your team can run its own evaluation harness against their system independent of their own testing, and ask what happens — contractually and technically — the next time they update the underlying model. If any of those questions produce a shrug, you've found the MRM gap before the examiner does. This is the same ownership question raised for compliance-first RAG deployments in RAG for Financial Services, extended specifically to the MRM lens.
Frequently asked questions
SR 11-7 was never rewritten for LLMs, and it probably won't be anytime soon — which means the banks and vendors that get this right are the ones translating its principles deliberately rather than waiting for a new rulebook that isn't coming. Talk to a Sphere AI Engineer about building an LLM deployment that produces the validation and governance artifacts your examiners will ask for.