Enterprise RAG Glossary: 50 Terms Every AI Leader Needs to Know

Enterprise RAG comes with a vocabulary that sits awkwardly between engineering and the boardroom — and conversations stall when a CISO, a CFO, and an ML engineer each mean something different by "accuracy" or "grounding." This glossary fixes that. It defines the 50 terms that matter most for enterprise RAG, in plain language with practical context, so technical and executive stakeholders can actually talk to each other. Definitions are kept short and usable; where a concept has a dedicated deep-dive, it's linked.

A

Adaptive RAG (adaptive routing). A pattern where a lightweight router classifies each query and sends it down the cheapest path that can answer it correctly — simple lookups to standard RAG, complex questions to agentic reasoning. It delivers advanced capability only where it's needed, controlling cost and latency.

Agentic RAG. RAG that adds planning, tool use, and iteration to the retrieve-then-generate loop, so the system can decompose a question, decide where to retrieve, call tools, and retrieve again. It investigates rather than just fetches — powerful for multi-hop questions, but slower and costlier, so it should be reserved for queries that need it.

Air-gapped deployment. Running the entire RAG stack — model, vectors, and ingestion — in an environment with no external network connectivity. Used for the most sensitive defense, intelligence, and critical-infrastructure workloads where nothing can leave the boundary.

Answer relevance. An evaluation metric for whether an answer actually addresses the question asked. An answer can be faithful (every claim grounded) yet have low relevance if it answers a slightly different question.

Audit logging. Recording every query and answer — who asked, what was retrieved, what was generated, which model, and when — in a retainable, often tamper-evident form. It underpins security detection, compliance evidence (SOC 2, HIPAA, FINRA), and quality monitoring, and can be mirrored to a SIEM.

B

BM25. A classic keyword/lexical ranking algorithm that scores documents by exact-term matches, rewarding rare, specific tokens. It excels at acronyms, IDs, and names that vector search misses, which is why it's the lexical half of hybrid search.

BYOK (Bring Your Own Key). A deployment model where the customer controls the encryption keys (and often the model API keys) used by the system, so sensitive data and access stay under the customer's control rather than a vendor's.

C

Chunking. Splitting documents into the retrievable units that get embedded and searched. Structure-aware chunking (by heading, clause, or table) preserves context; naive fixed-size chunking destroys it. Chunking sets the ceiling on retrieval quality. See chunking strategies.

Citation. An inline reference (e.g. [1]) linking a claim in an answer back to the exact source passage it came from — ideally with document, page, and the precise text. Citations make answers verifiable and reduce hallucination.

Confidence score. A signal of how strongly the retrieved context supports an answer, often derived from the strength of the top retrieval match. It lets the system flag low-confidence answers cautiously rather than asserting them, and route uncertain queries to a human.

Context precision. A retrieval evaluation metric: of the passages retrieved, how many were actually relevant. Low precision means noise is crowding the context window.

Context recall. A retrieval evaluation metric: of the information needed to answer, how much was actually retrieved. Low recall is the silent killer — the answer existed but retrieval didn't surface it.

Context window. The amount of text (measured in tokens) a model can consider at once, including the system prompt, retrieved passages, and the question. Tighter, higher-quality retrieval uses the window more efficiently and cheaply.

Cosine similarity. A common measure of how close two vectors (embeddings) are in meaning, used to rank retrieved passages by semantic similarity to the query.

D

Data egress / residency. Egress is data leaving your environment (e.g. to a third-party model API); residency is the requirement that data physically stay in a specific environment or region. For regulated buyers, "no egress" and controlled residency are often non-negotiable.

De-identification. Detecting and masking or removing personal identifiers (such as PHI) from content — ideally during ingestion — to reduce risk and compliance scope where a use case doesn't require identified data.

E

Embedding. A numerical vector representation of text (or an image) that captures meaning, so similar content has similar vectors. Embeddings are what make semantic search possible.

Embedding model. The model that converts text or images into embeddings. Its choice defines what "similar" means to the system and affects retrieval quality, multilingual coverage, cost, and where vectorization can run. See embedding models.

Engram. In SphereIQ, a persistent memory record — of a fact, entity, decision, preference, or insight — that forms as the system works and persists across conversations, recalled alongside document citations. It's the memory layer that makes RAG stateful: the system cites and remembers, rather than starting fresh each session.

Evaluation harness. The tooling and process for measuring RAG quality — scoring retrieval and answer metrics against a golden dataset, both before launch and continuously in production. It turns "it seems to work" into measured, trackable accuracy.

F

Faithfulness. An evaluation metric for whether every claim in an answer is supported by the retrieved context. It's the hallucination metric — the one that matters most in regulated environments.

Fine-tuning. Continuing to train a model on your examples so a style, vocabulary, or narrow skill is baked into its weights. It changes the model (the "how to talk"), versus RAG which changes the context (the "what's true now"). See RAG vs. fine-tuning.

G

Generation. The step where the model synthesizes an answer from the retrieved context. Production generation is grounded (answers only from context), cited, and confidence-aware.

Golden dataset. A curated set of real questions paired with known-good answers and supporting passages, used as the ground truth for evaluating retrieval and answer quality. Without it, accuracy can't be measured.

Graph RAG. An extension of RAG that builds a knowledge graph of entities and relationships and traverses connections across documents, rather than only matching similar text. Powerful for relationship-dense data, overkill for most document Q&A. See Graph RAG.

Grounding. Constraining a model's answer to information that's actually present in the retrieved context, rather than its training data. Grounding plus citations is the core reason RAG hallucinates less than a bare LLM.

Guardrails. Controls that constrain model inputs and outputs — content filtering, injection screening, output limits, and policy enforcement — to keep the system safe and on-policy.

H

Hallucination. A confident, fluent claim that isn't supported by the source (or any fact). In RAG, hallucinations are mostly an upstream retrieval problem, not a model problem. See hallucination prevention.

Hybrid search. Running keyword (BM25) and vector retrieval together and fusing the results, so the system finds the right passage for both exact-term and natural-language queries. It's the production baseline for enterprise RAG. See hybrid search.

I

Ingestion. The layer that connects to enterprise source systems and pulls content into the pipeline — securely, incrementally, and with permissions intact. It's where most RAG projects quietly stall. See data ingestion.

IVFFlat. A vector index type (used by pgvector, among others) that speeds up similarity search by partitioning vectors into lists, trading a little recall for much faster queries at scale.

K

Knowledge base. The corpus of enterprise content a RAG system retrieves from. RAG is only as good as the knowledge base, so keeping it current and free of contradictions ("KB hygiene") directly affects answer quality.

L

LLM (Large Language Model). The model that generates answers from retrieved context. In RAG, the LLM is the most replaceable layer — retrieval quality and grounding matter more — and an LLM-agnostic design keeps it swappable.

M

Metadata filtering. Restricting retrieval to candidates matching specified metadata — department, date, document type, or access level — before similarity ranking. It's essential for both relevance scoping and permission enforcement.

Multimodal RAG. RAG that retrieves from the visual content of documents — images, charts, diagrams, scans, slides — not just text, using OCR and vision models (and sometimes image embeddings). Essential for document-heavy archives. See multimodal RAG.

N

No-answer behavior. A system's ability to recognize when the retrieved context doesn't support an answer and say so — "the documentation doesn't cover this" — rather than fabricating. Explicitly permitting "I don't know" prevents a whole class of hallucination.

O

OCR (Optical Character Recognition). Converting scanned or image-based documents into machine-readable text so they can be chunked, embedded, and retrieved. Critical for the scanned PDFs common in enterprise archives.

P

Permission-aware retrieval. Enforcing access controls during retrieval, before ranking, so a user can only ever retrieve content they're entitled to see. The single most important RAG security control — done at the retrieval layer, not the UI. See RAG security.

pgvector. An open-source extension that stores and searches vector embeddings directly inside PostgreSQL. It keeps vectors in your existing governed database with no data egress, making it a strong owned-control default (and SphereIQ's Knowledge AI default).

Prompt engineering. Designing the system prompt that governs how the model uses retrieved context — enforcing answer-from-context, citations, confidence calibration, and no-answer behavior. Many "RAG hallucinations" are really prompt failures. See prompt engineering.

Prompt injection. An attack that plants malicious instructions in text the model processes. Indirect prompt injection hides instructions inside a document the system retrieves, making retrieved content part of the attack surface — so it must be screened, not just user input.

R

RAG (Retrieval-Augmented Generation). An architecture that retrieves relevant passages from your own knowledge base at query time and has a model answer from them with citations, rather than relying on the model's training data. It keeps answers current, governable, and attributable.

RBAC (Role-Based Access Control). Granting access based on a user's role, applied in RAG so retrieval honors who is allowed to see what. The foundation of permission-aware retrieval in regulated and enterprise deployments.

Reciprocal-rank fusion (RRF). A method for merging two ranked result lists (keyword and vector) into one, rewarding passages that rank highly in either, without needing the two scoring scales to be comparable. A simple, robust way to fuse hybrid retrieval.

Reranking. A step that reorders retrieved candidates by true relevance to the query — often with a cross-encoder — before passing the top few to generation. It turns a strong candidate set into the right top results.

Retrieval. The step that, given a query, returns the most relevant (and permitted) passages from the index. Where most accuracy is won or lost; the strongest retrieval is hybrid and permission-aware.

S

Semantic search. Searching by meaning rather than exact keywords, powered by embeddings and vector similarity. It handles paraphrase and natural-language questions but, alone, misses exact-match terms — hence hybrid search.

System prompt. The instructions that define a model's role, scope, and rules for using retrieved context — including grounding, citation, and refusal behavior. It's the cheapest, highest-leverage layer for controlling answer quality.

V

Vector database. A system that stores embeddings and answers similarity queries at scale (e.g. pgvector, Pinecone, Weaviate, OpenSearch). The "index" layer of RAG; the choice is mostly about operating model and data control, not raw performance. See vector database comparison.

VPC / private deployment. Running the RAG pipeline inside your own cloud account (VPC), on-premise, or air-gapped, so data and queries stay in your environment. The posture regulated and security-first organizations require. See private LLM + RAG.

Need to turn this vocabulary into a deployed system? Get a RAG Readiness Assessment — we'll translate the concepts that matter for your use case into an architecture.

Start with the enterprise RAG pillar guide, then the 6-layer architecture framework, hybrid search, RAG evaluation metrics, and enterprise RAG security.