RAG Data Ingestion: Connecting Enterprise Systems Without Breaking Security

Ask where enterprise RAG projects die, and most people will say "the model" or "the vector database." They're wrong. RAG projects die quietly in the ingestion layer — the unglamorous job of getting knowledge out of the dozens of systems it lives in and into a form the system can retrieve, without losing permissions, mangling formats, or going stale.

It dies quietly because the demo always works: someone uploads a clean folder of PDFs and the assistant answers beautifully. Then reality arrives — the real knowledge is spread across SharePoint, a CRM, a wiki, a SQL database, Slack threads, and a decade of email and legacy documents, each with its own permissions and quirks. Enterprise RAG data ingestion is a solved problem, but only if you architect it correctly from day one. Here's how.

Source connectors: meeting knowledge where it lives

The first principle: your assistant is only as good as the systems it can reach. Enterprise knowledge is federated by default, so the ingestion layer needs connectors across the real estate where work actually happens — SharePoint, Confluence, Salesforce, SQL databases, S3, SAP, REST/custom APIs, PDFs and Word documents, Slack, and Teams. Each connector has to do three things well: authenticate securely (ideally with scoped service credentials), pull content reliably, and carry metadata and permissions along with the content.

The PetroLedger engagement is the textbook case for why connector breadth matters. Critical institutional knowledge was buried across emails, legacy documents, training videos, workflows, and the heads of subject-matter experts — none of it in one tidy repository. The value came from unifying those scattered sources into a single generative knowledge platform, which is what cut new-hire time-to-productivity from 8–12 months toward 3–5 and drove roughly $1.2M/year in savings. The connectors weren't a detail; they were the project.

Permission inheritance: architecture, not cleanup

Here is the rule that separates a secure ingestion layer from a liability: permissions are part of the data, and they must be ingested with it. When a document comes in from SharePoint, its access controls come in too. When a record comes from Salesforce, its visibility rules come with it. Each chunk inherits the access model of its source, and that inheritance is enforced later at retrieval time — so a user can only ever retrieve from sources they're already allowed to open.

Teams that treat permissions as a later "cleanup" task build themselves a breach. Once everything is flattened into one index with no source-level access control, there's no clean way to re-impose it — and the first time the assistant surfaces a restricted document to the wrong person, the project is over. RBAC must be enforced at retrieval time, which means permission metadata must be captured at ingestion time. Architect it in on day one; you cannot bolt it on at the end. (We go deep on this in enterprise RAG security.)

Document format handling: the messy middle

Enterprise content is not clean text. The ingestion layer has to turn a chaotic mix of formats into consistently chunkable, retrievable units:

• PDFs — the hardest common case. Native-text PDFs are fine; scanned PDFs need OCR, and complex layouts (multi-column, tables, figures) need layout-aware extraction or you'll shred the meaning.
• DOCX / Office — generally clean text, but headings, tables, and embedded objects need structure-aware parsing so they chunk well.
• HTML / wiki pages — strip boilerplate (nav, footers) and preserve the real content structure.
• Structured data (SQL, spreadsheets, CRM records) — not "documents" at all; rows and fields need to be rendered into retrievable text with their schema/context intact, or a number arrives with no idea what it measures.

The output of this stage feeds directly into chunking, so format handling and chunking strategy have to be designed together: extract structure, preserve it, and attach metadata (source, title, section, access level) to every unit.

Incremental vs. full re-index

Enterprise knowledge changes constantly, so ingestion is not a one-time load — it's a living pipeline. The choice of refresh strategy matters for both freshness and cost:

• Full re-index — re-process and re-embed everything on a schedule. Simple, but expensive and slow at scale, and it leaves a freshness gap between runs.
• Incremental / change-data-capture — detect what changed in each source (new, updated, deleted) and re-process only those items, ideally on update triggers. Far cheaper and fresher, at the cost of more sophisticated connectors that track state.

For anything beyond a small static corpus, incremental refresh is the production answer: it keeps answers current without re-embedding millions of unchanged chunks, and it handles deletions — critical for both accuracy and compliance (when a document is removed or a person exercises a deletion right, it must disappear from the index, not linger). This is why Sphere treats ingestion as a designed data-pipeline blueprint — connectors, format handling, permission capture, and update triggers — rather than a one-off import script.

The ingestion layer, done right

Put it together and a production ingestion architecture looks like this: secure connectors reach every system where knowledge lives → content is extracted format-aware, preserving structure → permissions and metadata are captured and attached to every unit → chunks are embedded and indexed → and incremental update triggers keep the whole thing fresh and correct as sources change. Get this layer right and everything downstream — retrieval, citations, security — gets easier. Get it wrong, and the best model in the world is answering from stale, mangled, or improperly-permissioned data.

Worried your data isn't RAG-ready? Get a RAG Readiness Assessment — we'll map your source systems, permissions, formats, and refresh needs before you build.

Related: the enterprise RAG pillar guide, RAG chunking strategies, and enterprise RAG security and governance.