what your RAG index actually indexed

An engineering team at a 700-person company built an internal search tool in 2025. The tool was straightforward in concept: a semantic search across the company’s documents, using a vector database of embeddings derived from the full text of every document the company had. The intent was to give employees a better way to find internal knowledge, faster than the existing full-text search, smarter about synonyms and context, capable of answering questions directly from document contents.

The engineering team chose their source material carefully. Confluence pages. Google Drive documents. Slack messages, with PII scrubbing. Zendesk tickets. Internal wiki pages. Each source was connected to the indexing pipeline, which ran daily, pulling new and updated documents, embedding them, and storing them in the vector database.

The search tool worked well. Employees adopted it. Queries returned relevant results. The tool became, over several months, the default way to find internal information.

In December 2025, the security team ran a red-team exercise against the search tool. One of the red team members, a junior analyst with a standard employee account, queried the search tool for “the board presentation about the acquisition.” The analyst got back relevant results, including excerpts from a board deck that was supposed to be restricted to the executive team.

The search tool had indexed the board deck during a window when the deck had been temporarily moved to a more broadly-shared folder for a review meeting. The indexing pipeline pulled it. The embeddings were generated. The board deck ended up in the vector database. The restriction on the original file was restored the next day, but the vector database kept the embeddings. The search tool, asked a question, returned the content it had indexed.

The search tool had collapsed every document’s access control into “anyone with access to the search tool.” The access control on the vector database was at the tool level, not at the source-document level. The source documents’ original permissions, which had been designed to segregate sensitive content from general access, were not preserved in the embedding layer.

This is the RAG index problem. It is a problem every company deploying retrieval-augmented generation will face, and most companies are making the same mistakes.

the index strips the permissions

The core mechanic of retrieval-augmented generation is that you take a corpus of documents, split them into chunks, embed each chunk into a vector representation, and store the vectors in a database. When a user asks a question, the question is embedded, and the nearest vectors in the database are retrieved and provided as context to an LLM, which generates an answer using the retrieved context.

This architecture has a subtle but consequential property: the vectors in the database are copies of the content, in a form that is still searchable, still retrievable, and still recoverable as text. The vector database is not a pointer to the original document. It is a derived copy that exists independently.

The access controls on the original document, who can read it, who owns it, what permissions apply, are a property of the original document’s storage system. The vector database doesn’t inherit those controls, unless the deployment was designed to propagate them.

Most RAG deployments are not designed to propagate them. The common deployment pattern is to have a pipeline that reads documents (with whatever credentials the pipeline has, often admin-level), embeds them, and stores the vectors in a database that has tool-level access control. Everyone who can query the tool gets access to every vector in the database.

The source documents might have had permissions that restricted certain content to certain people. The vector database has none of those restrictions. The embeddings are uniformly available to anyone with query access to the tool.

This is a permissions flattening. A carefully-designed permission graph at the source is collapsed into a single tier at the vector database. Every document that was indexed is now available to every queryer.

how an index strips the permissions

1. 1source documentsEach one has its own access rules.
2. 2indexer reads everythingUsually with admin-level credentials, to avoid gaps.
3. 3chunks land in the vector databaseThe text arrives. The access rules don't.
4. 4anyone who can query, can retrieveUnless retrieval checks permissions again.

the indexing pipeline runs with broad permissions

The pipeline that populates the vector database typically runs with broad permissions, for the same operational reasons as any automation that touches many systems. Narrowing the pipeline’s permissions is tedious. Broad permissions are easier.

The pipeline might have credentials equivalent to a superadmin in the source systems. It might be able to read every document, regardless of the specific sharing on any given document. It might ignore or flatten sharing arrangements because its purpose is to index everything.

The result is that documents that should have been restricted to specific people end up in the index if the pipeline’s credentials allow it to read them. The pipeline doesn’t do permission-aware reading. It does indexing.

A specific pattern: documents shared with specific individuals (not with groups or the public) get indexed if the pipeline’s credentials have admin-level access that supersedes the individual sharing. The pipeline sees the document. The embeddings are generated. The content becomes available to every queryer of the tool.

This is a data exposure that most deployments haven’t accounted for. The engineers building the tool focused on making it work, getting the embeddings, building the retrieval, tuning the results. The permissions flattening was a detail that didn’t come up until someone asked a specific query and got a result they shouldn’t have.

permission-aware retrieval is the fix

The fix is known and requires specific design choices.

Permission-aware retrieval means that every query against the vector database is filtered by the querying user’s permissions. The database stores not just the embeddings but also the access control list for each chunk. The query filter checks the user’s identity against the ACLs and only returns chunks the user has permission to see.

Implementing this requires several things. First, the indexing pipeline needs to capture and preserve the source documents’ permissions. Not just read the content, record who has access to each document at index time. Second, the vector database needs to support per-vector metadata, including the ACL. Third, the retrieval query needs to join against the user’s permissions and filter accordingly.

All three of these are technically feasible. None of them is the default in most vector database deployments. The defaults assume that access control is handled at the tool level, not at the chunk level.

The companies that have implemented permission-aware retrieval have typically done so after a specific incident, someone queried for something they shouldn’t have seen, that forced the design. The incident-driven adoption is slower than it should be, but it’s what usually triggers the investment.

Microsoft’s approach with Copilot is to respect SharePoint and OneDrive permissions at query time, which is a form of permission-aware retrieval. This works reasonably well for content stored in Microsoft’s ecosystem, with permissions defined in the Microsoft Graph. It works less well when the RAG system is consuming content from systems outside Microsoft’s permissions model.

Glean takes a similar approach across a broader set of sources, aggregating permissions from the source systems it indexes. Glean’s effectiveness depends on how cleanly the source systems’ permissions can be queried at retrieval time.

Internally-built RAG systems usually don’t do this. The company built the system themselves, focused on core functionality, and permission-aware retrieval was a feature they planned to add but haven’t yet.

retrieval with and without permission checks

without

access control at the tool level

• Anyone with the tool sees every indexed chunk
• Restricted documents surface in answers
• Nobody notices until someone asks the wrong question

with

access control at the document level

• Each chunk carries its source document's permissions
• Every query is filtered by the asking user's rights
• Permission changes sync close to real time

the staleness problem

Permissions on documents change over time. A document that was public in January gets restricted in March. A document shared with a specific team has the team composition change as members leave and join.

The vector database, if it indexed the document in January, has the content. If the database was populated with the January permissions, the database’s ACL for that content is the January ACL. The March change does not propagate to the vector database unless the pipeline re-indexes and updates the ACL.

Most pipelines re-index content, but the frequency varies. Daily re-indexing catches most changes within a day. Weekly re-indexing leaves a window of up to a week during which the permissions are out of sync. Real-time indexing (via webhooks from the source systems) catches changes immediately but is more complex to implement.

The specific failure mode: a document that was public is moved to restricted access. The vector database still has the document’s embeddings and the old ACL. Queries continue to return the document to users who would no longer have access in the source system. The window persists until the re-index catches up.

For permissioned content, this window is a leak. For highly sensitive content, even a short window is unacceptable. The synchronization between source permissions and vector database permissions needs to be close to real-time, not close to nightly.

the deletion problem

The inverse issue: documents that are deleted from the source system remain in the vector database until the pipeline removes them.

A user deletes a document. The source system marks it as deleted. The pipeline, which only reads, doesn’t know to remove the vectors. The vectors persist in the database until the pipeline’s deletion-handling logic runs, which might be nightly, or weekly, or never, depending on how the pipeline was designed.

In the window between source deletion and vector deletion, queries continue to return the deleted content. This is the same problem as stale permissions, applied to a harder edge case. The content exists in the vector database as a copy. The source deletion doesn’t automatically propagate.

The implementations that handle this well have explicit deletion synchronization, deletion events from source systems trigger deletions in the vector database. The implementations that don’t handle it well have content that persists indefinitely after source deletion.

The specific legal exposure here is GDPR deletion requests. A user exercises their right to be forgotten. The source system deletes the user’s data. The vector database retains it. The company’s compliance with the deletion request depends on the vector database being synchronized, which most deployments don’t guarantee.

the exfiltration risk

A specific risk worth calling out: the vector database as an exfiltration target.

The vector database contains a copy of every document the pipeline has indexed. It’s in a structured format that’s optimized for retrieval. An attacker with access to the vector database has, effectively, an organized copy of every document the pipeline had permission to read.

The protections on the vector database are typically standard infrastructure protections, network isolation, credential-based access, cloud IAM controls. These protections are not designed for the specific content of the database. The content is a superset of the source documents, stripped of their individual access controls, consolidated into a single store.

If the vector database is compromised, the exposure is the full corpus. Not a single document. Not a single folder. The full indexed set.

This changes the sensitivity classification of the vector database. It should be treated as among the most sensitive stores in the company, on par with or above the source systems it indexed from. Most deployments don’t treat it this way. The vector database is stored in the same infrastructure as other operational databases, with the same protections.

The mitigation is to treat the vector database as a concentrated sensitive asset and apply commensurate controls. Network isolation. Dedicated access credentials. Encryption at rest. Access logging. Regular audits of who queries the database and for what.

the audit for your RAG deployment

If your company has any RAG deployment, an internal search tool, a customer-facing chatbot backed by your knowledge, an AI assistant that uses your documents, run this audit.

First, map the indexing pipeline. What credentials does it use? What sources does it read from? What permissions does it need to index those sources? Is it using admin-level credentials, or narrower ones?

Second, check the vector database’s access control model. Is there per-document or per-chunk access control? If yes, how is it populated from the source systems? If no, every user of the tool has access to every indexed piece of content.

Third, test the retrieval with specific queries. Pick a document that should be restricted. Query the tool as a user who shouldn’t have access to that document. Do you see the content? If yes, the permission-aware retrieval is not working.

Fourth, check the synchronization cycle. How frequently is the index updated? How are permission changes handled? How are deletions handled? What’s the window during which source changes are not reflected in the index?

Fifth, check the security posture of the vector database itself. Where is it stored? Who has access? Is there encryption? Are queries logged? If someone compromised the infrastructure, what’s the blast radius?

The findings from this audit, in most early RAG deployments, are uncomfortable. The vector database has more access than intended. The synchronization is slower than required. The deletion is incomplete. The permissions flattening is partial or total.

These are fixable. They require investment in the indexing pipeline’s permissions handling, in the database’s access control model, in the synchronization logic, and in the infrastructure protection. None of it is technically novel. All of it is design work that was skipped in the rush to deploy.

Your RAG tool is a useful feature. Your RAG index is a structured copy of your corporate data, detached from the source systems’ permissions. Those are two different facts. The first one gets the investment. The second one needs the investment too, if the first one is going to be safe.

Index less than you have permission to read. Preserve permissions in the index. Synchronize closely. Protect the index like a crown jewel.

The embedding was never the security perimeter. The source system’s ACL was. If your index doesn’t preserve the ACL, the embedding is the leak.