RAG Data Leakage: When the AI Reads Files It Shouldn't Quote

The 30-second explanation

Many company assistants use a technique called retrieval-augmented generation, or RAG. When you ask a question, the system first searches a pile of documents, then hands the best matches to the AI so it can answer from them. The leak happens when that search never checks who is asking. If the index holds every file in the company, the assistant can quote a confidential one to anyone, because the search found it relevant and nothing asked whether you were allowed to read it.

How it works

1. 1

   1. Ingest

   Documents from file shares, wikis and mailboxes are split into chunks and stored in a searchable index, often without their original permissions.

2. 2

   2. Ask

   A user asks the assistant a question.

3. 3

   3. Retrieve

   The system searches the whole index for the closest matches, whether or not this user may read them.

4. 4

   4. Answer

   The model writes a reply from those passages, including text the user was never allowed to see.

A language model only knows what it was trained on, so companies connect it to their own documents with retrieval-augmented generation. The documents are cut into chunks and stored in a searchable index. When someone asks a question, the system finds the chunks closest to it and puts them in front of the model, which writes an answer from them. The model has no idea which of those chunks are confidential. It uses what it is given. That means the permission check has to happen earlier, at the moment of retrieval, and it has to use the asker's real access rights from the source system. In many first versions it does not. Every document goes into one big index, the original permissions stay behind in the shared drive, and anyone who can talk to the assistant can reach everything in it. Other leaks come from the index drifting out of date, where a file is locked down or deleted but its chunks live on. An instruction hidden in a retrieved document can also try to steer the assistant, which is the prompt injection problem again. A line in the system prompt asking the model to keep secrets is not access control, because the model can be talked out of it.

Real-world example

How to spot it

• Search that ignores who is asking

  The index mixes everyone's documents, with no per-user or per-group filtering at query time.

• Permissions copied once

  Access rights captured at ingestion and never refreshed, so removed access keeps working through the assistant.

• Answers that cite files the user cannot open

  Quotes or citations from documents the person has no access to in the source system.

• Access control left to the prompt

  A system prompt telling the model not to reveal confidential information is the only barrier.

What to do

1. 1Enforce access at retrieval time: filter results using the asker's permissions from the source system before any text reaches the model.
2. 2Keep permissions in sync, refreshing the index when access changes and deleting chunks when a document is removed or locked down.
3. 3Test with real accounts. Ask as a low-privilege user for things only an admin should see, and log which documents were retrieved for each question.