An Exploration of Using Retrieval-Augmented Generation With Access Control

TL;DR

Enterprises want a chatbot over their internal documents, but a naive RAG system happily retrieves and summarizes whatever is most semantically similar to the query — including documents the user has no right to read. This thesis builds a Proof of Concept (over synthetic confidential patient records) that bolts a permission check between retrieval and generation: documents are tagged with an access level at ingest, every query is authenticated with a JWT, retrieved chunks are filtered against the user’s level, and only the surviving chunks are passed to a locally-hosted Mistral-7B. The headline result is a feasibility demonstration: across three test cases, the system answers accurately for authorized data and refuses (politely, with a “ask someone with higher access” hint) for unauthorized data. It’s not a research breakthrough — it’s an honest, end-to-end reference architecture for the single most common security question every AI-services client asks: “how do I stop the bot from leaking things people shouldn’t see?”

Problem & Motivation

The pain is concrete and it lands on anyone selling RAG into a real company.

You point a RAG system at a company’s document store. A user asks “summarize the executive comp plan” or “what’s in patient John Carter’s file.” The retriever does its job: it embeds the query, finds the most similar chunks by cosine distance, and hands them to the LLM. The LLM dutifully summarizes them. Nobody ever checked whether this user was allowed to see those documents. The vector store doesn’t know about org charts; semantic similarity is permission-blind.

The thesis frames the worst version of this: if you instead fine-tune an LLM on sensitive documents, the secrets are now baked into the weights. A user who was never supposed to see a dataset can extract it by simply talking to the model. There’s no “delete this person’s access” — the data has melted into the parameters.

Prior approaches fall short in two ways:

• Fine-tuning on private data offers no access control at all, and is expensive to update when permissions change.
• Vanilla RAG keeps data outside the weights (good) but does retrieval purely on similarity (bad) — the gate that should exist between “found relevant” and “shown to user” simply isn’t there.

The motivating domain is healthcare (HIPAA-style confidentiality), and the practical constraint is that the whole thing must run on-premise — no patient data leaving the network to a cloud API.

What’s New (Core Contribution)

Be honest: this is a Master’s thesis and a PoC, not a novel algorithm. The contribution is integration and a clean reference design, not a new model or training method.

• An access-control gate inserted into the RAG query pipeline. Before: retrieve → generate. Now: retrieve → check each chunk’s permission against the authenticated user → drop unauthorized chunks → generate. The novelty is the placement and the plumbing, not the idea of permissions.
• Permission metadata carried from ingest to retrieval. Each document gets a numeric access level at ingest time, stored in a relational DB keyed by document ID. At query time the system cross-references retrieved chunk IDs against that DB for the current user. Before: vector store holds only embeddings + text. Now: a parallel permission table makes “who can see this chunk” a first-class lookup.
• A “graceful refusal with assistance hint” behavior. When relevant-but-forbidden documents exist, the system doesn’t just say “no” — it tells the user that someone with a higher access level (named, e.g. the AL1 admin) could help. Before: refusal is a dead end. Now: refusal routes the user to a human who can authorize. This is a genuinely nice product touch.
• A fully on-premise stack that a small team can actually run. Mistral-7B quantized via llama-cpp-python on CPU/GPU, LlamaIndex for orchestration, Chroma for vectors, SQLite for users/permissions, FastAPI + React for the app. Nothing here requires a cloud LLM.

What is not new: RAG itself, JWT auth, role-based access control, metadata filtering in vector DBs. The paper combines well-known parts; its value to you is the assembled, working whole.

How It Works (Technically)

The system has two pipelines — ingestion (offline, builds the index) and query (online, answers a user) — and the access-control logic lives entirely in the query pipeline. Let’s trace both, then walk one real example end to end.

The ingestion pipeline (build the searchable, permissioned index)

1. Load documents (PDF/DOCX/PPT) into LlamaIndex Document objects = text + metadata.
2. Customize metadata — attach filename and a structured doc ID ("filename"_"page") so chunks trace back to a source document.
3. Record permissions — write each document’s unique name + doc ID (and its access level) into a relational table. This is the table the gate will consult later.
4. Chunk the documents. They use a SentenceWindowNodeParser: each “node” is a single sentence, but its metadata carries a window of neighboring sentences. (Why: you retrieve precisely on one sentence’s embedding, but you feed the LLM the surrounding context so the answer isn’t starved of meaning.)
5. Embed each node into a vector with BAAI/bge-small-en-v1.5 — a small English embedding model. “Embedding” = turning text into a fixed-length list of numbers where semantic closeness ≈ geometric closeness.
6. Persist the index to disk (LlamaIndex .persist()) so you don’t re-embed every restart.

The query pipeline (where access control bites)

1. Authenticate — user logs in, gets a JWT. The token identifies who is asking, which determines what they may see.
2. Embed the query with the same bge-small-en-v1.5 model (must match ingest, or the geometry won’t line up).
3. Semantic search the vector store for the top-k most similar chunks. This step is permission-blind — it can and will surface forbidden chunks.
4. Permission check (the gate) — for each retrieved chunk, look up its document ID in the relational DB and compare its access level to the user’s. Partition results into accessible and inaccessible sets.
5. Generate — discard the inaccessible chunks, then build the LLM prompt from only the accessible context + the user query. The LLM literally never sees forbidden text, so it cannot leak it. (This is the key safety property: filtering before the prompt, not asking the model to “please ignore” forbidden content.)
6. Assistance info — if relevant-but-forbidden documents existed, append a message suggesting the user contact someone with the required access level. Combine with the LLM answer and return.

The access-control model itself

Deliberately simplified vs. Unix owner/group/other × read/write/execute. Instead, a monotonic numeric hierarchy:

• AL1 (highest): sees everything (AL1 + AL2 + AL3).
• AL2: sees AL2 and AL3, but not AL1.
• AL3: sees only AL3.

So “can user see chunk?” reduces to user_level <= chunk_level (lower number = more power). Clean and easy to administer, but as the author admits, it can’t express real-world lattices (project-scoped, need-to-know, cross-cutting roles).

Architecture & data flow

flowchart LR
  subgraph Ingest[Ingestion - offline]
    D[Documents] --> M[Add metadata + doc_id]
    M --> P[(Permission table\nSQLite: doc_id -> access_level)]
    M --> C[Chunk: SentenceWindow]
    C --> E[Embed: bge-small-en]
    E --> V[(Chroma\nvector store)]
  end
  subgraph Query[Query - online]
    U[User + JWT] --> Q[Embed query]
    Q --> S[Top-k semantic search]
    V --> S
    S --> G{Permission gate\nuser_level <= chunk_level?}
    P --> G
    G -->|allowed chunks| L[Mistral-7B on-prem]
    G -->|denied + relevant| H[Assistance hint]
    L --> R[Answer]
    H --> R
  end

Interactive: a user asks a question; top-k retrieval pulls chunks of mixed access levels (the permission-blind step), then the gate drops everything the user can't see before the LLM ever reads it. Toggle the user's access level and watch which chunks survive to the prompt. Schematic, not the paper's data.

The algorithm, simplified

The whole contribution is one function — the gate between retrieval and generation:

# access_level: lower number = MORE access (AL1=1 strongest, AL3=3 weakest)
def answer_with_access_control(query: str, user) -> str:
    qvec = embed(query)                              # same model as ingest
    hits = vector_store.search(qvec, top_k=5)        # permission-BLIND similarity

    allowed, denied = [], []
    for chunk in hits:
        doc_level = perm_db.level_for(chunk.doc_id)  # lookup in SQLite
        if user.access_level <= doc_level:           # the gate: user strong enough?
            allowed.append(chunk)
        else:
            denied.append(chunk)                     # relevant but forbidden

    # CRITICAL: the LLM only ever sees 'allowed'. Forbidden text never enters the prompt.
    context = "\n".join(c.window_text for c in allowed)
    answer = llm(f"Context:\n{context}\n\nQuestion: {query}")

    if denied:                                       # graceful refusal, not a dead end
        admin = perm_db.who_can_see(denied)          # e.g. an AL1 user
        answer += f"\n\nSome relevant info needs higher access. Ask {admin}."
    return answer

The single most important line is the if user.access_level <= doc_level filter combined with building context only from allowed. Security comes from the forbidden text being physically absent from the prompt — not from trusting the model to keep a secret.

Built on Prior Work

Results & Evidence

What was tested. A synthetic dataset of six (the text also says “eight” in one place — an internal inconsistency) GPT-4-generated patient profiles, split across access levels. Two users: User 1 (AL1, sees all), User 2 (AL2, sees the AL2/AL3 subset). Three test cases:

• TC1 — Retrieve specific info for an accessible profile. System accurately summarized Mark Lee’s medical history, treatment plan, nutrition plan, and medications (with correct drug→purpose mapping: Lisinopril/ACE inhibitor, Metoprolol/beta-blocker, etc.). Passed.
• TC2 — Query spanning multiple profiles. Asked as AL2 to list all accessible patients, it returned exactly the AL2-permitted set (Anna, Mark, Kevin) and, when asked about Mark and an AL1-only patient (Lisa Nguyen), answered only about Mark and withheld Lisa. Passed.
• TC3 — Query an explicitly inaccessible profile. System recognized it had no authorized context and redirected the user to someone with higher access. Passed.

The headline: feasibility proven — fine-grained access control can be layered onto RAG, and the filter-before-prompt design enforces it.

Caveats — and they are large. Read this section before you quote any “result”:

• No quantitative metrics. “Accuracy” and “access-control compliance” are described qualitatively. There are no precision/recall numbers, no retrieval quality scores, no false-allow/false-deny rates. The author explicitly notes the RAG’s retrieval/generation quality was not evaluated.
• Tiny, synthetic dataset. Six-to-eight GPT-4-written profiles is far too small to claim robustness; it also can’t surface retrieval collisions you’d hit at scale.
• Self-graded outputs. Results are the author eyeballing answers against the source, not a held-out benchmark or human raters.
• Prompt injection acknowledged, not solved. The thesis flags that stuffing context into prompts risks injection, and argues RAG reduces exposure by including less text — but there’s no adversarial test of someone trying to jailbreak the gate.
• Security model is coarse. A 3-level total order can’t represent real org permissions; no test of permission changes/revocation propagation.

Net: this is solid evidence that the architecture works on a toy, and zero evidence about how well it holds up under scale, adversaries, or real permission complexity.

How You’d Use It

This maps almost one-to-one onto the most common request an AI-services shop gets: “build us a chatbot over our internal docs, but make sure people only see what they’re cleared for.”

• As a client offering: “Permissioned RAG / Secure Knowledge Assistant.” The thesis is essentially your reference architecture and SOW skeleton. The selling point isn’t the LLM — it’s the gate and the on-prem story you can show to a security/compliance reviewer.
• The defensible design pattern to standardize on: filter chunks before the prompt, never after. Many quick builds do the opposite — they pass everything and tell the model “only answer from documents the user can access.” That is not a control; it’s a suggestion. Lead with this distinction in sales conversations; it signals you understand the difference between a demo and a system that survives audit.
• In a multi-agent system (your ARC MAS background): make the permission gate a shared tool/service that any retrieval-using agent must call, rather than re-implementing per agent. The user’s identity/JWT becomes part of the context object passed between agents; the retrieval tool enforces the gate centrally. This prevents one rogue or sloppy agent from becoming the leak.
• On-prem as the moat for regulated clients (healthcare, legal, finance, defense). The Mistral-7B + llama.cpp + Chroma + SQLite stack is a credible “your data never leaves your network” deliverable. That constraint is exactly what kills cloud-LLM competitors in these deals.
• The “assistance hint” as a UX/upsell feature. Routing denied requests to the right human (the AL1 owner) is both good product design and a natural place to log access-request workflows you can later automate.

Build Your Own (Minimal Recipe)

You can stand up an 80%-value version in a few days. The smallest faithful build:

1. Vector store + embeddings. Chroma (or any local vector DB) + BAAI/bge-small-en-v1.5 (or bge-base for a bit more quality). Use the same model at ingest and query — this is the #1 silent bug if you don’t.
2. Permission table. A dead-simple SQLite table: doc_id -> access_level. Add a users table: user_id -> access_level. That’s the entire ACL substrate to start.
3. Ingest script. Load docs → chunk (LlamaIndex SentenceWindowNodeParser gives you precise retrieval + windowed context for free) → embed → store. At the same time, write each doc’s access level into the permission table.
4. The gate (the only part that matters). Retrieve top-k, look up each chunk’s level, keep user_level <= chunk_level, build the prompt from survivors only. ~15 lines (see the snippet above).
5. LLM. Mistral-7B (or Llama-3-8B today) via llama-cpp-python, quantized to Q4/Q5 so it runs on a single box. Swap to a cloud model behind a flag only for non-sensitive demos.
6. Auth + API. FastAPI with JWT; React (or anything) for the UI.

The two genuinely hard parts:

• Keeping permission metadata correct and in sync as documents are added, re-chunked, moved between levels, or deleted. The chunk→doc→permission mapping must never go stale; a single mislabeled chunk is a leak. This is where production effort actually goes, not the LLM.
• Chunk-level vs. document-level permissions. The thesis permissions whole documents. The moment a single document mixes sensitivity levels (most real docs do), you need per-chunk or per-section levels, and your ingest pipeline gets much harder.

Models/libraries to reach for: llama-cpp-python or Ollama (LLM), sentence-transformers/bge (embeddings), Chroma or Qdrant (vectors), LlamaIndex or LangChain (orchestration), FastAPI + python-jose (auth).

How to Improve It

The limitations are the roadmap. Each of these is a concrete, testable upgrade:

1. Replace the 3-level total order with real RBAC/ABAC. Move to role + attribute based access (groups, project scopes, need-to-know tags). Test: can you express “team A sees project X docs, team B sees project Y, leads see both” without hacks? Store policies as predicates evaluated against chunk metadata + user attributes.
2. Per-chunk and even per-span permissions. Tag at chunk granularity so a single mixed-sensitivity document is partially retrievable. Test: a doc with one AL1 paragraph in an AL3 body should yield the AL3 content to an AL3 user and redact the AL1 paragraph.
3. Adversarial / prompt-injection evaluation. Actually red-team the gate: can a user phrase a query, or embed instructions in their own uploaded doc, to extract forbidden context? Add an injection test suite and report a leak rate. This is where the paper is weakest and where a real product earns trust.
4. Quantitative retrieval + generation metrics at scale. Build a few hundred docs with ground-truth permissions and questions; report retrieval precision/recall, answer faithfulness, and crucially false-allow / false-deny rates for the gate. Without these numbers you can’t certify the control.
5. Audit logging + access-request workflow. Log every (user, query, chunks-allowed, chunks-denied) tuple. Turn the “assistance hint” into a real request-and-approve flow. This is both a security requirement for regulated clients and a sellable feature.
6. Embedding-leak hardening. Even when text is filtered, ensure no forbidden text leaks via cached prompts, logs, or error messages. Test that denied chunks never appear in any persisted artifact.

Glossary

• RAG (Retrieval-Augmented Generation) — answer a question by first retrieving relevant documents, then having the LLM generate an answer grounded in them, instead of relying on the model’s memorized weights.
• Embedding — a fixed-length list of numbers representing a piece of text, arranged so that semantically similar texts land close together geometrically.
• Vector store / vector database — a database (here, Chroma) that stores embeddings and finds the nearest ones to a query embedding quickly.
• Semantic search / top-k retrieval — returning the k chunks whose embeddings are closest to the query’s embedding (k=5 here).
• Chunk / node — a small slice of a document that gets embedded and retrieved independently.
• SentenceWindowNodeParser — a LlamaIndex chunker that indexes one sentence at a time but stores neighboring sentences in metadata, so retrieval is precise but the LLM still gets surrounding context.
• Access level (AL1/AL2/AL3) — this system’s permission tiers; lower number = more access (AL1 sees everything, AL3 sees only AL3 content).
• The gate — (this breakdown’s term) the permission check inserted between retrieval and generation that drops chunks the user isn’t allowed to see before building the prompt.
• JWT (JSON Web Token) — a signed token issued at login that carries the user’s identity, used here to determine their access level on each request.
• On-premise (on-prem) LLM — a model running on the organization’s own hardware/network, so sensitive data never leaves to a third-party cloud API.
• Quantization — compressing a model’s weights to lower precision (e.g. 4-bit) so it runs in less memory, here via llama.cpp / llama-cpp-python.
• Mistral-7B — the open-weight 7-billion-parameter LLM used as the generator.
• RBAC / ABAC — Role-Based / Attribute-Based Access Control; richer permission models than the thesis’s simple numeric hierarchy.
• Prompt injection — an attack where text in the input (the query or a retrieved/uploaded document) manipulates the LLM into ignoring its instructions or revealing what it shouldn’t.