Integrating Access Control with Retrieval-Augmented Generation: A Proof of Concept for Managing Sensitive Patient Profiles

TL;DR

LLM chatbots over private data have a quiet security hole: the model will happily answer from any document in its index, regardless of who is asking. This is a non-starter in healthcare, where GDPR and basic ethics demand that only a patient’s own care team see their records. The authors propose the obvious-but-underbuilt fix: filter retrieved chunks against an access-control list before they reach the LLM’s context window, so a restricted user’s prompt is literally constructed from only the documents they’re permitted to read. They built a working proof of concept (FastAPI + React + LlamaIndex + ChromaDB + a local Mistral-7B) and showed it (a) correctly refuses out-of-scope data while still answering in-scope parts of the same question, and (b) matches GPT-4-grade answer relevancy on permitted content. The contribution is not a new algorithm — it’s a clean, deployable pattern for permission-aware retrieval that an AI services shop can ship to regulated clients almost immediately.

Problem & Motivation

The concrete pain: a private-data chatbot has no concept of “who is asking.” Standard RAG indexes every document into one vector store and, at query time, retrieves the top-k most semantically similar chunks and stuffs them into the prompt. Nothing in that loop asks “is this user allowed to see this chunk?” So a nurse on Ward A, querying the assistant, can pull a patient’s full psychiatric history from Ward B just by phrasing the question well. For a healthcare deployment that is an instant GDPR violation and a breach of trust.

Why prior approaches fall short:

• Role-Based Access Control (RBAC) is mature and works great for databases and APIs — but it governs rows and endpoints, not generated text. It has no hook into “what tokens did the LLM condition on.” Putting RBAC in front of your app login does nothing to stop the model from leaking a document it already has in context.
• Most LLM-security research targets generic risks: prompt injection, training-data extraction, PII leakage from weights. Useful, but orthogonal. None of it answers “make the output respect this specific user’s permissions.”
• Fine-tuning per user doesn’t scale — you can’t train a model per access tier, and the model still memorizes whatever it saw.

The insight: RAG’s architecture already has a natural choke point. Retrieval is a discrete, inspectable step that happens before generation. If you filter there, the forbidden text never enters the context window, so the model cannot leak what it never received. The permission check moves from “police the output” (hard, probabilistic) to “police the input” (easy, deterministic).

What’s New (Core Contribution)

Be honest: this is a systems/PoC paper, not an algorithmic breakthrough. The novelty is in the integration pattern and the evaluation, not in inventing a new mechanism.

• Permission-aware retrieval as a first-class pipeline stage. Before: RAG retrieves by semantic similarity alone. Now: retrieval is a two-stage filter — semantic match then cross-reference each candidate document against a permissions database keyed by the requesting user, dropping anything they can’t access before assembling the prompt.
• A dual-database design separating content from authorization. Before: access rules (if any) are tangled into app logic. Now: a vector DB (ChromaDB) holds embeddings and a separate SQLite permissions DB maps document_id → allowed users/roles. The two are joined at query time. This separation is the actually-reusable idea.
• Graceful, permission-aware refusal. Before: a blocked query either errors out or silently omits data. Now: the system answers the parts it can and explicitly tells the user “I don’t have access to Lisa Nguyen’s plan — ask user1, who has level-1 access.” Partial answers on mixed-permission questions are handled cleanly.
• Empirical evidence it doesn’t cost quality. Before: unproven assumption that filtering degrades answers. Now: RAGAs metrics show the local, access-controlled RAG matches or beats a GPT-4 non-RAG baseline on Answer Relevancy while hitting perfect Context Recall.

How It Works (Technically)

The system has two flows — ingestion (offline, when documents are added) and query (online, when a user asks). The entire access-control trick lives in one added step in the query flow.

Ingestion flow (build the indexes):

1. Take each source document (here, a patient profile of 700–800 words).
2. Chunk it — split into smaller passages so retrieval can target the relevant slice rather than the whole document.
3. Embed each chunk — run it through BAAI/bge-small-en-v1.5, an embedding model that maps text to a fixed-length vector where semantically similar text lands nearby. (“Small/en” = lightweight, English-optimized; chosen so it runs locally for privacy.)
4. Store the vectors in ChromaDB (the vector database).
5. In parallel, update the permissions DB (SQLite): record, per document, which user IDs or roles may access it. This is the table that makes the whole thing work.

Query flow (answer a question, with the gate):

1. User submits a query and their identity is known from auth.
2. Embed the query with the same embedding model, so it lives in the same vector space as the chunks.
3. Semantic search: find the top-k chunks whose embeddings are closest to the query embedding. “Closest” almost always means highest cosine similarity — the cosine of the angle between two vectors. Two vectors pointing the same direction score 1.0; orthogonal ones score 0. The math is just dot(a, b) / (||a|| * ||b||) — a normalized dot product. No deep learning at this step; it’s pure geometry over the vectors the embedding model produced.
4. THE NEW STEP — permission filter: for each retrieved chunk, look up its source document_id in the permissions DB and check whether the requesting user is on the allow-list. Drop every chunk the user can’t access. This is an ordinary database join / set-membership test — deterministic, auditable, and impossible for the LLM to talk its way around because it runs outside the model.
5. Assemble the prompt from only the surviving (authorized) chunks plus the user’s question.
6. Generate with Mistral-7B (a 7-billion-parameter open-weight model, run locally so patient data never leaves the box). Because the forbidden text was never placed in the context, the model literally cannot quote it.
7. Graceful refusal: if a question spans accessible and inaccessible documents, the authorized chunks still produce a partial answer, and the prompt/template nudges the model to say what it lacks access to and who to ask.

The key conceptual point: security is enforced by what enters the context window, not by trusting the model to behave. That’s why this is robust — the LLM is downstream of the gate.

Architecture & data flow

flowchart LR
  subgraph Ingestion
    D[Patient documents] --> CH[Chunk]
    CH --> EMB1[Embed chunks]
    EMB1 --> VDB[(Vector DB / ChromaDB)]
    D --> PERM[(Permissions DB / SQLite)]
  end
  subgraph Query
    U[User + identity] --> Q[Query text]
    Q --> EMB2[Embed query]
    EMB2 --> SS[Semantic search top-k]
    VDB --> SS
    SS --> GATE{Permission filter:\nuser allowed for doc?}
    PERM --> GATE
    GATE -->|allowed chunks only| PR[Assemble prompt]
    GATE -->|dropped| X[Excluded]
    PR --> LLM[Mistral-7B local]
    LLM --> A[Permission-aware answer]
  end

Schematic of the query flow. Toggle the user's access level and watch which retrieved chunks survive the permission gate and reach the LLM context. Forbidden chunks (red) never enter the prompt, so they can't be leaked. Illustrative, not the paper's exact data.

The algorithm, simplified

The contribution is the four lines around # --- ACCESS CONTROL ---. Everything else is vanilla RAG.

# Permission-aware RAG: the gate lives between retrieval and generation.
# Stubs: embed(text)->vec, vector_db.search(vec,k)->[Chunk], llm(prompt)->str
# perms[doc_id] -> set of user_ids allowed to read that document.

def answer(query: str, user_id: str, k: int = 8) -> str:
    qvec = embed(query)                          # query into the shared vector space
    candidates = vector_db.search(qvec, k=k)     # top-k by cosine similarity (semantic only)

    # --- ACCESS CONTROL: the entire contribution ---
    allowed = [c for c in candidates
               if user_id in perms[c.doc_id]]    # set-membership check vs permissions DB
    blocked = [c for c in candidates if c not in allowed]
    # -----------------------------------------------

    if not allowed:
        # nothing this user may see — refuse, and point them to someone who can
        owners = {u for c in blocked for u in perms[c.doc_id]}
        return f"You don't have access to that. Ask one of: {owners}."

    context = "\n\n".join(c.text for c in allowed)   # ONLY authorized text enters the prompt
    prompt = (
        "Answer using ONLY the context. If part of the question concerns information "
        "not in the context, say you lack access and suggest asking an authorized user.\n\n"
        f"Context:\n{context}\n\nQuestion: {query}"
    )
    return llm(prompt)        # the model physically cannot leak what it never received

Note what makes it secure: the gate is plain Python over a database, upstream of the model. There’s no prompt the user can craft to retrieve a chunk that was filtered out — it isn’t in context to begin with.

Built on Prior Work

Results & Evidence

Setup: GPT-4 synthesized 40 patient profiles (700–800 words each). Two access levels — AL1 (full access to all 40) and AL2 (restricted to three profiles). Hardware: one GCE instance with 16 vCPUs, an NVIDIA L4 GPU, 64 GB RAM. Two evaluation tracks: (1) does access control hold, and (2) RAG vs. non-RAG quality via RAGAs.

Access control held. For “What symptoms do Mark Lee and Lisa Nguyen have?”, an AL2 user (allowed Mark, not Lisa) got Mark’s symptoms plus an explicit “ask User1 for Lisa.” AL1 got both. The mixed-permission case — the genuinely hard one — worked: partial answer on the allowed half, clean refusal on the rest.

Quality didn’t suffer (Table 2, RAGAs).

The local, filtered RAG slightly beat the GPT-4 non-RAG baseline on Answer Relevancy and hit perfect Context Recall (every relevant chunk was retrieved).

What the evidence does NOT establish — read this part carefully, because the authors are candid about it:

• Tiny, synthetic dataset. 40 GPT-4-generated profiles. No real distribution of messy clinical notes; external validity is low and they say so.
• Two-level numeric access model. AL1/AL2 is a toy. Real healthcare needs relationship-based, time-bound, break-glass, and hierarchical permissions. The hard part of access control was not tested.
• Context Relevancy is alarmingly low (~0.07). That means most retrieved context is not tightly relevant even though the right answer still emerges — a sign of loose chunking/retrieval. Perfect recall with near-zero precision suggests it’s over-retrieving and getting lucky on small data.
• No adversarial / red-team testing. They didn’t try prompt-injection or query-crafting attacks against the gate. The design should resist them (gate is upstream of the model), but it’s unproven.
• No scale or latency numbers for many users, large corpora, or concurrent queries.

Net: strong evidence the pattern works at PoC scale; no evidence it survives production access-control complexity or scale. The authors frame it honestly as a proof of concept.

How You’d Use It

This maps almost one-to-one onto an AI services offering for regulated clients (healthcare, legal, finance, HR). The pattern is the product.

• “Compliant RAG” as a packaged engagement. Most clients who want a chatbot over internal docs assume permissions are handled. They aren’t, by default. Selling permission-aware retrieval as a named deliverable (with an audit log of what was filtered) is a clear, billable scope that de-risks their legal exposure.
• Slot it into an existing agentic system as a tool-level guard. In a multi-agent setup (your ARC MAS experience applies directly), the retrieval tool becomes identity-scoped: every agent call carries the end-user’s identity, and the retriever returns only authorized chunks. This prevents a “helpful” sub-agent from laundering restricted data into a shared scratchpad — a real leak vector in MAS.
• Audit and explainability built in. Because filtering is a deterministic DB join, you can log exactly which documents were excluded for which user on which query. That log is gold for compliance reviews — far easier to defend than “we trust the model.”
• On-prem / sovereignty story. The all-local stack (Mistral-7B, bge embeddings, ChromaDB) is exactly what privacy-conscious clients want to hear: no patient data leaves their infrastructure, no third-party API sees PHI.
• Graceful refusal as UX. “You can’t see this, ask user1” is a better client experience than a hard error and reduces support load.

Realistic effort: a competent team can stand up the core gate in days, not months — the components are all off-the-shelf. The value-add (and where you earn the fee) is the permissions model and the audit trail, not the RAG plumbing.

Build Your Own (Minimal Recipe)

The smallest version that captures ~80% of the value:

1. Vector store with metadata filtering. Use ChromaDB, Qdrant, or pgvector. Critically, store doc_id (and ideally allowed_roles) as metadata on every chunk. Modern vector DBs support metadata filters inside the search call — push the permission filter down so you don’t even retrieve forbidden chunks.
2. Permissions store. Start with one table: (doc_id, principal_id, role). SQLite or Postgres. This is your source of truth; keep it separate from content.
3. Embeddings. bge-small-en-v1.5 (or bge-base) via sentence-transformers locally. Same model for ingest and query — non-negotiable, or the spaces won’t match.
4. The gate — the 4 lines in the pseudocode above. If your vector DB supports metadata filters, fold the allow-list into the query (where={"allowed_roles": {"$in": user.roles}}) so filtering happens at search time.
5. LLM. Mistral-7B / Llama-3-8B locally via Ollama or vLLM for on-prem; or any hosted model if data residency allows.
6. Refusal prompt + template. Instruct the model to answer only from context and to name an authorized contact for missing pieces.

The two genuinely hard parts:

• The permissions model itself. AL1/AL2 won’t survive a real client. You’ll need roles, hierarchies, relationship-based access (this doctor ↔ this patient), and time/consent bounds. This is 80% of the real work and where domain modeling matters.
• Chunk-level vs. document-level permissions. A single document may mix access tiers (a record with a “sensitive notes” section). Document-level allow-lists are easy; sub-document redaction is hard and the paper sidesteps it.

Reach for: llama-index or langchain for orchestration, chromadb/qdrant for vectors with metadata filters, sentence-transformers for embeddings, ollama/vllm for local inference, casbin if you want a real policy engine instead of hand-rolled checks.

How to Improve It

Limitations are the roadmap. Each of these is concrete and testable:

1. Swap numeric levels for a real policy engine. Integrate Casbin or OPA (Open Policy Agent) so access decisions support RBAC + ABAC + relationship-based rules, break-glass, and time bounds. Test: encode realistic hospital policies and verify the gate enforces them on the same query set.
2. Push filtering into the vector search (pre-filter, not post-filter). The paper retrieves top-k then filters, which can starve allowed results when most top-k are forbidden. Add metadata pre-filtering so you fetch top-k among permitted chunks. Test: measure recall when a user is permitted only a small slice of the corpus — post-filter will drop badly, pre-filter won’t.
3. Attack the gate. Run a red-team suite: prompt injection, jailbreaks, query-splitting, and embedding-collision attacks. The design should hold (gate is upstream of the model), but prove it and publish the failure modes.
4. Fix the precision problem. Context Relevancy ~0.07 says retrieval is sloppy. Add a reranker (e.g., bge-reranker) after retrieval and before the gate, and tune chunk size/overlap. Test whether precision rises without hurting that perfect recall.
5. Sub-document redaction. Move from document-level to span-level permissions: tag sensitive spans, redact them from authorized chunks before they hit the prompt. Lets one document serve multiple access tiers — the realistic case.
6. Scale and latency evaluation. Benchmark with tens of thousands of documents, thousands of users, concurrent queries, and report p95 latency. The PoC’s silence on scale is the biggest gap between it and a sellable product.

Glossary

• RAG (Retrieval-Augmented Generation) — pattern where you fetch relevant documents and put them in the LLM’s prompt so it answers from real data instead of memory.
• Access control — rules deciding which identities may read which resources; here applied to retrieved chunks.
• RBAC (Role-Based Access Control) — assign permissions to roles, users to roles; classic enterprise access model.
• ABAC (Attribute-Based Access Control) — finer-grained: decisions based on attributes (department, relationship, time), not just role.
• Embedding — a fixed-length vector representation of text where semantic similarity ≈ geometric closeness.
• Cosine similarity — normalized dot product measuring the angle between two vectors; the usual “closeness” metric for embeddings.
• Vector database — store optimized for nearest-neighbor search over embeddings (e.g., ChromaDB).
• Chunking — splitting documents into smaller passages so retrieval targets the relevant slice.
• Top-k retrieval — return the k most similar chunks to the query.
• Pre-filter vs. post-filter — applying the permission check during search vs. after retrieving top-k.
• RAGAs — Retrieval-Augmented Generation Assessment; automated, reference-light metrics for RAG quality.
• Answer Relevancy — RAGAs metric: how well the answer addresses the question.
• Context Recall — RAGAs metric: fraction of needed information actually retrieved.
• Context Relevancy — RAGAs metric: fraction of retrieved context that is actually relevant (precision-like).
• Mistral-7B — open-weight 7-billion-parameter LLM, runnable locally.
• bge-small-en-v1.5 — small English embedding model from BAAI, used here for local embeddings.
• GDPR — EU data-protection regulation; the compliance driver behind this work.
• PHI / sensitive data — protected health information; the data the gate is protecting.
• Break-glass access — emergency override allowing access outside normal rules, with heavy auditing.
• OPA / Casbin — policy engines that externalize access-control decisions from app code.