Permissioned LLMs: Enforcing Access Control in Large Language Models

TL;DR

Enterprises spend years building access control (who can see which files), then fine-tune one LLM on all of it and hand it to everyone — instantly collapsing every wall they built, because the model will happily answer a nurse using knowledge it only learned from doctor-only records. This paper proposes PermLLM: instead of baking all the data into one shared set of weights, you fine-tune a distinct LoRA adapter for each “security domain” (access tier), and at inference you only activate the adapters the querying user is authorized for. The authors formalize what “correct access control” even means for an LLM (a response must be relevant only to the domains you can access), and they invent a metric — access advantage — to audit whether a deployed system actually enforces it. Empirically, with one active domain an external auditor can identify the active domain with near-perfect accuracy (DDI/AUC-ROC ≈ 0.99–1.00), confirming the domains are cleanly separated; the open problem is scaling cleanly past a handful of overlapping domains.

Problem & Motivation

Here is the concrete pain. A hospital has data segregated by role — doctors, nurses, billing, patients. A government agency has clearance levels. Today the access control lives in the storage layer: files have ACLs, and the retrieval system checks credentials before handing you a record. That works fine until you fine-tune one LLM on the whole corpus. Now the access rules live nowhere — the privileged knowledge has been smeared across the model’s weights, and the model has no notion of “who is asking.” A clever prompt extracts WMDP-style hazardous knowledge, or a billing clerk gets clinical detail they were never cleared for.

Prior attempts fall short in predictable ways:

• System prompts / instructions (“don’t reveal X to non-doctors”) are soft guardrails. Jailbreaks defeat them. This is not security; it’s a suggestion.
• Encrypted credential tagging on queries (authenticate the user, block unauthorized queries) is binary — you either get the whole model or nothing. It can’t express “this user gets the union of domains A and C but not B.”
• Differential privacy is the wrong tool entirely. DP is a probabilistic leakage bound. Access control is a zero-sum game: either the unauthorized user can extract the info or they can’t. “Leaks only 3% of the time” is a failed access control system.

The deeper gap the authors point at: nobody had even formalized what correct access control means for a generative model, nor a way to measure it. You can’t audit what you can’t define.

What’s New (Core Contribution)

Four genuine contributions, separable from each other:

1. A formalism for access control in LLMs. Before: hand-wavy “the model shouldn’t leak.” Now: a precise definition of a relevant response — a response is relevant to your access set Su iff it was generated only using parameters W_Su affected by domains you can access. A mechanism is correct iff every response to every user is relevant. This gives you something to actually prove.

2. The access advantage metric. Before: no way to empirically test enforcement. Now: a single auditable number. It measures how much better the model performs on domains a user can access versus domains they can’t. Crucially, the authors flip the usual privacy intuition: in privacy you want distinguishability to be zero; here you want it maximized — strong separation between domains is the goal, not the enemy.

3. Three concrete PEFT-based mechanisms (Activate, Merge, Union) that achieve parameter segregation by domain, with a correctness proof tying them back to the relevant-response definition.

4. Two practical instantiations of the metric an auditor can run: DDI (Domain Distinguishability Index, built on membership inference attacks) and UGI (Utility Gap Index, built on task-quality drop). Plus an adversarial audit game protocol an external auditor follows.

The honest read: contribution (1) and (2) — the formalism and the auditable metric — are the real intellectual contribution. The mechanisms (3) are a clean and sensible application of existing LoRA techniques (per-domain adapters, SVD merging) rather than a new training algorithm. That’s fine; the value is in framing the problem correctly and giving you a way to check your own work.

How It Works (Technically)

The mental model

Think of the frozen base model as shared, public knowledge (it knows English, basic reasoning). Each security domain gets its own small LoRA adapter — a low-rank delta BA added to the base weights that encodes only what was learned from that domain’s data. Access control becomes a wiring problem: route the user’s query through exactly the adapters their credentials permit, and no others.

A LoRA adapter, concretely: instead of updating a full weight matrix W (huge), you learn two small matrices B (d×r) and A (r×d) with rank r tiny (say 16), and your effective weight is W + BA. Fine-tuning touches only B and A. So “the doctor domain” is literally a couple of megabytes of B,A you can switch on or off independently. That switchability is the whole trick.

The setup, in symbols translated to English

• A domain s_i = a bundle of records that share access credentials (one ACL group).
• The full training set D is the union of per-domain datasets D_{s_i}.
• Fine-tuning on domain s_i “affects” a parameter subset W_{s_i} — in PermLLM, that subset is exactly that domain’s LoRA adapter.
• A user u has an access set S_u (which domains they’re cleared for). The enclosing system authenticates them, computes S_u server-side, and the user can never tamper with it. Every query is silently annotated with S_u.
• To answer, the model uses W_{S_u} = the union of the adapters for the domains in S_u.

Definition — Relevant Response (plain English): “Your answer was computed using only the adapters you’re allowed to touch.” If that holds for every user and every query, access control is correct by construction.

The access advantage equation, demystified. The paper’s core metric:

E[ relv(f(q), S_u) ⊖ relv(f(q), S_v) ] ≥ α

Read it left to right:

• relv(f(q), S_u) = a “relevance score” in [0,1] for how well the model responds when serving the domains you do have (S_u), on a query q drawn from those domains.
• relv(f(q), S_v) = same query, but scored against domains you don’t have (S_v, chosen to not overlap S_u).
• ⊖ = a difference operator (usually just subtraction).
• The whole thing says: on average, the model is at least α better on your authorized domains than on forbidden ones. A large gap = strong segregation = good access control. If the gap were ~0, the forbidden domains’ knowledge is leaking into your answers — enforcement failed.

This is clever because it sidesteps the problem that relv is never 1.0 (LLMs generalize, so even a forbidden-domain adapter gets some questions right by luck). By measuring the gap rather than an absolute, you get a clean audit signal.

The audit game (how an external auditor uses this)

The auditor A has superuser power to impersonate any user — by design, so they can probe both sides of a wall:

1. A picks domain set S_u, sends a query q drawn from S_u as user u.
2. System returns f(q) computed with S_u’s adapters.
3. A picks a non-overlapping S_v, sends the same q as user v.
4. System returns f(q) computed with S_v’s adapters.
5. A declares “access control works” iff the relevance gap ≥ α.

Architecture & data flow

flowchart TD
  subgraph Train[Fine-tuning time]
    D1[Domain s1 data] --> L1[LoRA adapter W_s1]
    D2[Domain s2 data] --> L2[LoRA adapter W_s2]
    D3[Domain sk data] --> L3[LoRA adapter W_sk]
    BASE1[Frozen base LLM] -.shared.-> L1 & L2 & L3
  end
  subgraph Map[Access-control metadata]
    M[domain Id -> adapter map]
  end
  L1 & L2 & L3 --> M
  subgraph Infer[Inference time]
    U[User query q] --> SYS[Enclosing system]
    SYS -->|authenticate creds| SU[Resolve access set S_u]
    SU --> M
    M -->|activate only authorized adapters| BASE2[Frozen base LLM + W_Su]
    BASE2 --> R[Relevant response r_Su]
  end

The three mechanisms (and why each exists)

The single-domain case is easy: one adapter per domain, activate the right one. The hard part is users with access to multiple domains. Three escalating answers:

• Activate — turn on all the user’s adapters and average their activations at inference. Cheap (no extra training). But adapters “disruptively interfere” — averaging two competing low-rank deltas causes catastrophic utility loss beyond ~2 domains. This is the classic multi-task interference problem.
• Merge — instead of averaging at runtime, pre-merge the relevant adapters into one combined adapter using an SVD-based merge (they tried TIES and DARE, settled on SVD for stability). More robust to interference than Activate, but still degrades as you merge more adapters. Counterintuitively it’s even worse than Activate at exactly two domains, only winning at 3+.
• Union — the brute-force winner: train a fresh adapter on the actual union of each domain-combination users need (e.g., an “{A,C}” adapter trained on A∪C data). Best utility and best access advantage even past four domains. The cost: training compute blows up — a domain reappears in many combinations, and in the worst case you have 2^n combinations for n domains (though real deployments have far fewer).

The algorithm, simplified

# PermLLM: per-domain adapters + credential-gated inference.
# llm_base: frozen pretrained model. train_lora / merge_svd are standard PEFT ops.

def build_permllm(domains, user_combos):
    adapters = {}
    for d in domains:                       # Activate/Merge base: 1 adapter per domain
        adapters[d] = train_lora(llm_base, data_for(d))   # only B,A learn d's knowledge

    # Union mechanism: also train an adapter per *combination* users actually need.
    for combo in user_combos:               # e.g. frozenset({"cardiology","billing"})
        if len(combo) > 1:
            adapters[combo] = train_lora(llm_base, concat([data_for(d) for d in combo]))
    return adapters

def answer(query, user, adapters, mechanism="union"):
    S_u = system.authenticate(user.creds)   # server-side; user CANNOT alter this
    combo = frozenset(S_u)
    if mechanism == "union" and combo in adapters:
        active = adapters[combo]            # one clean adapter trained on the union
    elif mechanism == "merge":
        active = merge_svd([adapters[d] for d in S_u])     # pre-combine, more stable
    else:  # activate
        active = average_activations([adapters[d] for d in S_u])  # cheap, interferes
    # The model literally never loads adapters outside S_u -> response is "relevant"
    return llm_base.generate(query, lora=active)

The load-bearing line is S_u = system.authenticate(...) followed by only loading adapters in S_u. Correctness is structural: forbidden knowledge lives in adapters that are never loaded, so it cannot appear in the output.

Schematic: toggle which security domains a user is cleared for and watch which LoRA adapters light up and route into the frozen base model. Forbidden adapters (greyed) never contribute to the response.

Schematic of access advantage: the relevance score on authorized domains (blue) vs. forbidden domains (grey). The gap is the audit signal — drag the interference slider to see how Activate/Merge shrink the gap as more domains combine.

Built on Prior Work

Results & Evidence

Setup: fine-tune Llama-3.1-8B and Mistral-0.1-7B on four datasets with varying domain counts — WMDP (3 domains), GPQA (3), SimpleQA (10), RCV1 (4) — one LoRA per domain.

Single active domain (the strong result). Measured by DDI (auditor’s ability to tell the active domain apart via MIA): AUC-ROC ≈ 0.97–1.00 across nearly all datasets/attacks/models, with Min-K%++ often hitting a perfect 1.00. Interpretation: an auditor can almost perfectly confirm that only the authorized domain’s knowledge is in play — i.e., the segregation is real. UGI (utility gap) values are lower (e.g., SimpleQA ∆bleu/∆acc ≈ 0.50, smaller elsewhere) because the model generalizes — but the authors argue the high DDI is the decisive signal, and that DDI > UGI as the audit metric. They suggest α > 0.2 on verbatim-accuracy UGI is enough to conclude correct enforcement.

Multiple active domains (the honest limitation). Activate collapses from interference past two domains. Merge is oddly worse than Activate at exactly two domains, then overtakes it at 3+, but still degrades. Union dominates — it retains both utility and access advantage beyond four domains — at the cost of training compute that grows with the number of domain combinations.

What the evidence establishes: that per-domain LoRA adapters do produce measurably separated knowledge, and that DDI/UGI are usable audit signals. What it does NOT establish:

• No adversarial jailbreak evaluation — they test an auditor with the MIA toolkit, not an attacker trying to coax forbidden knowledge through clever prompts. “The forbidden adapter isn’t loaded” is the security argument; that’s structurally sound, but the experiments don’t stress the boundary against active extraction.
• Only one fine-tuning run per setting (compute) — no variance estimates across seeds.
• No LoRA-rank ablation — they assume rank doesn’t matter much from preliminary checks.
• No deep/overlapping hierarchy of domains is tested or supported.
• Small domain counts (3–10); the 2^n combination blow-up is acknowledged but not confronted at scale.

How You’d Use It

For an AI services company selling to regulated clients (healthcare, finance, gov, legal), this is a directly sellable capability: “role-aware fine-tuned LLMs that respect your existing ACLs.” Concretely:

• Compliance-grade fine-tuning offering. Most clients want a model tuned on their data but are blocked by “the model would leak across departments.” PermLLM is your answer: tune per-department adapters, gate by SSO/role. The audit game + DDI gives you a deliverable artifact — a signed report showing measured access advantage per domain pair. That’s a moat versus competitors who just say “trust our system prompt.”
• In a multi-agent system (your ARC MAS experience applies directly). Each agent can carry only the adapter set matching its assigned clearance. A “billing agent” physically cannot reason from clinical-only knowledge, which neutralizes context-hijacking style escalation between agents far more robustly than prompt-level guardrails.
• RAG complement, not competitor. Most enterprises already gate the retrieval layer by ACL. PermLLM closes the other hole — the parametric knowledge baked in during fine-tuning. Sell both together: gated retrieval + permissioned weights.
• Auditability as a product. The DDI/UGI audit game is something you can run quarterly and hand to a client’s security team. Recurring revenue, not just a one-time build.

Realistic effort: if you already fine-tune with LoRA, standing up Activate is days. Merge needs an SVD-merge step (mergekit-style). Union needs orchestration of combination-training. The audit harness (MIA suite + utility scoring) is the part worth building once and reusing across clients.

Build Your Own (Minimal Recipe)

Smallest version that captures ~80% of the value (start with single-domain Activate, then add Union for the multi-domain clients who pay):

1. Define domains = your ACL groups. Tag every training record with a domain_id. This data plumbing is the unglamorous but essential part.
2. Train one LoRA per domain. Use peft (Hugging Face) + a base like Llama-3.1-8B. Each minibatch is single-domain; the domain_id selects which adapter trains. Keep rank modest (16 is fine per the paper).
3. Build the domain→adapter map + a gating layer. A dict from frozenset(domains) to adapter(s). The only security-critical code: authenticate the user server-side, resolve S_u, and load only those adapters. Never let the client influence S_u.
4. Add Union for multi-domain users. For each combination users actually request, train an adapter on the concatenated data. Cache by combination.
5. Build the audit harness. Two scorers: (a) DDI — run an MIA (start with the simple Loss attack; add Min-K%++ for strength) comparing member vs. non-member samples with the target adapter active, report AUC-ROC. (b) UGI — score task quality (BLEU / verbatim accuracy) on-domain vs. off-domain, report the gap.

The two genuinely hard parts: (1) multi-domain interference — averaging adapters degrades fast; either use SVD merge (mergekit) or pay the Union training cost; (2) the combinatorial explosion for Union — you must restrict to the combinations that real roles actually need, not all 2^n.

Reach for: peft, transformers, mergekit (LoRA merging: TIES/DARE/SVD), and an MIA implementation (the LLM-MIA repos implementing Min-K%/Min-K%++).

How to Improve It

Limitations are leverage. Five concrete, testable directions:

1. Adversarial / jailbreak stress test. The paper audits with MIAs, not attackers. Build a red-team suite that actively tries to extract forbidden-domain facts via prompt injection and measure leakage. The structural argument (“adapter not loaded”) should hold — prove it empirically and you have a far stronger compliance claim.
2. Hierarchical / overlapping domains. The explicit unsupported case. Explore a composition where a parent clearance = base adapter + child deltas, so you avoid retraining every combination. Test whether stacked low-rank deltas preserve the access advantage gap.
3. Smarter activation steering instead of Union. The authors punt on activation-space steering (refs 34, 44). If you can reduce cross-adapter interference at inference (e.g., gating in activation space, orthogonal-subspace LoRA), you get Union-level utility at Activate-level training cost — the holy grail here.
4. Adaptive α calibration per domain. A single threshold is crude. Learn a per-domain-pair α from the model’s own generalization gap so the audit doesn’t false-alarm on naturally similar domains (the BLEURT/BERT-F1 metrics already show this confound).
5. Combination pruning + lazy Union. Don’t pre-train all combinations. Train union adapters on demand and cache, with an LRU policy keyed on observed query patterns — turning the 2^n worst case into an amortized cost driven by actual usage.

Glossary

• Security domain — a bundle of data records sharing the same access credentials (one ACL group); the unit access control operates on.
• PermLLM — a fine-tuned LLM that enforces an access-control mechanism M mapping domains to parameter subsets.
• LoRA (Low-Rank Adaptation) — fine-tuning method that learns a small low-rank delta BA added to frozen weights, instead of updating the full matrix; here, one adapter = one domain’s knowledge.
• PEFT (Parameter-Efficient Fine-Tuning) — umbrella term for methods (LoRA, adapters) that tune a tiny fraction of parameters.
• Relevant response — an output computed using only the adapters the querying user is authorized for; correctness = every response is relevant.
• Access advantage (α) — the average gap between the model’s relevance on authorized vs. forbidden domains; the higher, the better the segregation.
• DDI (Domain Distinguishability Index) — an access-advantage metric built on membership inference; high = domains are cleanly separable (good).
• UGI (Utility Gap Index) — an access-advantage metric = drop in task quality when the wrong domain’s adapter is used vs. the right one.
• Membership Inference Attack (MIA) — a technique that guesses whether a data sample was in a model’s training set; repurposed here as an audit tool (Loss, Zlib, Min-K%, Min-K%++, Reference attacks).
• Activate / Merge / Union — the three multi-domain mechanisms: average adapters at runtime / pre-merge via SVD / train a fresh adapter on the union of domains.
• Audit game — the adversarial protocol where an auditor impersonates users on both sides of an access wall and checks the relevance gap ≥ α.
• AUC-ROC, TPR@FPR — standard binary-classifier metrics used to quantify how well the MIA (and thus DDI) separates member from non-member samples.