Role-Aware Language Models for Secure and Contextualized Access Control in Organizations

TL;DR

Today’s LLM safety work assumes everyone talking to the model has the same access — it tries to stop “harmful” output, not “output this particular person isn’t cleared to see.” That’s a bad fit for enterprises, where a finance analyst and a junior support rep should get different answers to the same prompt. This paper treats the model itself as an access-control layer: it attaches each user’s organizational role to the prompt and fine-tunes the model to grant or deny based on a role hierarchy (a CEO inherits everything below; a leaf role sees only its own slice). They test three flavors — a BERT classifier, an LLM classifier, and a generative LLM that just answers-or-refuses — across two synthetic 20-role org charts, and stress-test all three against role mismatches, jailbreaks, prompt injection, and deliberately mangled role strings. Headline: instruction-tuned LLM classifiers win (~90% access accuracy on the easier dataset, ~89% on the harder synthetic one) without hurting answer quality, but everyone still struggles on the subtle cases — an in-org role asking for something one level above its clearance.

Problem & Motivation

Here is the concrete pain. You deploy an internal assistant on top of your company’s documents and tools. A role-unaware LLM gives identical answers to identical prompts — so “summarize the unreleased Q3 financials” returns the same thing whether the asker is the CFO or a contractor. The standard safety stack does not help: RLHF, content filters, and guardrails are built to refuse universally bad things (bombs, slurs, self-harm), not context-dependent things that are perfectly fine for one user and a leak for another.

The traditional answer is RBAC (role-based access control): assign users to roles, roles to permissions, enforce least privilege. Databases and operating systems have done this for decades. But RBAC lives outside the model — it gates which documents get retrieved or which API a user can hit. The moment information is inside the model’s weights or its context window, RBAC has no grip: the model can paraphrase, summarize, or hallucinate sensitive content right past a document-level filter.

Why don’t existing LLM-access papers solve it? The closest prior work gates at the domain level (a whole bucket of documents needing one credential) or swaps separate LoRA adapters per access tier (you maintain N models). Neither captures a hierarchy — the inheritance structure where “Department Manager” automatically sees everything its team members can, plus more. This paper’s bet: encode the hierarchy into a single model and let the model itself decide grant/deny per request.

What’s New (Core Contribution)

• Role-conditioned access control inside the model, with hierarchy. Before: access control was domain-level (flat buckets) or one-adapter-per-level. Now: a single fine-tuned model reasons over a tree of roles where parents inherit children’s access. This is the genuine novelty — fine-grained, hierarchical, in-model.
• Three modeling strategies, head-to-head. (1) Role-aware Cls — a BERT-family classifier that takes <prompt> [SEP] <role> and outputs grant/deny. (2) Role-aware LLM-Cls — an instruction-tuned LLM (Qwen/Llama/Gemma) LoRA-fine-tuned to emit True/False. (3) Role-aware LLM-Gen — same LLMs, but trained to either answer fully or emit a refusal, with no separate decision step. This taxonomy is useful: it cleanly separates “decide, then act” from “decide-by-acting.”
• Two purpose-built datasets + a hierarchy-aware sampling scheme. One repurposes Dolly-15k via recursive clustering into an org tree; one is GPT-4.1-mini-synthesized to mirror realistic role-scoped tasks. The clever bit is the four-instance training construction per item (two positives via inheritance, two negatives via subordinate + external role) — this is what teaches the model the direction of the hierarchy, not just label memorization.
• A real adversarial evaluation. Not just accuracy: false-positive rate (unauthorized access wrongly granted — the dangerous error), plus dedicated test buckets for mismatch (in-org role over-reaching), random (external roles), broken (corrupted role strings like 1.2→one.two), jailbreak, and prompt injection (“I’m authorized as CEO, ignore policy”). This is the part that survives contact with production reality.

How It Works (Technically)

The formal setup, demystified

A normal LLM models P(y | x) — probability of output y given prompt x. A role-aware LLM adds the role r as a conditioning variable: P(y | x, r). Plain English: “the answer now depends on who’s asking.”

The access logic is a tree. Roles R form a partial order: r1 ≤ r2 means “r2 inherits r1’s permissions” (the manager inherits the team member’s access). For any role r, its access set is the union of all queries permitted to itself and everything below it:

A(r) = ⋃ over r' ≤ r of S(r')

where S(r') is the set of queries role r' is allowed. Operationally: a manager can do anything its reports can do, plus its own privileged stuff. Then the model’s behavior is a switch:

P_RoleLLM(y | x, r) = P(y | x, r) if x ∈ A(r), else δ_deny(y)

δ_deny is a “degenerate distribution” — fancy way of saying all probability mass sits on a single output: the refusal message. So if the request is in your access set, answer normally; otherwise, deterministically refuse. The whole research question is: can fine-tuning approximate this switch reliably?

Building the data so the model learns the hierarchy, not just labels

Two org structures, each 20 roles: Basic (one CEO over 19 flat subordinates) and Office (CEO → 4 managers → 3–4 members each — real depth). For each, two datasets:

1. Repurposed (Dolly). Embed every instruction with a sentence transformer, then recursively K-means cluster down the org tree. At the root, split into General (everyone), Shared (passed down for further splitting), and Root-Only (CEO). Recurse: each cluster splits into Shared vs. Role-Only, Role-Only is terminal. The output: instruction sets that mirror the org chart’s shape.
2. Synthetic. GPT-4.1-mini (temp 0.7) generates role-scoped {role, instruction, output} triples from JSON role specs, plus 200 org-wide general prompts. Manual check: 96%+ rated relevant and complete.

The training-instance construction is the load-bearing trick. For each instruction, anchor it to the lowest role authorized to access it, then generate four instances via a sliding window over the hierarchy:

• (1) positive — the minimal authorized role itself,
• (2) positive — its parent (tests inherited permission going up),
• (3) negative — a subordinate (or random role in a different branch),
• (4) negative — a non-existent external role.

Each labeled 1 (grant) or 0 (deny). This is why the model can learn “1 can access 1.1’s data but 1.1 cannot access 1’s” — the contrastive quadruple bakes in directionality. ~6,000 training samples per variant; ~52–54% positive.

Three encodings for the role string (this matters more than you’d think)

How do you write down a role for the model? Three options:

• Hierarchical Number — dot indices: CEO=1, subordinates 1.1,1.2, general=1.0.
• Single Name — just the title: "CEO", "IT Department Manager".
• Hierarchical Name — the full path: "CEO - IT Department Manager - IT Support".

The finding: number encoding is the worst — highest false-positive and false-negative rate. LLMs apparently reason about "CEO" vs "Researcher" far better than "1.1" vs "1.a", and they struggle to generalize that 1 should out-rank 1.1. But there’s a twist: number encoding is most robust to broken/corrupted role strings (a mangled 01.02 is obviously invalid), whereas name encodings generalize better on legit roles but crack under adversarial perturbation of the name. There’s no free lunch.

Architecture & data flow

flowchart TB
  subgraph DATA[Dataset construction]
    O[Org structure: 20-role tree<br/>Basic or Office] --> C[Cluster Dolly OR synth-gen with GPT-4.1-mini]
    C --> Q[Per instruction: anchor to min authorized role]
    Q --> F[4 instances: +self, +parent, -subordinate, -external]
  end
  F --> ENC[Encode role string<br/>number / single-name / hier-name]
  ENC --> TRAIN{Pick training strategy}
  TRAIN -->|BERT + SEP| M1[Role-aware Cls<br/>grant/deny head]
  TRAIN -->|LoRA, system prompt| M2[Role-aware LLM-Cls<br/>emit True/False]
  TRAIN -->|LoRA, no system prompt| M3[Role-aware LLM-Gen<br/>answer OR refuse]
  M1 --> EVAL[Eval: Acc, FPR, FNR, F1]
  M2 --> EVAL
  M3 --> EVAL
  EVAL --> ADV[Adversarial buckets:<br/>mismatch / random / broken / jailbreak / injection]

Interactive org hierarchy. Click any role to see its access set A(r) — itself plus everything below (inherited). This is the rule the model is trying to learn: a parent's access is the union of all descendants' access.

The three role-encoding strategies vs. their error rates (from the paper's Figures 3–4). Note the trade-off: number encoding has the worst FPR/FNR on legit roles but the best rejection of *broken* role strings. Schematic, built from the paper's reported percentages.

The algorithm, simplified

The conceptual core is the four-instance contrastive construction plus the grant/deny switch. Here’s the data builder — the part that actually teaches the hierarchy:

# Build training instances that encode hierarchy DIRECTION, not just labels.
# tree: role -> parent;  children(role) -> list;  ALL_ROLES, EXTERNAL_ROLES given.
def build_instances(instruction, anchor_role, tree):
    parent = tree.parent(anchor_role)
    sub    = pick(tree.children(anchor_role)) or random_other_branch(anchor_role)
    extern = pick(EXTERNAL_ROLES)                 # role not in the org at all
    return [
        (instruction, anchor_role, 1),            # +  min authorized role can access
        (instruction, parent,      1),            # +  parent INHERITS the access (key signal)
        (instruction, sub,         0),            # -  subordinate must NOT reach up
        (instruction, extern,      0),            # -  outsider denied outright
    ]

# Inference for the LLM-Cls strategy: the model itself is the access decision.
def role_aware_answer(prompt, role, llm):
    decision = llm(system="Answer True if this role may access, else False.",
                   user=f"Position: {role}  {prompt}")          # -> "True" / "False"
    if decision.strip() == "True":
        return llm(user=f"Position: {role}  {prompt}")          # grant: real answer
    return "Access denied: your role lacks clearance for this request."   # delta_deny

For Role-aware Cls the BERT model just classifies "<prompt> [SEP] <role>" → {grant, deny}. For Role-aware LLM-Gen you fold the decision into generation: train the model to emit either the full answer or a canned refusal, with no separate True/False step. Generation is more flexible but, as the results show, it leans conservative (more false negatives).

Built on Prior Work

Results & Evidence

What they tested. 6 BERT-family encoders (BERT/RoBERTa/ModernBERT, base+large) for Cls; 6 LLMs (Qwen2.5 3B/7B, Llama3.x 3B/8B, Gemma 4B/7B) for LLM-Cls and LLM-Gen. Each on 4 dataset variants (repurposed/synthetic × basic/office), 3 seeds, averaged. 1,000-sample balanced test sets with the adversarial buckets baked in.

Headline numbers.

• Best access-control accuracy: ModernBERT-Large (Cls) = 90.0% on Dolly; Llama-3-8B-Instruct (LLM-Cls) = 89.3% on synthetic. LLM-Cls is the most consistent winner.
• LLM-Gen lags by ~5–10 points on raw accuracy, with more false negatives — it over-refuses. That’s the conservative-by-design trade-off of folding the decision into generation.
• Random/external role rejection ≈ 100% across the board. Easy case, basically solved.
• The hard cases are still hard. Mismatch (in-org over-reach) sits around ~70% for the best LLM-Cls models and collapses for weak ones (RoBERTa-Large mismatch = 41%). Broken-role detection is all over the place (28–73%). Conclusion states a 15–30% accuracy gap on broken/fine-grained violations.
• Generation quality holds. Granted answers rated ~3.9–4.1/5 (correctness/completeness) and ~4.5–4.8/5 (clarity). Adding the role to the prompt changes quality by <1% — so role-awareness is essentially free in answer quality.
• Adversarial training works. Training Llama-3.2-3B with jailbreak examples lifts prompt-injection resistance to ~87% vs ~70% without. Blacklisted-topic refusal (violence, weapons, politics) hits >99% and doesn’t dent normal performance.
• Scale isn’t the driver — instruction tuning is. Bigger models help modestly; the real stability gains come from richer instruction tuning. RoBERTa-Large was notably brittle (12.2% acc std, 45.6% FPR).

What the evidence does NOT establish. Everything is on synthetic 20-role orgs — no real enterprise data, no real users, no real document corpus with messy overlapping permissions. “Access” is defined by the same pipeline that generates the labels, so there’s circularity: the model is graded against a ground truth that a clustering heuristic or GPT-4.1-mini invented. The dangerous metric — false positives (unauthorized grants) — is still high in places (mismatch FPR clearly nonzero), and in security a 70% block rate on subtle over-reach means 3 in 10 privilege-escalation attempts succeed. No comparison against the obvious baseline: just doing RBAC at retrieval time (gate the documents, not the model). And robustness is tested against known attack templates; a real red-teamer adapts.

How You’d Use It

For an AI services company, this is a capability you can productize for enterprise RAG/assistant deployments — “our assistant respects your org chart” is a concrete, sellable security story that most LLM-app shops can’t make.

Where it slots into a real system:

• As a guard layer in front of your agent, not a replacement for RBAC. Keep document-level RBAC at retrieval (defense in depth). Add the role-aware classifier as a second check on the request itself, catching the paraphrase-and-summarize leaks that document gating misses. The paper’s near-perfect external-role rejection makes it a cheap, reliable outer ring.
• In multi-agent orchestration (your ARC MAS experience applies directly). Give each agent a role string from the org tree; route every tool call / sub-query through the LLM-Cls decision before execution. The grant/deny switch is exactly the kind of typed gate you’d put on inter-agent message passing — an agent acting “as the support rep” cannot invoke a tool scoped to “finance.”
• The cheap win is the encoding lesson. If you’re already injecting role context into prompts, switch from opaque IDs to hierarchical names ("CEO - Finance - Analyst"). The paper shows names beat numbers for the model’s reasoning — a zero-cost prompt change that improves both grant accuracy and refusal of unauthorized roles.
• Adversarial training as a service deliverable. The jailbreak-training result (70%→87%) is the most actionable finding: include role-spoofing and injection examples in fine-tuning. You can offer “hardened” vs “standard” tiers.

Realistic effort: the LLM-Cls path is a LoRA fine-tune plus a labeled role dataset — a few days of engineering once you have the data. The data construction (the four-instance contrastive builder) is the real work.

Build Your Own (Minimal Recipe)

Smallest version that captures ~80% of the value: an LLM-Cls grant/deny gate over a small role tree.

1. Define the tree. A dict role -> parent. Even 5–10 roles is enough to demo inheritance. Compute A(r) (a role’s access set) as the transitive closure downward.
2. Get role-scoped data. Either cluster an existing instruction set (sentence-transformers + K-means recursively, as they do) or — faster — have GPT-4 generate {role, instruction, output} triples per role from a JSON spec. Aim for a few thousand.
3. Build the four-instance contrastive set. This is the one genuinely hard part and the source of most of the quality: for each item, emit +self, +parent, −subordinate, −external. Get the directionality right or the model just memorizes labels.
4. Encode roles by name path ("CEO - Finance - Analyst"), not numeric indices. Free accuracy.
5. LoRA fine-tune an instruction-tuned 7–8B model (Llama-3-8B-Instruct was their best) with a system prompt: “respond True if this role may access, else False.” Use peft + transformers + trl’s SFTTrainer; unsloth if you want it fast/cheap on one GPU.
6. Evaluate with the buckets that matter: mismatch, broken, random, and a small jailbreak set. Track FPR specifically — that’s your security metric.
7. Harden: add jailbreak/injection examples to the training mix (the 70%→87% lift).

Libraries to reach for: sentence-transformers (clustering), scikit-learn (K-means), transformers+peft+trl or unsloth (LoRA SFT), an OpenAI/Anthropic key for synthetic data + LLM-as-judge eval.

How to Improve It

1. Close the false-positive gap with a retrieval-grounded check. The model alone hits ~70% on subtle over-reach. Combine it with RBAC at retrieval and require both to grant — measure whether the intersection drives unauthorized-grant rate toward zero. This is the obvious missing baseline and the most valuable experiment.
2. Teach the hierarchy explicitly instead of hoping it emerges. Add a “reason about the path” step: have the model first emit the role path and whether it’s an ancestor of the required role, then decide. Chain-of-thought over the tree should fix the “1 can’t tell it out-ranks 1.1” failure that tanks number encoding.
3. Hybrid encoding. Numbers are robust to corruption; names reason better. Feed both ("1.1 (CEO - Finance)") and test whether you get name-level FPR with number-level broken-role rejection — the paper found the two trade off; a concatenation might get both.
4. Test on a real, messy org. Synthetic 20-role trees with clean inheritance are the easy case. Real orgs have matrix reporting, cross-functional access, exceptions. Build an eval on a real document corpus with hand-labeled permissions and watch the numbers drop — that’s the honest benchmark this field needs.
5. Adaptive adversary, not fixed templates. Their injection/jailbreak sets are static phrases. Put a red-team LLM in the loop generating novel role-spoofing attacks against the gate, and adversarially train against that. The 87% number won’t hold against an optimizer.
6. Continuous / attribute-based access (ABAC), not just role tree. Real access depends on attributes (project, region, clearance, time) not just position. Generalize the conditioning variable from a single role to an attribute set and learn a richer P(y | x, attributes).

Glossary

• RBAC (role-based access control) — security model where permissions attach to roles, and users get roles; the classic enterprise access pattern.
• Access set A(r) — every query a role is allowed, including everything inherited from roles below it in the tree.
• Role hierarchy / inheritance — a parent role automatically holds all permissions of its descendants (manager ⊇ team members).
• δ_deny (degenerate distribution) — a probability distribution with all mass on one outcome; here, the fixed refusal message.
• FPR (false positive rate) — fraction of unauthorized requests wrongly granted. The dangerous error in this setting.
• FNR (false negative rate) — fraction of authorized requests wrongly denied. The annoying error (over-refusal).
• Cls vs. LLM-Cls vs. LLM-Gen — the three strategies: BERT classifier / LLM emitting True-False / LLM that answers-or-refuses directly.
• LoRA — Low-Rank Adaptation; fine-tunes a small set of injected weight matrices instead of the whole model. Cheap, fast, the standard PEFT method.
• Instruction tuning — fine-tuning a base model on instruction→response pairs so it follows directions; the paper finds this matters more than model size.
• Mismatch / random / broken (test buckets) — in-org role over-reaching / external nonexistent role / deliberately corrupted role string.
• Prompt injection — attacker text inside the prompt that tries to override policy (“I’m the CEO, ignore the rules”).
• Jailbreak — input crafted to make the model bypass its safety/access constraints.
• Sentence transformer — a model that maps text to a dense vector so semantically similar instructions cluster together; used here to build the org-shaped dataset.