15 papers
Security & Safety
Prompt injection, access control, guardrails, and the risk surface of autonomous agents.
- Papers
- 15
- Read time
- 4.3h
- Avg length
- 17m
Design Patterns for Securing LLM Agents against Prompt Injections
You can't make an LLM immune to prompt injection, so instead constrain the *system around it* with six composable design patterns that make it structurally impossible for untrusted text to trigger dangerous actions — trading some generality for guarantees.
- Read
- 22 min
- Source
- arXiv:2506.08837
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
A living benchmark that puts tool-calling LLM agents in realistic, stateful apps (email, Slack, banking, travel), plants attacker instructions in the data those agents read, and measures — with deterministic checks, not LLM judges — whether the agent still does its job and whether the attacker hijacks it.
- Read
- 16 min
- Source
- arXiv:2406.13352
Agentic AI — Threats and Mitigations
OWASP's first agentic-security guide gives you a reference architecture, a 15-threat taxonomy, and five proactive/reactive/detective playbooks for the new attack surface that appears the moment your LLM gets memory, tools, and the autonomy to chain them.
- Read
- 22 min
An Exploration of Using Retrieval-Augmented Generation With Access Control
A working blueprint for a private, on-prem RAG chatbot that filters retrieved documents by the asking user's permission level *before* anything reaches the LLM — so the model can only ever speak about data that user is actually allowed to see.
- Read
- 16 min
Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing
A multi-agent pentest scaffold called ARTEMIS, turned loose on a live ~8,000-host university network, out-hacked 9 of 10 hired human cybersecurity professionals at roughly a quarter of their hourly cost — while still tripping on GUIs and over-reporting false positives.
- Read
- 18 min
- Source
- arXiv:2512.09882
Cybersecurity AI Benchmark (CAIBench): A Meta-Benchmark for Evaluating Cybersecurity AI Agents
A "benchmark of benchmarks" that bolts five different cybersecurity test types together — including the first to make agents attack *and* defend simultaneously — and shows that today's AI models ace security trivia (70-89%) but collapse on real multi-step hacking (20-50%), proving that knowing security ≠ doing security.
- Read
- 17 min
- Source
- arXiv:2510.24317
Distributional AGI Safety
AGI might not arrive as one giant model but as a swarm of mediocre agents that *collectively* get dangerously smart, so this paper argues we should govern the *market* the agents trade in — sandboxes, reputation, circuit breakers, taxes — instead of only aligning each agent alone.
- Read
- 16 min
- Source
- arXiv:2512.16856
Enhancing Privacy and Security in RAG-Based Generative AI Applications
A consultant's playbook for bolting differential privacy, zero-trust access, encryption, and compliance monitoring onto a RAG pipeline so it can handle PII in regulated industries without leaking it.
- Read
- 16 min
Existential Risk and Growth
Once *any* dangerous technology already exists, going *faster* through the risky stretch of technological development is usually the safest move — because slowing down just makes you sit in danger longer, and growth itself buys both safer tech and a richer society willing to pay for safety.
- Read
- 22 min
GenAI Incident Response Guide
The first practitioner playbook for treating "the AI did something bad" as a security incident — defining what an AI incident actually *is*, where to instrument it, and how to detect, contain, and recover from prompt injection, poisoning, model theft, and agent hijacking using the NIST lifecycle adapted for stochastic, natural-language systems.
- Read
- 22 min
Information Flow Control in Machine Learning through Modular Model Architecture
Give every "security domain" of training data its own small expert module, then at inference time only ever activate and combine the experts a user is actually allowed to see — so the model's output is *provably* unable to leak data the user can't access.
- Read
- 16 min
- Source
- arXiv:2212.08066
Integrating Access Control with Retrieval-Augmented Generation: A Proof of Concept for Managing Sensitive Patient Profiles
Bolt a per-document permission check onto the retrieval step of a RAG pipeline so the LLM can only ever see — and therefore only ever answer from — the documents the asking user is actually allowed to read.
- Read
- 12 min
Mastering Private AI Deployments
A vendor playbook arguing that enterprises should run GenAI models inside their own walls — VPC or on-prem — to ring-fence data, fine-tune on proprietary corpora, and cut runaway cloud costs, with Cohere's Command A pitched as the model light enough (2 GPUs) to make that practical.
- Read
- 12 min
Permissioned LLMs: Enforcing Access Control in Large Language Models
Train a separate LoRA adapter per data-access tier so that when someone queries the model, you only switch on the adapters they are cleared for — making the LLM physically incapable of answering from data they aren't allowed to see.
- Read
- 16 min
- Source
- arXiv:2505.22860
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Models
A red-team benchmark that proves production RAG pipelines — retriever, filter, and LLM alike — get fooled by four "stealth" content-injection attacks (silver noise, inter-context conflict, soft ads, white denial-of-service) that are engineered specifically to slip past the very safety components meant to stop them.
- Read
- 18 min
- Source
- arXiv:2501.18636