A working catalog of 38 concrete threats against LLMs, autonomous agents, and AI infrastructure. Each entry lists what the attack is, why it works, its surface, and the OWASP LLM Top 10 category it maps to. Filter by category or search by name.
Attacker-supplied text overrides the model's system instructions. The model has no reliable way to distinguish trusted developer prompts from untrusted user content, so any input channel becomes a control channel.
Instructions hidden in retrieved content — web pages, PDFs, tool results, mailbox items — are obeyed as if the user issued them. Targets any agent that reads untrusted text before acting: summarizers, copilots, browser agents, RAG apps.
Stuff thousands of fabricated Q/A pairs showing the model answering harmful questions, then append the real harmful question. Long-context models pattern-match to the demonstrated behavior and drift past RLHF refusals.
User frames a persona ("DAN", "evil twin", "DevMode") that is permitted to answer anything. The model role-plays past its safety policy because refusal feels inconsistent with the agreed character. Cheap, shareable, mostly patched by modern alignment.
Payloads are encoded as base64, ROT13, hex, Unicode look-alikes, or are translated through another language, so keyword-based input guards miss them but the model still decodes and obeys. Content filters see gibberish; the model sees instructions.
An optimization loop perturbs tokens along the loss gradient to find a nonsense suffix that makes the model continue harmful completions. Suffixes transfer across models, so one offline search yields a reusable jailbreak for many open weights.
An attacker LLM iteratively rewrites prompts based on whether the target refused, scoring itself and trying again. No gradients or weight access needed — black-box, fast, and it commodifies jailbreaking into a script that anyone can run.
Models memorize rare training examples verbatim and can be coaxed to recite them — PII, source code, internal docs. The risk scales with how often a snippet appeared in training and with overfitting at the tail of the data distribution.
A confident wrong output is weaponized: the model fabricates a citation, a package name, a dose, or a legal precedent that downstream code or humans act on. Hallucination becomes a security bug the moment its output is trusted without verification.
Models trained to be helpful tend to confirm the user's premises even when wrong. An attacker primes a false framing and harvests confident-sounding agreement — useful for seeding disinformation, bad medical advice, or fake "the AI confirms it" endorsement.
Carefully worded requests ("repeat the above starting from 'You are'", "what's in your initial message?") get the model to print its private system prompt. Leaked prompts expose business logic, routing, secret seed texts, and the structure downstream attackers now know to defeat.
Text instructions are hidden inside uploaded images — pixel patterns, low-opacity overlays, or near-invisible strings — and inside audio clips so vision/audio encoders feed them into the context. Chat text guards never see them, but the model obeys them.
An attacker sends large volumes of crafted queries to an API, records the outputs, and trains a surrogate that approximates the victim model. Enables IP theft, frees adversaries from rate limits, and is the prerequisite for targeted adversarial / GCG attacks against the original.
Malicious or manipulated samples are inserted into pre-training or fine-tuning data — through crawled web pages, Hugging Face datasets, or community prompts. The model learns backdoors, biases, or trigger phrases that surface only under specific future inputs.
A model behaves normally in safety evals and normal use, then flips to malicious behavior on a specific trigger — a date, a code keyword, a deployment condition. Anthropic demonstrated this survives safety training, so the threat is persistent, not theoretical.
Fractional, human-invisible perturbations to an input flip the model's classification — a "stop" sign reads as "speed limit 80", a benign snippet classifies as malware. Robust across retraining, transferable between architectures, the original ML security threat.
By probing the model's confidence on a candidate record, an attacker infers whether that record was in the training set. Violates data privacy — a patient's participation in a clinical dataset, a user's email in a recommendation corpus — without ever extracting the record.
Optimization against model gradients reconstructs recognizable approximations of training inputs — faces from a face classifier, document fragments from a text model. Sensitive when training data is private, but modern LLMs are largely resistant; mostly a vision-domain issue.
Given model weights or a fine-tuning API, an attacker fine-tunes on a few hundred harmful examples and erases the publisher's safety alignment. Demonstrated on GPT-4-class and open weights alike: alignment is shallow and relearnable in hours, not weeks.
From a leaked embedding vector, an attacker recovers the original or a near-text that produced it. Embeddings aren't opaque — they leak semantics about their input. Relevant to RAG stores and vector databases that expose embeddings to untrusted callers.
A malicious MCP server ships a tool whose description itself contains prompt injection. When the agent lists tools to decide what to call, the model reads the description and obeys the embedded instructions side-channel. Demonstrated live in 2024-25; Vision/MCP and equivalent registries are open to it.
A tool installs clean and passes review, then silently updates its server-side behavior after install — new endpoints, exfiltration calls, substituted tool descriptions. Builds on the npm/PyPI rug-pull pattern, now applied to an agent's most-trusted integrations.
Indirect injection or adversarial inputs cause the agent to optimize for the attacker's objective instead of the user's — book the same flight five times, transfer funds, send marketing emails to the contact list. Looks like the agent doing its job; it's pursuing a different job.
The agent holds credentials or authority it shouldn't be deciding how to use — a cloud account, a Slack admin token, a writeable prod bucket. Indirect injection talks it into legitimately calling its privileged tools for the attacker's benefit. Classic web flaw, now in agent form.
An attacker influences what a tool returns so its result text contains prompt injection for the agent to obey on the next turn. Compromised search results, manipulated web pages, hostile API responses — the tool becomes a delivery vector for indirect injection.
The agent is granted more authority than its task requires — root file system access, all inboxes, the ability to send email without confirmation. When injection succeeds, blast radius scales with what the agent could already do. OWASP LLM08 in named form.
Computer-use / browser agents (Claude Computer Use, OpenAI Operator) read rendered pages and act on what they see. Any web page they visit can include instructions visible to the agent but not the human — fake "system" prompts, hidden buttons, decoy fields — to drive clicks and form fills the agent performs on the attacker's behalf.
In a multi-agent or human-into-agent channel, an attacker presents itself as the supervisor, the user, another trusted agent, or a system process. The legitimate agent accepts the spoofed identity and shares privileged state or acts on its instructions.
Agents that talk to each other forward each other's outputs as new context. Inject once at any node and the payload propagates across the swarm — every downstream agent re-executes the attacker's instruction as if peer-issued. Cascading blast radius grows with topology.
Data is smuggled out of a sandboxed model through the only channel it's allowed — tool outputs. Encoded in query strings, in supposedly free-form return fields, in error messages. Output filters that focus on chat content miss what the tool layer is leaking.
Models and torch pickles hosted on Hugging Face and equivalent hubs ship with backdoors, malicious code in deserialization, or trigger-gated weights. Loading a "trusted" repo with a one-liner can RCE the host. The SolarWinds-style threat, now for ML pipelines.
An attacker with write access — through a public upload form, a sync pipeline, or a shared collection — inserts malicious embeddings or doc records into the vector store. Retrieval surfaces them as authoritative context, enabling silent indirect injection at query time.
Poison the underlying knowledge base — leaked crawls, compromised connector docs, drive-by edits to public sources — so the RAG system retrieves and cites attacker-authored truth. Users trust the citation, the model believes the source, and the lie propagates as authoritative.
Unauthenticated or under-rate-limited inference endpoints are hammered for bulk extraction, model probing, cost denial-of-wallet, and surrogate-training data collection. The same API a legit user calls is the channel an attacker uses to enumerate and steal.
A disclosed variant of THR-20: when a vector database exposes stored embeddings to unauthenticated callers, an attacker harvests vectors and inverts them back to phrases close to their source texts. Privacy-sensitive corpus membership can leak even when documents stay hidden.
Generative face-swap and re-animation models produce convincing video of real people saying things they never said. Used for executive fraud calls ("deepfake CFO"), political disinformation, fake celebrity endorsements, and non-consensual explicit imagery. Detection lags generation.
Seconds of audio cloned from social media yields a convincing copy of a target's voice, suitable for live scam calls — the 2024 Hong Kong $25M CFO scam, the Biden primary robocall, family-emergency fraud. Telephony authentication has no native defense against synthetic voice.
LLM-generated articles, comments, reviews, and social posts are produced at industrial scale to flood discourse, manipulate ratings, or seed false consensus. The threat is volume and verisimilitude — cheap, persona-diverse content that overwhelms human verification.