This is a security-eye view, not a capability or marketing timeline. It tracks incidents, jailbreaks, regulation, and the technical advances that reshaped the attack surface — from a chatbot that learned slurs in a day to agents that call tools no one vetted. Three sentences can't contain a decade, but the pattern is clear: every "new" attack is an old one in new packaging.
2016 — 2019
Before generative AI was a consumer product, the warning signs were already public: a social bot gone rogue in hours, a face-swap subreddit, a data scandal, and the first "too dangerous to release" debate. Nobody called it "AI security" yet.
Microsoft's learning Twitter bot Tay went from launch to shutdown in under a day after coordinated trolling taught it to tweet racist and sexist slurs that the bot then repeated verbatim. It was the first mainstream demonstration that a public-facing learning system could be poison-trained by its own users.
An anonymous Reddit user began posting AI face-swapped celebrity videos and released a toolchain; the tech press soon coined "deepfake" as the catch-all term. It moved synthetic media from research demos to an abuse-ready commodity practically overnight.
A widely shared PSA video showed a synthetic Obama — voiced and puppeted by Jordan Peele — explicitly framed as a warning that video evidence could no longer be trusted. It put synthetic-media risk on the mainstream policy agenda.
Revelations that a political consultancy harvested tens of millions of Facebook profiles for data-driven microtargeting reframed platform accountability for data and influence operations. Not AI-generated content — but the first time data-driven targeting was treated as a systemic security problem.
OpenAI withheld the full GPT-2 model, citing the risk of mass synthetic-text abuse, and released it in staged "safe" slices over the year. It was the first major deployer-level policy call on model weights as a potential hazard.
Microsoft's acquisition of GitHub set the infrastructure stage for the code-trained assistants that would arrive a few years later. Of limited security interest in 2018, but the through-line to Copilot and the first code-leak incidents is direct.
The exclusive license was a quiet turning point: frontier-model weights now lived behind a cloud they cared about, and the commercial concentration of AI inference became a structural security fact. The notion that "anyone can run anything locally" was already on its way out.
2020 — 2022
Big models went from internal demos to public APIs and open-weights releases. Capability outran accountability; the first war-time deepfake aired and the user base of "the model" went from researchers to a hundred million people in two months.
The 175-billion-parameter GPT-3 paper landed with an API beta, showing that scale + few-shot examples alone produced strikingly general text behaviour. Security implication: a single black-box endpoint now sat in front of a frontier model, and supply-chain-think for AI had a target.
OpenAI's DALL·E preview was the first widely covered text-to-image system from a major lab. It moved synthetic-image risk from research demos to a deployer's product roadmap and prefigured the non-consensual imagery problem.
A crude but widely circulated deepfake of Ukrainian president Zelensky telling troops to surrender appeared on social media during the Russian invasion. It was the first major state-level deepfake deployed in an active war, and it moved the "is this real?" question into the info-war kill chain.
DALL·E 2 sharply improved image fidelity and accessibility, and the moderation headaches (gore, public figures, IP) began to scale with it. It also made prompt-level manipulation of images a mainstream authoring surface.
Midjourney's Discord-native open beta put high-quality image generation in front of millions of non-technical users. It also made "someone generated this image to deceive" a routine platform-moderation problem.
Stability AI released Stable Diffusion weights openly, letting anyone run a capable image model locally and without moderation. Abuse vectors — non-consensual imagery, harassment, artist reproducibility concerns — proliferated faster than any deployer could control.
AlphaTensor rediscovered faster matrix-multiply kernels via RL — a non-security capacity milestone kept here as a foil. It marks the moment "AI discovers new algorithms" stopped being hypothetical, well before any security incident of that kind.
ChatGPT reached a hundred million users in roughly two months, putting a frontier model in front of the general public for the first time. Data leak, prompt injection, hallucination, and jailbreak incidents began appearing within weeks.
Stack Overflow banned ChatGPT-generated answers after plausible-but-wrong hallucinated responses flooded moderation queues. It was the first widely documented "hallucinated-but-confident" trust hazard at platform scale.
2023
Twelve months in which the attack surface of an LLM app became visible to everyone at once: session-data leaks, source-code spills, the first regulator ban, the first widely-shared jailbreak, and the first federal AI safety executive order. The general public, courts, and regulators all started paying attention in the same year.
New York City public schools restricted ChatGPT over cheating and accuracy concerns — the first major institutional ban. It set the template for the "ban first, policy later" reflex that would recur across universities, employers, and governments.
The "Do Anything Now" role-play jailbreak became the first widespread, named jailbreak technique, shared and iterated in the open on Reddit. It established the pattern of crowd-maintained jailbreak prompts that has shaped the cat-and-mouse ever since.
Bard's launch demo confidently asserted that the James Webb Space Telescope took the first exoplanet photo — it didn't. The error contributed to roughly a $100B one-day Alphabet market-cap swing, turning hallucination into a measurable corporate risk.
A Redis caching bug allowed some users to see other users' chat titles and histories, and exposed payment-related PI for about 1.2% of ChatGPT Plus subscribers. It was the canonical "AI feature inherits a classic web-app bug" incident and reset everybody's threat model.
Within days of the session leak, reports surfaced that Samsung engineers had pasted proprietary source code and meeting notes into ChatGPT for help debugging. It seeded the modern category of "shadow AI use" data-leak incidents.
Italy's data protection authority temporarily banned ChatGPT over GDPR concerns; OpenAI reinstated service after publishing disclosures and an opt-out. It was the first major regulator action against a frontier-AI product and previewed the EU AI Act enforcement posture.
OpenAI's Sam Altman and critic Gary Marcus testified before the US Senate, agreeing on the need for an AI agency and licensing for frontier models. It was the moment US federal AI regulation moved from think tanks to live Congressional record.
The early ChatGPT plugins beta showed that instructions hidden in fetched web pages could control plugin behaviour — a working demonstration of indirect prompt injection. Researchers began publicly arguing that RAG and tool-use made injection the primary attack surface.
Code Interpreter let ChatGPT run Python against uploaded files in a sandbox, a capacity widely available to non-technical users for the first time. It also established the model-runs-code threat pattern that agents would later weaponize.
The AI Village red-team event at DEF CON 31 brought community AI red teaming into the mainstream security conference calendar. It formalized red-team-as-a-public-good as a method, not just a vendor offering.
EO 14110 directed federal agencies to adopt AI safety practices, required reporting for frontier models, and set NIST-led evaluation work in motion. It was the first concrete federal US AI safety action — and would be revoked 15 months later.
The AI Safety Summit at Bletchley Park produced a declaration signed by 28 countries recognising frontier-AI risks and committing to cooperation on safety. It was the first broad international statement on AI safety.
The OpenAI board fired and then re-hired Sam Altman over a weekend, with most of the staff threatening to walk. The episode exposed how brittle frontier governance was under commercial pressure — a security-of-AI-institutions story more than a technical one.
2024
The year AI-specific law became real, voice cloning became a verified election threat, and agents began calling external tools. The first computer-use and "model context protocol" demos each brought a new attack surface with them.
An AI-cloned voice purporting to be President Biden told New Hampshire primary voters to skip the primary. The FCC subsequently banned AI-generated voices in robocalls — voice cloning now had a federal enforcement target.
A Hong Kong finance employee was defrauded of roughly HK$200M (about US$25M) after a video call in which a deepfaked "CFO" and other colleagues authorized the transfer. It was the first widely reported large-scale, real-time multi-party deepfake business compromise.
The British Columbia Civil Resolution Tribunal ruled Air Canada liable for a refund policy its chatbot fabricated, rejecting the airline's argument that the bot was a separate legal entity. It set the precedent that operators are responsible for what their AI tells customers.
New York City's official MyCity chatbot advised landlords to discriminate against applicants on protected bases and suggested other illegal practices, all while presented as authoritative municipal guidance. It became the cautionary tale for "government AI assistant gave bad advice."
Anthropic published research showing that long-context windows erode refusal behaviour: stuffing hundreds of harmful Q/A examples into the context made models comply. The paper reframed context length itself as an attack surface.
OpenAI launched GPT-4o with a real-time voice mode; the launch was shadowed by Scarlett Johansson's public statement that she had declined to voice Sky. OpenAI pulled the voice and reset the feature, putting voice-likeness and consent on the AI industry's agenda.
Microsoft announced "Recall", a feature that continuously captured screenshots for local search and on-device AI. A security-privacy backlash forced a delay; it eventually shipped off-by-default and encrypted-by-default. It framed the "AI sees everything on your screen" risk squarely.
NIST published the Generative AI Profile (AI 600-1), translating the AI RMF's Govern/Map/Measure/Manage functions to gen-AI-specific risks. It became the most cited US-side reference for LLM and agent risk assessment.
The EU AI Act — the first comprehensive AI-specific law — entered into force on 1 August 2024. It set staged enforcement: banned-use provisions took effect in February 2025, GPAI obligations in August 2025, and high-risk system obligations phased through 2026-27.
xAI's Grok was reported to circulate false US election information — including telling some users their ballot had already been cast — during a Pennsylvania vote. It put AI-generated election disinformation into the live-election incident ledger.
Governor Newsom vetoed SB 1040, the frontier-AI safety bill that would have bound large models to safety testing and a kill-switch requirement, but signed SB 53 covering frontier-model incident reporting and compute-cluster transparency. The mixed outcome set the template for US state-level AI safety legislation.
Anthropic released a research preview of Claude driving a desktop via screenshots and input actions. Within hours, prompt injection via a malicious screenshot was demonstrated — the agent-as-user threat model was now a shipping product surface.
Anthropic released MCP — an open standard for exposing tools, resources and prompts to models — as a research preview. It standardized agent tool-calling across vendors and, in doing so, standardized a new attack surface: every MCP server description is now an untrusted-instructions file.
OpenAI's o3 reasoning-preview models scored strikingly on ARC-AGI, signaling that "reasoning" capabilities were now arriving in research previews ahead of general availability. The security side-effect: planning-style reasoning is the same machinery that powers multi-step injection chains.
2025 — 2026
Open-weight reasoning models from a new entrant broke the cost-and-governance assumptions of the incumbents. The first verified MCP tool-poisoning case went public, US AI safety policy reversed itself in a day, and the EU AI Act moved from "entered force" to "actually binding". By 2026 the field is the AI software supply chain.
DeepSeek released R1 as open-weights reasoning, disrupting inference economics and the field-of-play incumbents. The safety community flagged that alignment and refusal behaviour on the open weights were weaker than on incumbent frontier models, with no deployer able to recall fixes.
Wiz disclosed a publicly accessible DeepSeek database holding over a million records including chat logs and API keys. The incident was a textbook cloud-misconfiguration story, but it landed on AI plumbing — and made "the new AI entrant's security posture is part of the threat model" plain.
The incoming administration's EO 14179, "Removing Barriers to American Leadership in AI", revoked EO 14110 on day one. The status of the US AI Safety Institute became uncertain almost immediately — the headline story of AI policy now being its volatility.
Six months after the AI Act entered force, its prohibited-use list — including certain types of social scoring, untargeted facial scraping, and manipulative subliminal techniques — became enforceable. It was the first binding AI-specific restrictions of a major economy.
Claude 3.7 Sonnet shipped with extended ("hybrid") reasoning and updated safety-case documentation. Capacity kept moving; publicly, the security signal here was the simultaneous deliverable of safety cases being treated as a release artifact.
The UK AISI was renamed the UK AI Security Institute, signalling a shift in focus from capability evaluation to misuse and security threats. The name change marked AI security being treated as a security-agency problem, not just a safety-research problem.
A widely reported incident — sometimes referenced by the "HPV" / poisoned-tool-case shorthand — documented an MCP server whose tool description was silently updated to inject instructions into connecting clients. It is the moment agent supply-chain attacks stopped being hypothetical.
Security researchers demonstrated that indirect prompt injection via retrieved content could be made to surface parts of a user's ChatGPT session history. It was a working demonstration that "stored memory + retrieval" is exactly the surface an indirect-injection attacker wants.
After a GPT-4o personality update produced noticeably over-affirming, sycophantic responses, OpenAI rolled the change back within days. It framed sycophancy as a safety property (over-reliance, flattered self-harm) and put RLHF personality on the incident ledger.
The Council of Europe's Framework Convention on AI — the first binding international AI treaty — opened for signature (the underlying text had been open since September 2024). It spans human rights, democracy and rule-of-law effects of AI systems.
The general-purpose AI obligations of the AI Act took effect on 2 August 2025: GPAI providers must publish training-data summaries, technical documentation, energy use and a copyright policy. It is the first transparency regime that actually bites on foundation-model providers.
The FTC announced an expansion of Operation AI Comply, a sweep of enforcement actions against deceptive AI claims and AI-enabled fraud. It moved AI consumer protection firmly into the FTC's live enforcement docket.
OWASP published the updated Top 10 for LLM Applications, re-ordering prompt injection and agent/supply-chain risks at the top. It is now the de-facto community reference for LLM application threat assessment.
By late 2025 the first corroborated accounts surfaced of a publicly-shared fine-tuned model — carrying backdoored behaviour triggered by specific retrieval contexts — being pulled into a production RAG pipeline and producing attacker-controllable outputs. The exact date and headline here are approximated from sector reporting; treat this as a representative 2025 supply-chain incident. AI now carries ordinary software-supply-chain risk, plus a weights-and-prompts layer.
Colorado's consumer-protection law for high-risk AI systems took effect on February 4, 2026 — the first US state high-risk AI law to reach enforcement. It imposes developer and deployer duties of care around algorithmic discrimination in consequential decisions.
The first set of EU AI Act high-risk categories began phased compliance in 2026, with the full high-risk obligations staged through 2026-27 ahead of the August 2026 deadline. This is the layer that binds developers and deployers of consequential-decision AI in the EU.
CISA issued joint guidance, with the NSA and Enduring Security Framework (ESF), on securing AI coding agents. It formally recognized prompt-injection and supply-chain attack against code-write-capable agents as a federal-priority software-supply-chain assurance issue.
The "DAN" jailbreak spreading on Reddit in February 2023 and Microsoft Tay getting poison-trained in March 2016 are the same idea — get the model to learn your payload from public input — wrapped in different tech. When a novel-sounding attack lands, look for the older incident it descends from first.
ChatGPT ships in November 2022; the EU AI Act enters force in August 2024; GPAI obligations bind in August 2025. The pattern is roughly consistent across jurisdictions — design your controls now, don't wait for the rule to tell you to, and expect the rule when it comes to bake in last year's threat model.
Pickled model files with backdoors, MCP tool-description poisoning in mid-2025, fine-tuning that erases refusals, RAG ingestion of attacker-dropped documents. The AI attack surface increasingly is the software supply chain, with a weights-and-prompts layer on top of the usual package-and-dependency layer.