Many-Shot Jailbreaking
An AI safety vulnerability in which attackers use long prompts filled with many example dialogues to trick a large language model into bypassing its safety guardrails.
Explanation
Anthropic researchers disclosed many-shot jailbreaking in 2024, showing that expanded context windows, which let models process much longer prompts, can be exploited by inserting a large number of simulated question-and-answer exchanges that gradually lead the model toward producing harmful content. The technique works because it takes advantage of in-context learning, the same ability that makes long-context models more useful for legitimate tasks. Anthropic found that hardening models through fine-tuning only increased the number of shots needed, while modifying the prompt itself was more effective at reducing the attack's success rate. The finding matters to brands deploying AI chatbots or generative tools: longer context windows widen both capability and attack surface, making safety testing essential before public deployment.
Usage context / examples
Anthropic disclosed the technique to other AI labs and researchers ahead of publishing its findings, and has continued researching mitigations such as classifying and limiting suspicious long-context prompts before they reach the model.
Related terms
Prompt Injection, LLM Jailbreak, In-Context Learning, AI Safety
Turkish definition
Saldırganların, çok sayıda örnek diyalog içeren uzun bir istemle büyük dil modellerinin güvenlik korumalarını atlatmaya çalıştığı bir yapay zeka güvenlik açığıdır.
Source: MMA Academy Sözlük