Prompt injection exploits the model instruction channel by inserting adversarial content that competes with or overrides trusted directives. Attackers embed malicious instructions in user prompts, retrieved documents, tool outputs, or web content to force unsafe responses and actions. The risk grows in agent systems where model output can trigger tools and external side effects.
Separate trusted control instructions from untrusted content, and enforce policy checks before any sensitive action. Constrain tool usage with allowlists and context-aware guards, and apply output moderation for actionability and policy compliance. Include adversarial prompt testing in CI to validate defenses against known injection patterns.
The same feature written the risky way, then with the fix applied.
How this risk has ranked in each OWASP release.
2024Latest
#01llm01-2024 Prompt Injection
