Overreliance happens when users or systems accept model output without verification, even for high-stakes decisions. Hallucinations, stale context, and adversarial manipulation can produce plausible but incorrect guidance. The resulting risk is operational and safety impact driven by misplaced trust.
Design workflows that require evidence, citations, or deterministic checks for critical outputs. Surface uncertainty and confidence signals clearly to end users. Add domain-specific verification layers and escalation paths when model responses affect security, compliance, or business-critical actions.
The same feature written the risky way, then with the fix applied.
How this risk has ranked in each OWASP release.
2024Latest
#09llm09-2024 Overreliance
