Researchers claim breakthrough in fight against AI’s frustrating security hole
1 min read
Summary
CaMeL (CApabilities for MachinE Learning) is a new security feature created by Google DeepMind to protect against prompt injection attacks.
Prompt injection is a problem for AI developers that allows for a breach in security, enabling an AI model to ignore its intended purpose and behave in a way that is unsafe.
Existing solutions have focused on the AI models themselves, with developers tuning them to identify malicious input.
CaMeL treats language models as untrusted components, shifting the focus to establishing clear boundaries between user interactions and any potentially unsafe material.
By adopting established security principles such as Control Flow Integrity, Access Control, and Information Flow Control, the framework adapts decades of security engineering wisdom to the challenges of LLMs, according to independent AI researcher Simon Willison’s analysis of the new technique.
CaMeL is the first credible prompt injection mitigation attempt not to rely on additional AI resources, according to Willison.
Rather, it embraces tried-and-proven concepts from security engineering, such as capabilities and data flow analysis.
