A new paper has proposed a structured threat model, called ATFAA, to address the security risks of Generative AI (GenAI) agents, which plans actions, uses external tools and has persistent memory.
The nine threats it identifies are reasoning path hijacking, objective function corruption and drift, knowledge and memory poisoning, unauthorised action execution, computational resource manipulation, identity spoofing and trust exploitation, human-agent trust manipulation, oversight saturation attacks and governance evasion and obfuscation.
The threats are seen as arising from the AI’s architectural features and the potential for long-term harm is increased by the fact that many of them do not produce immediate effects.
The paper recommends a holistic strategy to security, including determining which tasks are suitable for AI, introducing robust identity verification and authentication and continual monitoring of both user interactions and the agent itself.
These suggestions are especially important in enterprise settings, where GenAI agents can operate across multiple systems and decision layers.