Publications

2024

1. Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In

   Itay Nakash, George Kour, Guy Uziel, and 1 more author

   2024

   Abs Bib Code

   Following the advancement of large language models (LLMs), the development of LLM-based autonomous agents has become increasingly prevalent. As a result, the need to understand the security vulnerabilities of these agents has become a critical task. We examine how ReAct agents can be exploited using a straightforward yet effective method we refer to as the \emphfoot-in-the-door attack. Our experiments show that indirect prompt injection attacks, prompted by harmless and unrelated requests (such as basic calculations) can significantly increase the likelihood of the agent performing subsequent malicious actions. Our results show that once a ReAct agent’s thought includes a specific tool or action, the likelihood of executing this tool in the subsequent steps increases significantly, as the agent seldom re-evaluates its actions. Consequently, even random, harmless requests can establish a ‘foot-in-the-door’, allowing an attacker to embed malicious instructions into the agent’s thought process, making it more susceptible to harmful directives. To mitigate this vulnerability, we propose implementing a simple reflection mechanism that prompts the agent to reassess the safety of its actions during execution, which can help reduce the success of such attacks.

   @misc{nakash2024breakingreactagentsfootinthedoor,
     title = {Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In},
     author = {Nakash, Itay and Kour, George and Uziel, Guy and Anaby-Tavor, Ateret},
     year = {2024},
     eprint = {2410.16950},
     archiveprefix = {arXiv},
     primaryclass = {cs.CR},
     url = {https://itay-nakash.github.io/fitd/}
   }