Publications

2025

1. NAACL 2025

   Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In

   Itay Nakash, George Kour, Guy Uziel, and 1 more author

   In Findings of the Association for Computational Linguistics: NAACL 2025, Apr 2025

   Abs

   Following the advancement of large language models (LLMs), the development of LLM-based autonomous agents has become prevalent.As a result, the need to understand the security vulnerabilities of these agents has become a critical task. We examine how ReAct agents can be exploited using a straightforward yet effective method we refer to as the foot-in-the-door attack.Our experiments show that indirect prompt injection attacks, prompted by harmless and unrelated requests (such as basic calculations) can significantly increase the likelihood of the agent performing subsequent malicious actions.Our results show that once a ReAct agent‘s thought includes a specific tool or action, the likelihood of executing this tool in the subsequent steps increases significantly, as the agent seldom re-evaluates its actions. Consequently, even random, harmless requests can establish a ‘foot-in-the-door’, allowing an attacker to embed malicious instructions into the agent‘s thought process, making it more susceptible to harmful directives.To mitigate this vulnerability, we propose implementing a simple reflection mechanism that prompts the agent to reassess the safety of its actions during execution, which can help reduce the success of such attacks.

2. COLM 2025

   AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptation

   Itay Nakash, Nitay Calderon, Eyal Ben David, and 2 more authors

   Apr 2025

   Abs Bib

   Large Language Models (LLMs) have shown impressive versatility as general purpose models. However, their broad applicability comes at a high-cost computational overhead, particularly in auto-regressive decoding where each step requires a forward pass. In domain-specific settings, general-purpose capabilities are unnecessary and can be exchanged for efficiency. In this work, we take a novel perspective on domain adaptation, reducing latency and computational costs by adapting the vocabulary to focused domains of interest. We introduce AdaptiVocab, an end-to-end approach for vocabulary adaptation, designed to enhance LLM efficiency in low-resource domains. AdaptiVocab can be applied to any tokenizer and architecture, modifying the vocabulary by replacing tokens with domain-specific n-gram-based tokens, thereby reducing the number of tokens required for both input processing and output generation. AdaptiVocab initializes new n-token embeddings using an exponentially weighted combination of existing embeddings and employs a lightweight fine-tuning phase that can be efficiently performed on a single GPU. We evaluate two 7B LLMs across three niche domains, assessing efficiency, generation quality, and end-task performance. Our results show that AdaptiVocab reduces token usage by over 25% without compromising performance

   @misc{nakash2025adaptivocabenhancingllmefficiency,
     title = {AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptation},
     author = {Nakash, Itay and Calderon, Nitay and David, Eyal Ben and Hoffer, Elad and Reichart, Roi},
     year = {2025},
     eprint = {2503.19693},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2503.19693}
   }

3. ACL 2025

   Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models

   George Kour, Itay Nakash, Michal Shmueli-Scheuer, and 1 more author

   In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), Jul 2025

   Abs

   As Large Language Models (LLMs) become deeply integrated into human life and increasingly influence decision-making, it’s crucial to evaluate whether and to what extent they exhibit subjective preferences, opinions, and beliefs. These tendencies may stem from biases within the models, which may shape their behavior, influence the advice and recommendations they offer to users, and potentially reinforce certain viewpoints. This paper presents the Preference, Opinion, and Belief survey (POBs), a benchmark developed to assess LLMs’ subjective inclinations across societal, cultural, ethical, and personal domains. We applied our benchmark to evaluate leading open- and closed-source LLMs, measuring desired properties such as reliability, neutrality, and consistency. In addition, we investigated the effect of increasing the test-time compute, through reasoning and self-reflection mechanisms, on those metrics. While effective in other tasks, our results show that these mechanisms offer only limited gains in our domain. Furthermore, we reveal that newer model versions are becoming less consistent and more biased toward specific viewpoints, highlighting a blind spot and a concerning trend.POBS: https://ibm.github.io/POBS

4. Under Review

   Effective Red-Teaming of Policy-Adherent Agents

   Itay Nakash, George Kour, Koren Lazar, and 3 more authors

   Jul 2025

   Bib

   @misc{nakash2025effectiveredteaming,
     title = {Effective Red-Teaming of Policy-Adherent Agents},
     author = {Nakash, Itay and Kour, George and Lazar, Koren and Vetzler, Matan and Uziel, Guy and Anaby-Tavor, Ateret},
     year = {2025},
     date = {2025-06-20},
     eprint = {2506.09600},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     url = {https://arxiv.org/abs/2506.09600}
   }