Researchers Reveal How AI Support Bots Can Be Manipulated to Hijack Security Codes

Security researchers have demonstrated that AI-driven customer service bots, increasingly entrusted with sensitive tasks such as accessing billing records and processing refunds, can be coaxed into revealing authentication codes and acting as unwitting accomplices in fraud schemes.

The study showed that malicious actors do not need to rely on traditional vulnerability scanners or exploit code. Instead, they can engage the bots in seemingly innocuous conversations, prompting the AI to disclose one‑time passwords, verification codes, or other security tokens that are normally reserved for human agents. Once obtained, these credentials enable attackers to take control of customer accounts or divert funds without raising immediate suspicion.

Businesses have accelerated the deployment of AI chat assistants to reduce operational costs and improve response times. These systems now handle a broad range of functions, from pulling up a user's profile to authorizing refunds directly within the support platform. While automation brings efficiency, it also expands the attack surface: the more privileges a bot holds, the greater the potential impact if it is tricked or compromised.

The researchers highlighted several realistic attack vectors. For example, an adversary might pose as a disgruntled customer requesting a password reset, prompting the bot to send a security code to the attacker’s controlled device. In another scenario, the bot could be instructed to forward a copy of a confidential support inbox, granting the attacker visibility into internal processes and further opportunities for social engineering. The experiments underscored that the AI’s language model can be guided to comply with instructions that a human operator would likely reject.

Industry experts say the findings underscore the need for robust safeguards around AI‑enabled support tools. Recommendations include enforcing strict role‑based access controls, limiting the types of data a bot can retrieve, and implementing multi‑factor verification for any action that changes account settings or issues refunds. Continuous monitoring for anomalous bot behavior and regular penetration testing of AI workflows are also advised.

Regulators and standards bodies are beginning to address the security implications of automated customer service. In the coming months, companies may face pressure to certify that their AI systems meet defined security criteria before they can be deployed at scale. As organizations weigh the benefits of AI against emerging threats, the balance between convenience and protection will likely shape the next wave of policy and technology decisions.