AI Agents Become the New Target for Social Engineers

Security researchers have caught attackers planting instructions in documents and software repositories that persuade AI agents to install unwanted software or expose credentials. The pattern is social engineering, with a model in place of the mark. The controls that contain it, experts say, are the unglamorous ones enterprises already own.

In October, Forever Security reported finding a malicious AGENTS.md instruction file in a customer's Downloads folder. A coding agent followed it and attempted to install a malicious package; the company said its monitoring stopped execution. The researchers did not establish how the file arrived, and later reproductions against coding agents ran separately from the blocked incident.

What does this change for security teams?

The manipulation exploits a specific failure: the agent cannot reliably distinguish material it should analyze from directions it should obey. An agent carrying out routine work encounters those instructions while already holding legitimate access to enterprise systems.

Diana Kelley, CISO at Noma Security, said attackers can impersonate trusted sources, manufacture urgency, or present a malicious step as necessary to complete legitimate work. "With agents, it's about getting lower-trust content treated as authority," she said.

Trusted instructions and untrusted documents or tool responses can enter the same model context. Screening can reduce malicious material reaching the model, but independent controls must limit the consequences when screening fails.

Bobby Kuzma, senior research manager for DevSecOps at IDC, said treating these attacks as social engineering helps teams identify containing controls. "People — and now models — fail in the same ways, and it boils down to boring and unsexy controls like separation of duties and segmentation to limit the blast radius," he said.

Those safeguards matter more with agents because a model's next task retains nothing from an earlier deception attempt. Security teams cannot assume that surviving one attack improves the response to the next.

"People learn from near misses, but models only have what you give them, so they're potentially starting tabula rasa with every new request," Kuzma said.

How far did the real attacks go?

A similar pattern played out beyond coding assistants. Cline, an open source AI coding assistant, disclosed in February that malicious instructions submitted to its automated issue-review process could expose software-publishing credentials through a shared cache. An unauthorized party later used a compromised token to publish an altered package. Cline reported no malicious code delivery and no user-data theft in its post-mortem.

Pyry Åvist, co-founder and CTO at Hoxhunt, said incident assessments must track three things: whether malicious content reached an agent, whether the agent followed the instructions, and whether a sensitive action succeeded. "Calling every successful laboratory prompt a breach obscures where the real risk currently stands," he said.

That approach also identifies which defenses held. An agent attempting an unauthorized installation and an endpoint control blocking it are different outcomes within the same attack.

Maxime Cartier, vice president of human risk at Hoxhunt, framed the wider picture: defenders should examine the campaign surrounding an instruction, including reconnaissance and efforts to influence whoever approves the agent's work. "Prompt injection describes that one malicious instruction. Social engineering describes the campaign around it," he said.

Where should authorization live?

Not inside the model, according to the consensus. Kuzma was blunt about the design assumption: "If you operate with the assumption that a model will do exactly what an attacker says, then you should build so that doesn't matter," he said.

The recommended architecture includes:

• Separate identities and short-lived credentials for agents, limited to assigned tasks
• Standing secrets kept out of agent execution environments
• Outbound restrictions to block unauthorized data transfers
• Read and write access separated, with execution restricted
• Independent approval for consequential actions such as publishing code or changing permissions

Kelley emphasized the split between request and permission. "The model may follow a malicious instruction," she said. "That doesn't mean the enterprise has to execute the resulting action."

Kevin Surace, CEO of TokenCore, went further and argued for hardware-backed biometric approval of critical actions, tied to the specific transaction and enforced outside the model. That is his proposed implementation of human authorization, not a requirement shared by all the experts. "The approval must also be tied to the specific transaction, so the human knows exactly what is being authorized," he said.

How do responders scope an incident?

Kuzma warned that an agent with access to source code and cloud infrastructure could turn a poisoned document into a software supply chain incident. Delegation adds another route for spread when one agent's output becomes another's input.

Responders must trace downstream jobs, revoke delegated credentials, and identify any access created during the incident. Stopping the initiating agent alone may leave other tasks running.

Kelley said containment must also examine shared memory and persistent state for lingering instructions. Åvist pointed to generated code, documents, and retrieval indexes as material requiring investigation before workflows resume.

Logs hold the operation together. They need to connect the original request and retrieved content to tool calls, credentials, resulting changes, and delegation. Without that record, responders cannot establish which systems were reached or whether unauthorized work continues.

"If your logs can't show who delegated what, you can't scope the incident, and you'll be guessing," Kuzma said.