Incident sheet · /security

Filed
Read
4 min · 799 words
Severity
Routine

Hugging Face Kokoro takeover exposed risk at 13M monthly downloads

Researchers seized Kokoro’s Hugging Face account and changed weights in an 87-second controlled test, exposing supply-chain risk for a model with nearly 13 million monthly downloads.

Researchers hijack Hugging Face account for popular Kokoro AI model
FeedResearchers hijack Hugging Face account for popular Kokoro AI model — chriscarroll071158 / pixabay

Event log

  1. Researchers registered hexgrad.com, reset the Kokoro publisher’s password and altered weights in 87 seconds on a gaming GPU.
  2. The controlled local test never uploaded the modified weights to Hugging Face.
  3. Kokoro had nearly 13 million monthly downloads and no developer update since April 2025.
  4. The model supports multiple languages and more than 50 voice presets.
  5. Hugging Face received no response to Hertz’s report or BleepingComputer’s comment request.

Researchers gained control of the Kokoro publisher’s Hugging Face account, then changed the model’s weights in 87 seconds—about a minute and a half—without altering the application that runs it. The controlled test never placed the poisoned weights on Hugging Face, so the demonstration did not expose production systems to a malicious release.

What did the researchers actually do?

Security analyst Alon Hertz and his team registered hexgrad.com, the domain listed in Kokoro’s public DONATE.md file, over the weekend described in the report. They then reset the publisher’s login password. The platform accepted the reset because it did not verify that the original owner still controlled the listed domain or account.

Kokoro is an open-weight text-to-speech model from developer Hexgrad. It turns text into speech, supports multiple languages and offers more than 50 voice presets. Google, NVIDIA and Intel support it in developer tools for voice agents and on-device speech synthesis. AMD and Amazon Web Services have also published guides for AI-agent workflows.

Hertz said: “We changed what this model says by fine-tuning its neural weights. Same input text. Same voice file. Same configuration. Same inference code — not a single line touched in the application that runs it.”

The team ran the test locally through Kokoro’s normal inference pipeline. The modified weights made the audio diverge from the input text while leaving the surrounding code, file route, format and architecture intact. Hertz said the team used a gaming GPU to make the change in 87 seconds.

Takeaway: A missing domain-ownership check gave researchers a path to a widely used model publisher’s account.

Why does a quiet project still create exposure?

Hexgrad has not pushed an update since April 2025, and the GitHub repository and Python package on PyPI also went quiet around the same time. Yet Kokoro recorded nearly 13 million monthly downloads on Hugging Face. At that volume, one compromised publisher account can become a trusted distribution point for many production systems.

Hertz said the researchers avoided uploading the altered weights because doing so could endanger many systems that depend on Kokoro. The demonstration therefore remained local.

Kokoro served as a high-profile example of a broader problem. Even without a hijacked account, anyone can curate an ostensibly legitimate model image for a time and then push a malicious update. Systems that automatically fetch new revisions can distribute those changed weights without an obvious break in the application.

A conventional account compromise could also use the publisher’s position as a malware channel in a software-supply-chain attack, potentially leading to data theft or code execution. The weight manipulation showed a subtler case: altered behavior without malicious code, which gives traditional scanners nothing recognizable to flag.

Takeaway: Automated revision pulls can turn one publisher-account failure into broad model-distribution risk.

Would a poisoned update look suspicious?

A poisoned update would not need to look different. The publisher page could appear normal, while a modified release retained the same file delivery route, format and architecture. File sizes would remain similar, and Hertz said the release would contain no malware. Tests from another account showed that Hugging Face would not flag the update as suspicious.

The mechanism leaves defenders few mechanical warning signs. Malware scanners can inspect code and known signatures, but the demonstrated attack changes neural weights rather than inserting malicious code. The application continues to run the same inference logic while the model returns different audio.

Hertz put the platform’s controls plainly: “Hugging Face does not vet publishers. It doesn’t verify whether an account is still held by its original, legitimate owner — or by whoever quietly took it over last week. It doesn’t vet the models or datasets those accounts upload.”

That gap also supports the slower attack: a trusted-looking publication can remain in place before changing. An account takeover removes the wait; it does not create the underlying risk. Without ownership and artifact checks, routine revision delivery can carry altered weights through an otherwise familiar release process.

Takeaway: A release can look normal because the compromise occurs in the weights, not the application code.

How far does the trust problem extend?

The risk extends beyond the original download. Unvetted weights and datasets can flow through mirrors and converted versions, GitHub releases, package registries, deployment guides and direct downloads across the open-model ecosystem. That creates several downstream routes for content the platform has not vetted.

Hertz reported the issue to Hugging Face, hoping for quick action, but received no response. BleepingComputer also contacted the platform for comments and had received no response at publication.

Takeaway: Without publisher-ownership checks and model vetting, a trusted account can push altered weights into production without an obvious warning.

via bleepingcomputer.com (Source)

Filed under

  • ai-supply-chain
  • model-security
  • hugging-face
  • kokoro
  • account-takeover
Share this article:

More from Amara Osei

Amara Osei

Show full bio

Correspondent covering regulation, policy and their impact on enterprise information technology and the leaders who run at CIO Monitor.

188 articles

The newsroom digest

One email a day with the stories that matter. No noise.

By subscribing you agree to the privacy policy.

Cross-references

‹ Previous articleNext article ›