Microsoft's AI bug hunter finds 11-year-old memory flaw in Rust standard library

Microsoft researchers have found a memory-safety bug in Rust's standard library that sat undetected for nearly 11 years, along with other flaws that can trigger undefined behavior even in applications written entirely in safe Rust. Two of the findings affect stable Rust, including one that can lead applications to access freed memory.

The oldest bug involves Rust's process::exit function. Researchers at Microsoft's FORGE Lab discovered that calling the function from a child thread can cause the runtime to remove the main thread's alternate signal stack while that thread is still running. If the main thread then hits a stack overflow, the operating system may fail to deliver the signal correctly, producing a segmentation fault instead of Rust's intended stack-overflow handling.

"The process::exit signal stack bug was in the standard library for nearly 11 years," the researchers wrote in a Microsoft security blog post. They found no reliable exploitation path under the conditions they examined and assessed practical exploitation as unlikely.

The finding matters because Rust's ownership model and type system exist precisely to prevent the memory-safety failures common in C and C++. Yet the standard library relies on unsafe code internally for operations the compiler cannot verify. Components such as Vec and String use these operations but expose them through safe APIs — and those APIs must meet a strict soundness requirement: "no matter how safe code uses it, it must be impossible to trigger undefined behavior."

In short, the bugs sit below the safety guarantee developers rely on.

How did AI find what humans missed for a decade?

The research used Hyenas, FORGE Lab's AI-powered vulnerability-scanning system, to investigate unsafe code in the Rust standard library. Hyenas is the lab's research version of codename MDASH, Microsoft's agentic vulnerability-discovery system.

Microsoft said some of the bugs were found with help from Hyenas. Researchers then examined the underlying code, reproduced the issues, assessed their security impact, and reported them to Rust maintainers.

The company grouped the findings into three categories:

• unsafe code that assumes operations will not panic;
• code that trusts user-provided implementations to behave correctly;
• code that makes incorrect assumptions about the underlying system.

Microsoft stressed the practical risk is limited. Each flaw requires unusual conditions to trigger, and a standard library bug becomes "a serious vulnerability only if a real program reaches the API in that way," the researchers wrote.

What exactly was broken?

The first category produced a panic-safety bug in Rust's MapWindows::clone implementation. The function created a buffer before fully initializing it, then called a user-provided clone operation to populate it. If that operation panicked, Rust's unwinding mechanism could invoke the buffer's destructor and access uninitialized memory. The reported fix prevents the partially initialized buffer from being destroyed before initialization completes.

A second finding involved str::replace. The function made an incorrect assumption about a custom implementation of the nightly Pattern trait: a custom implementation could supply a non-ASCII byte where the code expected an ASCII character, causing the replacement to produce invalid UTF-8 data. Subsequent string operations could then trigger undefined behavior.

Microsoft said the first two categories were already being studied in 2021 through Rudra, a static analyzer built by members of the same research team. The standard library had addressed panic-safety problems as far back as 2015, yet similar weaknesses keep appearing in its implementations.

The third category concerns interactions between Rust, operating systems and C libraries. Beyond process::exit, researchers identified a flaw in CommandExt::exec involving the process-wide environment variable array. The function could modify this shared state while holding a read lock, potentially letting other threads read inconsistent data. Under certain conditions, concurrent operations could leave the environment pointer referencing freed memory.

Both the process::exit and CommandExt::exec findings affect stable Rust. A proposed fix for the environment-variable issue — which avoids modifying the shared array by calling execve directly — was under review when the blog was published. Two issues from Microsoft's team remained open at publication, alongside related reports from other researchers and fixes still in review.

Where do the findings go next?

Microsoft said the results feed into Microsoft Perception, its multi-agent security system, and will help improve MDASH, which brings agentic vulnerability discovery into Microsoft Defender. The company's Web App Pen Test Agent, for example, can use MDASH findings to prioritize testing, assess exploitability against live applications, and support remediation.

"Our goal isn't to simply find more bugs," the researchers wrote, but to identify meaningful issues and improve the safety of foundational software that developers and organizations depend on. The Rust project had not responded to a request for comment at the time of writing.

Even safe languages carry unsafe foundations — and AI tooling is now auditing them.