Anthropic's Mythos model is purportedly so good at finding vulnerabilities that the Claude-maker is afraid to make it available to the general public for fear that criminals will take advantage. But early analysis shows that Mythos may not be as dangerous as some would have you believe.

Anthropic made Mythos available in preview to a select but ever-growing number of organizations under the title of Project Glasswing so they could find and fix vulnerabilities in their environment before criminals got hold of the purported zero-day machine and caused mayhem.

That plan didn't quite work as intended. On Wednesday, an Anthropic spokesperson confirmed to The Register that some non-Glasswing partners may have accessed the model - but not through Anthropic's production API.

"We're investigating a report claiming unauthorized access to Claude Mythos Preview through one of our third-party vendor environments," the spokesperson told us.

Intruder alert

The AI biz declined to name the third-party vendor, but said that it's a company Anthropic works with on model development. There's no evidence that unauthorized activity extended beyond the third-party vendor's environment or that Anthropic systems are affected, we're told.

Bloomberg, which originally reported the unauthorized access, said that "a handful" of people gained access to Mythos by making "an educated guess about the model's online location" based on Anthropic's previous models, and that these details were revealed in the recent Mercor data breach. 

Mercor is an AI staffing startup that supplies specialized contractors to major AI labs, including Anthropic. Earlier this month, Mercor said that it was "one of thousands of companies" affected by the LiteLLM supply-chain attack.

This group of unauthorized users reportedly belongs to a private Discord channel and gained access to Mythos on the same day that Anthropic announced Project Glasswing. Since then, it's been "playing around" with the bug-hunting machine, and doesn't have any interest in using the model for evil, according to Bloomberg. 

Regardless of what the group is doing with Mythos, their access illustrates a couple of key points. 

First: it's really hard to keep code under wraps (as also evidenced by Anthropic's earlier Claude Code source leak), especially when the folks who want to kick the tires on the new model are cybersecurity and engineering types - and they didn't even need to hack into any network or database to do it. Insider and supply-chain threats are the real deal.

"The Mythos breach didn't require a sophisticated attack," Ram Varadarajan, CEO at Acalvio, a deception-tech firm, told The Register. "It just required a contractor, a URL pattern, and a day-one guess, which means the 'controlled release' model failed at its weakest link before the model's capabilities were ever the issue."

Additionally, considering all the hype Anthropic spun around its new model, we shouldn't be surprised the genie is out of the lamp.

Anthropic's marketing message for Mythos was effectively a challenge, not dissimilar to a capture-the-flag exercise

"Anthropic's marketing message for Mythos was effectively a challenge, not dissimilar to a capture-the-flag exercise, where success includes claims of unauthorized access to Mythos," Tim Mackey, head of risk strategy at supply chain security shop Black Duck, told The Register.

Cutting through the hype

That marketing may have outstripped reality. Early reports from Mythos preview users including AWS and Mozilla indicate that while the model is very good and very fast at finding vulnerabilities, and requires less hands-on guidance from security engineers - making it a welcome time-saver for the human teams - it has yet to eclipse human security researchers. 

"So far we've found no category or complexity of vulnerability that humans can find that this model can't," Mozilla CTO Bobby Holley said, after revealing that Mythos found 271 vulnerabilities in Firefox 150. Then he added: "We also haven't seen any bugs that couldn't have been found by an elite human researcher." In other words, it's like adding an automated security researcher to your team. Not a zero-day machine that's too dangerous for the world.

It's a nothingburger. The adversary doesn't need Mythos to hack you

Anthropic, in announcing the new model, claimed Mythos identified "thousands of additional high- and critical-severity vulnerabilities." VulnCheck researcher Patrick Garrity, however, put the count as of last week at maybe 40. Or maybe none at all.

Another engineer, Devansh, scoured the Mythos-related CVE advisories and Anthropic's exploit code, 44-prompt transcript, and 244-page system card, along with Glasswing partner agreements, red-team writeups. He also looked at Aisle's replication study, which tested Mythos' showcase vulnerabilities on small, cheap, open-weights models and found they produced much of the same analysis.

Devansh ultimately concluded that while the bugs it found are real, the true Mythos story is "one of misinformation and hype."

• Mythos found 271 Firefox flaws – but none a human couldn't spot
• Nobody knows how many CVEs Anthropic's Project Glasswing has actually found
• Anthropic's mysterious Mythos AI threatens to upend the infosec world
• Anthropic: All your zero-days are belong to Mythos

For example, the Anthropic-claimed 181 Firefox exploits ran with the browser sandbox turned off and the FreeBSD exploit transcript "shows substantial human guidance, not autonomy." 

Additionally, the "'thousands of severe vulnerabilities' extrapolates from 198 manually reviewed reports. The Linux kernel bug was found by Opus 4.6, the public model, not Mythos," Devansh said.

Another researcher, Davi Ottenheimer, pointed out that the security section (Section 3, pages 47-53) of Anthropic's 244-page documentation "contains no count of zero-days at all. With no CVE list, no CVSS distribution, no severity bucket, no disclosure timeline, no vendor-confirmed-novel table, no false-positive rate."

Ottenheimer likens it to "the ending of the Wizard of Oz, a sorry disappointment about a model weaponizing two bugs that a different model found, in software the vendor had already patched, in a test environment with the browser sandbox and defense-in-depth mitigations stripped out."

Snehal Antani, co-founder and CEO of offensive AI hacking company Horizon3.ai, told The Register, "attackers didn't need Mythos to accelerate vulnerability research, 4.6 and open source models have already been accelerating the vulnerability process."

When asked if the security community should be concerned about unauthorized Mythos access, Antani said no. "In my honest opinion, it's a nothingburger," he told us. "The adversary doesn't need Mythos to hack you." ®

More about

• AI
• Anthropic
• Security

More like these