cross-posted from: https://feddit.org/post/28915273
[…]
That marketing may have outstripped reality. Early reports from Mythos preview users including AWS and Mozilla indicate that while the model is very good and very fast at finding vulnerabilities, and requires less hands-on guidance from security engineers - making it a welcome time-saver for the human teams - it has yet to eclipse human security researchers.
“So far we’ve found no category or complexity of vulnerability that humans can find that this model can’t,” Mozilla CTO Bobby Holley said, after revealing that Mythos found 271 vulnerabilities in Firefox 150. Then he added: “We also haven’t seen any bugs that couldn’t have been found by an elite human researcher.” In other words, it’s like adding an automated security researcher to your team. Not a zero-day machine that’s too dangerous for the world.
Couldn’t even write a decent viral marketing ARG. Like if you’re going to go that route at least respect the art form.
Anthropic Mythos shaping up as nothingburger
AI PR doing AI PR stuff… At this point they will push any outrageous claim about capabilities or spend nearly any amount of money to keep that insane AI bubble from bursting.
AI PR is absolutely correct: https://www.flyingpenguin.com/freebsd-cve-2026-4747-log-suggests-mythos-is-a-marketing-trick/
One user pointed out that Claude “wrote code to exploit a known CVE given to it” and did not “crack” FreeBSD.
Worth noting that a company used extremely small models and found something like 80+% of what mythos is claimed to have found.
The claimed circumstances are different, “mythos, go find exploits in open source codebases” vs “model, find a vulnerability in FreeBSD”, but there’s a subtle distinction.
The system card’s own next figure kills the finding. When the top two most-exploitable bugs are removed from the corpus, Mythos’s FCE rate drops from 72.4% to… wait for it… 4.4%. (Figure 3.3.3.B, page 52) Under 5%!
Anthropic’s own language: “almost every successful run relies on the same two now-patched bugs.” (page 51)
There was still a comment on the bottom trying to cite the 73% number back to the author lol.
Ok this is much more interesting.
Man, I hate media. More drama than information.
I mean, idk why it’s a nothingburger. The narrative don’t sit with me. The comparison is weirdly dismissive. If it’s adding a new “elite researcher” then it’s already a win, and in fact it’s adding at least 3 - if assuming 8 hour work day, except they don’t need mental, bio and otherwise breaks and you can add as many as you have money for at any point with no ramp up.
Is this The Register backing up from some hype pieces they wrote? Or are they ashamed to have bought into marketing?
It’s acceleration tool in a field that’s very valuable to both blue and red actors, it’s time consuming and already blink and you miss it zero days and supply chain attacks.
Weird af article.
Right but essentially the Anthropic Marketing team made extraordinary claims about what Mythos is, pretended like it’s too dangerous for public use, that it can find and patch vulnerabilities undiscovered by human researchers for 20 years, etc. etc. but it’s own technical team and big firm partners haven’t brought forward evidence that it’s all that much. It’s probably not useless, and it might be an improvement in some ways to its other models, but having an “additional researcher” is not really something that is too scary for public use.
It’s like if a drug company was hyping up a miracle drug, but it’s really just acetaminophen and ibuprofen in a different dosage combination. It treats pain and fever fine enough, and maybe better in some cases but not as game changing as they say.
Right, that’s the general context I missed.
Yeah, the hype is always about making absurd promises and predictions of The End of All Things.
In reality, it’s like having a half competent Jr. security researcher. Nothing superhuman. The bonus is that it’s really hard to make new security researchers and so they’re too expensive for random github projects to employ.
Having a ‘good enough’ tool to check over code is an improvement over the current state of things, but it isn’t like we’re on the brink of a new generation of AI-enabled hacks or anything. Just a slightly better automated code reviewer.
No one ever said that the new model would not be usefull. But Anthropic hyped it up to a 0-Day machine, who finds 0-Days in every project with easy and in places they could not have been found by humans.
I think my problem is I’m so disentisized to hype I barely register it even intellectualy. I saw the glasswing page, skimmed it over and basically thought ‘curious’. No corpo marketing can get a rise out of me.
Turns out it’s just 4.7 or whatever and all the hype is BS marketing. Still a threat (like any coding machine), but not god mode.
