cross-posted from: https://feddit.org/post/28915273
[…]
That marketing may have outstripped reality. Early reports from Mythos preview users including AWS and Mozilla indicate that while the model is very good and very fast at finding vulnerabilities, and requires less hands-on guidance from security engineers - making it a welcome time-saver for the human teams - it has yet to eclipse human security researchers.
“So far we’ve found no category or complexity of vulnerability that humans can find that this model can’t,” Mozilla CTO Bobby Holley said, after revealing that Mythos found 271 vulnerabilities in Firefox 150. Then he added: “We also haven’t seen any bugs that couldn’t have been found by an elite human researcher.” In other words, it’s like adding an automated security researcher to your team. Not a zero-day machine that’s too dangerous for the world.
Immediately after the big announcements about Mythos there were followups by other teams that were able to find most of the same vulnerabilities with other existing models. I think the main takeaway there was that it’s just a matter of actually looking. Anthropic’s advantage may have been in the framework that let them do so in industrial-scale quantity rather than the cleverness of the particular model they used.
This sort of security scan is still new and important to pay attention to, but it’s not something that’s unique to Anthropic or that can be kept “contained.” Shades of how GPT-2 was considered “too dangerous to release” back when it first appeared. Comical in hindsight, and impossible to prevent anyway.
The one I saw was marketing hype by a company claiming to be able to do the same thing but cheaper. But when you read the fine print, you could tell that it was all just fudged.
It’s comical how people who need to believe that it’s all just marketing hype bought that marketing hype hook, line, and sinker. The implication that this would mean that LLMs are far, far more capable than anyone gives them credit for, completely slipped past them. Stochastic parrots with no understanding.
LLMs can be useful in this context, but Anthropic blew Mythos way way out of proportion. It absolutely was overly hyped.
Their own demonstrator had to work with a downlevel firefox so it would still have vulnerabilities that were already fixed before they even started.
It seems that their narrative is that other tools, some LLM and some not may be as good or better than Mythos at finding issues, but there were a couple of issues where Mythos was able to actually create a demonstrator, which the other models did not do. Which is relatively less interesting, as a human going from finding to demonstrator is generally not a huge part of the tedium, the tedium usually is in the finding.
They pitched it as “it is dangerous, it will escape confinement”, etc etc. But instead they had to explicitly start with a downlevel firefox with known vulnerabilities unpatched and they further had to disable all the security mitigations that in practice had already made the two “vulnerabilities” impossible to exploit.
It’s a matter of degree and exaggeration.
Or as they put it, they turned Firefox 147 in an evaluation.
I admit that I didn’t study their marketing materials, but that sounds kinda off. Maybe something got garbled?
That’s because those implications are blatant, open, clear lies. Your slop generator provides negative value to everyone except those who own it.