cross-posted from: https://feddit.org/post/28915273
[…]
That marketing may have outstripped reality. Early reports from Mythos preview users including AWS and Mozilla indicate that while the model is very good and very fast at finding vulnerabilities, and requires less hands-on guidance from security engineers - making it a welcome time-saver for the human teams - it has yet to eclipse human security researchers.
“So far we’ve found no category or complexity of vulnerability that humans can find that this model can’t,” Mozilla CTO Bobby Holley said, after revealing that Mythos found 271 vulnerabilities in Firefox 150. Then he added: “We also haven’t seen any bugs that couldn’t have been found by an elite human researcher.” In other words, it’s like adding an automated security researcher to your team. Not a zero-day machine that’s too dangerous for the world.
And if it’s like a lot of security scans, most of the results are technically correct, but, within the context of the project, not something anyone’s going to take the time to fix.
Note that in this case, very specifically, they had to yank Firefox’s javascript engine out of Firefox "but without the browser’s process sandbox and other defense-in-depth mitigations.” They had to remove the mechanisms designed to quash vulnerabilities.
And they had to test explicitly against Firefox 147 vintage because Firefox 148 had already fixed the two issues that Mythos exploited to get an impressive number. Before Mythos even ran the key problems had been found and patched…
I don’t mind leaving “technically correct” vulnerabilities in place while there’s no known way to create an exploit. If you’ve got a vuln with a known exploit and are relying on “but nobody is ever going to actually try that on us” - then you’re part of the problem, a big part.
It might be a config thing, but pretty often these scans will find issues which are only relevant on e.g. windows, when building a Linux container. Or the issue is in some XML parsing library in the base OS but the service never receives XML and isn’t public facing anyway. Context matters.
One that I have to copy-paste over and over are vulnerabilities in the CUPS printer driver chain that don’t apply because we don’t print arbitrary things, we only print things that we create. Yeah, there’s a vulnerability here in image-magick if you throw it such and such maliciously crafted… well, we only allow it to process our internally generated reports and there’s no pathway for maliciously crafted input to reach it, so…
This is why CVE scoring is used for severity. A vuln that doesn’t really give you anything, that you can only exploit locally, when already having elevated privileges? That’s going to be low priority for a fix.
And, yet, here I am - rebuilding a new interim image for our security team to scan so they can generate a spreadsheet with hundreds of lines of “items of concern” which are above our “threshold of concern” and most of them are being dismissed because of those justifications you just gave: local exploit only, etc. but I have to read every one, tease out the “local exploit only” language, quote it for the justification, over and over and over every few months.
Corporate anxiety is limitless.
You’re allowed to do that? Must be nice. We recently got told that you get one six-month justification, after that it must be remediated.
These are vulnerabilities for local access on a console which is operated in kiosk mode - users never have command line access, and the consoles themselves are rarely if ever network connected.