• Aeder@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    edit-2
    5 hours ago

    The funny thing is that one of the reports on the glasswing project found out a lot (~55%) of the vulnerabilities AI finds are just false positives and not really worth reporting so they just increase the cognitive workload of unpaid maintainers.

    • JohnEdwa@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      1
      ·
      3 hours ago

      Yes, a 55% false positive rate technically means more often than not, it isn’t actually a valid vulnerability and it is wasting your time. But if you have 1000 boxes to go through while looking for a certain item, someone with a 50% false positive rate could identify two boxes for you, and it would be in one of them.

      So unless that number is “AI claims there’s a vulnerability on every other line of code you wrote”, which I kinda doubt, and is actually based on made bug/vulnerability reports, then isn’t is extremely helpful even if it’s only right half the time?

      • MrScottyTay@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        3
        ·
        3 hours ago

        Getting what I’ve seen even the actual vulnerabilities are the sort of stuff that is very very very hard to pull off, like it’s identified a .2ms window where if you input something specific, you break something and gain access. Like race conditions that even though they theoretically exist, they’re actually impossible to execute