• Madagaskar_sky@lemmy.world
      link
      fedilink
      English
      arrow-up
      43
      ·
      2 days ago

      It’s just open weights. You never know what it was trained to do. That’s actually a hysterical take, because you can always check its work against other LLMs, or God forbid check it yourself, and as yet no such activity has been seen.

      • chaospatterns@lemmy.world
        link
        fedilink
        English
        arrow-up
        17
        ·
        2 days ago

        It means they release the model so you can run it yourself with enough hardware. Kind of like open source. Governments and large companies should be self-hosting this kind of stuff because of security, availability, and IP-leak risks anyway.

        With LLMs, there is some risk of hidden malicious actions embedded in the weights that you can see until you ask it a magic question and it says “activate skynet.”

        Whether or not the government is doing this because of fears of hidden actions in the weights or just because OpenAI/Anthropic CEOs told the government to give them more money is unclear.

        • sp3ctr4l@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          4
          ·
          1 day ago

          I can run Bonsai, basically a distilled version of Qwen 3.5 (Qwen being the other major family of open weight Chinese developed LLMs) on my Steam Deck.

          It ain’t fast, but its kind of nuts that it can run at all on hardware that is that underpowered, by typical LLM hardware standards.

          You just run the thing in a sandbox of some kind, do not give it access to the outside internet or the rest of your computer, and the worst thing that is likely to happen is … you configure it slightly wrong, it uses too much RAM, and your system crashes. And your system can then reboot again, just fine, tone it down a bit to use less RAM.

          For a large organization with networked security concerns and consistent uptime requirements, it is simply insane to just plug an LLM directly in to that, without architecting an entire system built around ‘assume the LLM will fuck up/go insane’; sanboxes within sandboxes with highly controlled connections to other sandboxes.

          But that would be a significant feat of actual software engineering to construct that for/around an actually complex network, and would take time, so fuck that, we want line go up is what most companies decided to do instead, just fucking raw dog it.

          But yeah the main ‘problem’ with LLMs in this sense is idiot CEOs just demanding the magic machine god be plugged right in to their very fragile, barely sensible networks and workflows, where the LLM is then going to break things in roughly the same way as would the nepo hire intern who was just directly promoted to sysadmin…

          …not that an LLM has a secret activation phrase that will cause it to shut down or go berserk in a specific way.

        • deadcream@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          2 days ago

          Nobody runs it on their own hardware though (which costs tens of thousands of dollars). I’m willing to bet $100 that government institutions that were using it were just using cloud API (probably even Chinese one).

        • Noxy@pawb.social
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          thanks for the explanation, I’m still not sure what “weights” are here, but I’m also not sure I want to know anything more about LLMs than I already unfortunately do, so I won’t ask you to explain it. :P

          • wonderingwanderer@sopuli.xyz
            link
            fedilink
            English
            arrow-up
            3
            ·
            2 days ago

            LLMs are basically large matrices of parameters called weights. It’s more complicated than that, but the weights are the parameters. Those embedded decimal numbers that get adjusted during training.