• Boost@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 day ago

      I use Claude every day for actual production software used in a real product and I can say with confidence that even the most advanced publicly available models totally lack the higher order thinking required to build good software. They’re pretty good at writing code, when directed well. They are not good at architectural or design decisions and they’re even worse at actually following decisions made outside of their context. Those are the actually challenging parts of software engineering and you have to beat them into the model every step of the way or your software quickly becomes an unmaintainable mess.

      • redballooon@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        4
        ·
        1 day ago

        Use Spec driven development and a good verification loop to overcome the “beat it into the model at every step”.

        • Boost@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          18 hours ago

          Then someone has write the spec and develop the verification. Which is, again, the hard part. If you’ve ever worked when a business or requirements team, you’ll also know that the spec and verification are constantly moving targets. With a lot of time and attention, you can create a whole ecosystem of agents that will output somewhat acceptable code at scale (or so I’ve heard from people I trust), but that just moves the human effort required to develop and maintain the software over to developing and maintaining the automation. Then if anything breaks catastrophically in a way AI can’t fix, your team’s actual software skills have totally stagnated.

      • red_tomato@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        2
        ·
        1 day ago

        Most human programmers are bad at architecture. They’re making lots of mistakes too. Perfection is not a criterion for intelligence. In that case no human is intelligent.

        • Boost@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          17 hours ago

          Yeah, that’s why software engineering is a valuable skill that demands significant compensation. Even many of the people who do it for a living aren’t that great at it. I don’t even consider myself to be an especially skilled software engineer. There is a lot more to the job than simply writing code and it takes several years to become truly competent, with lots of mistakes along the way.

          Go ahead and read my comment again and let me know where I demand perfection or even talk about the criteria for intelligence. Defining intelligence is (imo) more of a philosophical question than a practical one so I don’t find a lot of practical value in speculating on whether or not LLMs posses some kind of intelligence. It’s not really relevant to what I’m talking about.

          My point is that, as it stands right now, I use these models on a daily basis to complete actual work and they cannot be trusted to competently make the kinds of decisions required to build good, maintainable software. Sometimes, if you’re working on a well known common design structure or a common problem, an LLM can pantomime good design decisions (because it’s well represented in the data set) but as soon as you step into a situation that requires actual creativity or must account for non-obvious pitfalls in the codebase, it’s a crapshoot. Far worse than an inexperienced software engineer. I believe this is an inherent limitation of transformer based models because they seem to lack any kind of mechanism for this type of higher order thinking.

          • red_tomato@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            1
            ·
            edit-2
            12 hours ago

            My comment was: AI in 2018 is intelligent in a very different way from AI in 2026.

            The common thread I see is: well AI can do these things well, but fail at these other things. Therefore, it cannot be intelligent.

            Even if the things it fails at are things humans would normally fail at. Any test for intelligence in AI must also apply for humans!

            I want to have a clear cut answer: why isn’t AI of 2026 considered intelligent. No one in this entire thread has given a clear answer - besides gut feelings because they feel uncomfortable with the idea that AI is intelligent.

            • Boost@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              edit-2
              5 hours ago

              Sure, there’s been a lot of progress in the field and AI models are certainly more useful now than they were in 2018, but I think that’s mostly do to 1) the size and scale of the models, and 2) the harnesses around them. I think that commenter is right in that there are some apparently inherent limitations to the technology (by which I mean transformer based models) which still persist today.

              well AI can do these things well, but fail at these other things. Therefore, it cannot be intelligent.

              Well that’s a dumb argument and one I’m not making. After using these things quite a bit for actual work, it’s clear to me personally that these things are pretty good at parsing data and spitting out statistically correct responses, based on the training data, but that there is no real reasoning or creativity happening in the process.

              Even if the things it fails at are things humans would normally fail at. Any test for intelligence in AI must also apply for humans!

              Why? Why would our standards for people be the same as our standards for a machine? There are many things that an LLM is much better at than any human will ever be (spitting out large amounts of mostly coherent text in a short amount of time, searching and parsing large amounts of text in a short amount of time, etc) and some things that an LLM is way worse at than most people (counting the number of letters in a word when that specific scenario hasn’t be well covered in training, due to how tokenization works. Arithmetic, etc.) There are many many reasons why people and machines are not really comparable, I don’t think inventing a machine that’s pretty good at natural language changes that.

              I want to have a clear cut answer: why isn’t AI of 2026 considered intelligent. No one in this entire thread has given a clear answer - besides gut feelings because they feel uncomfortable with the idea that AI is intelligent.

              I think the reason you won’t get a clear cut answer is that there isn’t really a clear cut definition of what intelligence even is or how to quantify it in any objective way. If you ask 20 people “what is intelligence?” or how to tell if something possesses it, you’ll get 20 different answers. Our own methods of rating our own intelligence are notoriously subjective. People who have devoted their entire professional life to this subject will tell you how little we know. I am not uncomfortable with the idea that AI is intelligent because I know that it isn’t. I can tell you why I feel that way but this comment is long enough and I have a feeling it won’t satisfy you anyway.

              All I’ll say is it’s pretty irritating to hear a bunch of people insist that this thing is going to take over your job and then world, and then when you actually use it, you quickly see all the gaping holes in its capabilities, despite the impressive first impression. It becomes pretty clear pretty quickly that this is a useful new technology but not exactly the next industrial revolution. At least not as it currently exists. I think we’ve already started to hit diminishing returns, but maybe I’m wrong. I can’t help but feel like the people still smugly replying to “AI kinda sucks actually” with “well you must not be using it right 😏” at this point either don’t know, don’t care, or are actively lying for financial or ideological reasons.

        • aksdb@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          24 hours ago

          Intelligence isn’t a one-dimensional axis either. Intelligence exists in many facets. Agents (so LLMs combined with tools, memory, etc.) overlap pretty much with one aspect that we perceive as intelligent. The main risk is basically the same as with humans of similar one-sided intellect: if you expect them to be equally intelligent in other areas, you will get disappointed or even hurt.

    • Wildmimic@anarchist.nexus
      link
      fedilink
      English
      arrow-up
      16
      arrow-down
      1
      ·
      2 days ago

      I am not a non-user. I run local models, because i am a tech enthusiast, and am watching the space closely. I will never spend a cent or use the offerings of OpenAI and Anthropic tho - i will not finance the people who are providing the tools to bind us all.

      • redballooon@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        12
        ·
        2 days ago

        Open weight models are not far behind and good enough to get software developers jobless, but there is a huge difference in what you can do with 1M context and 500token/sec or 60k context and 40token/sec.

        • Wildmimic@anarchist.nexus
          link
          fedilink
          English
          arrow-up
          12
          ·
          2 days ago

          Sure! End the AI Bubble and i’ll go ahead and buy a bunch of H200’s.

          But for real, if you structure your tasks well, you can get by for most things with local models, and for a small business in IT, investing in dedicated hardware you own is a smart move. The employer of an acquaintance of mine bought some inference hardware for his 5 engineer business, and they rely mostly on that. Around 80% of all tasks are done by a GLM5.3 model, and while they currently still use Claude for some things, the percentage of external inference has dropped dramatically in the last months, because people have adapted on how to structure tasks for the local model.

          Noone can explain to me how Claude, OpenAI and the NeoClouds plan to become profitable businesses when you can reduce inference costs by 90% by getting your own inference - and it’s not even that hard to set up. if you already have IT staff for your inhouse stuff, they will probably manage.

          • BenevolentOne@infosec.pub
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            3
            ·
            2 days ago

            Someone will come up with a decent accelerator eventually. Right now we’re in the pre-soundblaster days where audio sythesis is being done by the CPU but for neural networks.

            We’re repurposing graphics hardware, these are primitive times.

            Also, there, I explained it for you.

            • Wildmimic@anarchist.nexus
              link
              fedilink
              English
              arrow-up
              6
              ·
              2 days ago

              I have to disagree here. We are on the absolute edge of current physics regarding miniaturization - without a massive breakthrough in quantum physics on how we can use subatomic particles for transistors we are nearing the maximum we can shrink these circuits. These are already optimized for exactly this kind of math - a Vera Rubin will never be able to render a game. Also, the mathematical basis (a ton of matrix multiplication) does not give much hope that we can replace all this math with something simpler. Maybe a radical redesign taking a completely different approach can help, but i wouldn’t count on that - it would mean reinventing the whole damn thing on a completely new base.

              I can’t really see any explanation when i take the above into account.

      • redballooon@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        1
        ·
        2 days ago

        In any case, confidence without knowledge about the matter, that is something we laughed at LLMs just 2 years ago. Those became much much better at it.

        Now suddenly it’s en vogue with humans.