Linux is the name of a computer operating system kernel and a group of systems built on it, like Ubuntu or Mint. Linux is the basic infrastructure of the Internet. It’s important. A report ju…

tux snipes-hesitation

  • kibiz0r@midwest.socialEnglish
    18·
    3 天前

    This notion of “quality of output” misses something important though, which I think our economic system predisposes us to ignore.

    I think most of us have this concept that the function signature of “working on code” is something like:

    implement(spec): CodeChanges;

    But I think this hides some stateful side-effects. There’s a monadic thing going on here, and I think it’s really more like:

    implement(knowledge, spec): [CodeChanges, KnowledgeBundle];

    …where KnowledgeBundle is a whole bunch of things: latent info about how things currently work, obviously, but also the yet-unnamed missing abstractions you noticed, the existing named abstractions that are starting to strain — basically the sense of the trajectory of the codebase.

    Even really shitty programmers get some sense of the trajectory of the codebase, but this is something that even the best frontier models fail at.

    Tangent:

    I saw a pretty cool visualization of Kalman filters the other day (substantially AI-generated, coincidentally), and I thought it matched something about this dynamic.

    If you’re only concerned about the “next step”, aka “quality of output”, your error will accumulate and you’ll drift over time. You need to be able to evaluate where you are, so you can correct that error and stay on course. And to do that, you need to be present inside of the code and feel where the pain points are, or your estimation will be so far off that it’s useless.

    I think our economic focus on “output” and our obliviousness to internal states (or even denial that they exist at all) is going to make it very easy to fall into the trap of taking a long series of next steps that individually look very sensible, but over time accumulate unmitigated error and drift towards disaster.

    • space_comrade [he/him]@hexbear.netEnglish
      51·
      3 天前

      Even really shitty programmers get some sense of the trajectory of the codebase, but this is something that even the best frontier models fail at.

      Eh in my experience even good programmers are kinda bad at estimate the future trajectory of the codebase, I’ve seen experienced people design fancy abstractions only for those abstractions to fall flat on their face when future requirements present themselves.

      If you’re only concerned about the “next step”, aka “quality of output”, your error will accumulate and you’ll drift over time. You need to be able to evaluate where you are, so you can correct that error and stay on course. And to do that, you need to be present inside of the code and feel where the pain points are, or your estimation will be so far off that it’s useless.

      I feel like this inevitably happens to pretty much every piece of software, hell I’d bet Torvalds regrets a some technical decisions he made with Linux decades ago. Are LLMs worse at this than humans? Probably, but I’m not sure by how much.

      I dunno maybe I’m biased because I was always working with fairly crappy and haphazardly written commercial software but I don’t see LLMs being that much worse at writing code than humans in general.