LLM is a statistical model, it does not reason. It is not capable of neither honesty nor deliberate lies. It works sometimes, because it was trained on thousand standard case analyses by human programmers, but it is also prone to hallucinations, which are unavoidable by design.
I know, the reason I shared that example is because there is no humanizing the LLM and it’s used exclusively for its ability to pretty effectively parse syntax to find common higher level bugs in specific C frameworks.
For context:
I never use LLMs when programming because I find them to be more of a hassle. They show me down and muddy the water on design principles.
Even a “bad” design pattern that you’re consistent with is 1000x better than a “good” design pattern that is inconsistently applied. LLMs are fantastic at the later. Things that look great in isolation, but begin to fall apart when they come up against the weight of a larger project.
Especially when it’s a new project without a ton of existing code for it to kinda copy/paste.
Using one to find minor use after frees that are almost impossible to hit, or duplicated calls is fine by me though. Those are usually a pain to find anyways, and there’s almost always 100 more important things to spend time on. If someone wants to burn tokens creating a backlog of easily fixed, reproducible errors like that all while disclosing exactly what they’re using to find them sure. That’s not annoying at all, especially when they only open the PRs after discussion with the project maintainers.
Well if you’re going to be pedantic, they don’t hallucinate either. They just repeat patterns, and sometimes false patterns can emerge in large data sets.
Well thats not always why they give an incorrect prediction. It’s also because statistics are inherently never going to reflect reality perfectly (“all models are wrong, some are useful”), but also we intentionally inject randomness into LLMs and other ML programs to ensure the output isn’t over fitting and spitting out raw training data. Usually adjustable via seed values, temperature, etc
Here’s one of the few examples where I’ve seen it be actually used in an okay way.
Full transparency, used to generate a standard check and not do giant thousand line PRs, honesty about false positives, and no bloated claims really.
LLM is a statistical model, it does not reason. It is not capable of neither honesty nor deliberate lies. It works sometimes, because it was trained on thousand standard case analyses by human programmers, but it is also prone to hallucinations, which are unavoidable by design.
I know, the reason I shared that example is because there is no humanizing the LLM and it’s used exclusively for its ability to pretty effectively parse syntax to find common higher level bugs in specific C frameworks.
For context: I never use LLMs when programming because I find them to be more of a hassle. They show me down and muddy the water on design principles.
Even a “bad” design pattern that you’re consistent with is 1000x better than a “good” design pattern that is inconsistently applied. LLMs are fantastic at the later. Things that look great in isolation, but begin to fall apart when they come up against the weight of a larger project.
Especially when it’s a new project without a ton of existing code for it to kinda copy/paste.
Using one to find minor use after frees that are almost impossible to hit, or duplicated calls is fine by me though. Those are usually a pain to find anyways, and there’s almost always 100 more important things to spend time on. If someone wants to burn tokens creating a backlog of easily fixed, reproducible errors like that all while disclosing exactly what they’re using to find them sure. That’s not annoying at all, especially when they only open the PRs after discussion with the project maintainers.
All the code is “in there” – it’s a matter of forcing it to produce correct code. There’s tremendous profit if you can.
The same applies to a random number generator.
Well if you’re going to be pedantic, they don’t hallucinate either. They just repeat patterns, and sometimes false patterns can emerge in large data sets.
Well thats not always why they give an incorrect prediction. It’s also because statistics are inherently never going to reflect reality perfectly (“all models are wrong, some are useful”), but also we intentionally inject randomness into LLMs and other ML programs to ensure the output isn’t over fitting and spitting out raw training data. Usually adjustable via seed values, temperature, etc