I mean, the thing with LLMs is that they tend to overdo it. They test completely unnecessary details. Your idea of having tests for all features is probably still way less tests than the typical LLM-generated commit…
And they’ll miss important test cases such that their change will pass all of their tests (they keep iterating until it does) but then will still fail on some common edge cases.
I mean, the thing with LLMs is that they tend to overdo it. They test completely unnecessary details. Your idea of having tests for all features is probably still way less tests than the typical LLM-generated commit…
And they’ll miss important test cases such that their change will pass all of their tests (they keep iterating until it does) but then will still fail on some common edge cases.