The Haiku 5.5 model is actually quite inexpensive. I’ve used it to implement multiple features this month and have only spent $4 of my $20 monthly budget.
Same. AI has fallen down repeatedly trying to analyse both of my repos. And they were written by a bad human originally. The problems are all the same, except AI will get you spaghetti faster.
That is a pitfall that is difficult to mitigate. When using these models your own code base is the primary input, and it will match the style. Garbage in, garbage out. It may be worthwhile to refactor prior using a model. Even having intelligible method and variable names can make a difference. Maybe do a sweep of simple renaming, and adding supplemental explanatory comments, then you might get better results?
Yours is a good point. I know that the guy who solo-created Redis uses AI effectively to write tests - he gives it a lot of handwritten tests and a lot of context, and then asks it to match the style and pattern, which it does well.
I’ve got something like half a million to a million lines, a crazy level of cyclomatic complexity, and a team that doesn’t see a problem with it. That’s the tough part - socialising the changes. We tend to arbitrarily rename vars/methods and not keep things in the same place, so there is a lot of drift.
I think AI might be able to unpack some of that, but there is a sort of irreducible accidental complexity that would require a human to make decisions about what some things mean. In some cases, you could get a gaggle of devs in the same room and they would not be able to arrive at a decision about what the code is doing or even what it should be doing.
Messy brownfield code requires a lot more meta-analysis from previous sessions, just to get all of the spaghetti code into a manageable set of rules and documentation.
Don’t just send the agent blindly into the code. Every session, the agent is just going in fresh without any context on how the code works. Break the codebase down to a 10,000 foot view as a summary, and then get more documentation and agent rules for each of its major sections. Good tests help a great deal, too, and if there aren’t tests, start building them. Start creating skills just for the codebase, or any common tasks you’re asked to do against it.
Also remember that you don’t have to write any of this stuff. Have it write the docs, rules, skills, tests, and just review it to make sure it looks right.
Everything is copilot. No more windows, copilot is my operating system. Microsoft has changed its name to Copilot, Inc., and Satya Nadella has updated his CopilotedIn profile name to Copilot Copilot. Copilot is my copilot.
The Haiku 5.5 model is actually quite inexpensive. I’ve used it to implement multiple features this month and have only spent $4 of my $20 monthly budget.
The legacy code I’m managing is a clusterfuck of the highest order. Even Sonnet struggles to analyze it sometimes 😑
I’m happy this is only a small part of my job, but oh my…
Same. AI has fallen down repeatedly trying to analyse both of my repos. And they were written by a bad human originally. The problems are all the same, except AI will get you spaghetti faster.
That is a pitfall that is difficult to mitigate. When using these models your own code base is the primary input, and it will match the style. Garbage in, garbage out. It may be worthwhile to refactor prior using a model. Even having intelligible method and variable names can make a difference. Maybe do a sweep of simple renaming, and adding supplemental explanatory comments, then you might get better results?
Yours is a good point. I know that the guy who solo-created Redis uses AI effectively to write tests - he gives it a lot of handwritten tests and a lot of context, and then asks it to match the style and pattern, which it does well.
I’ve got something like half a million to a million lines, a crazy level of cyclomatic complexity, and a team that doesn’t see a problem with it. That’s the tough part - socialising the changes. We tend to arbitrarily rename vars/methods and not keep things in the same place, so there is a lot of drift.
I think AI might be able to unpack some of that, but there is a sort of irreducible accidental complexity that would require a human to make decisions about what some things mean. In some cases, you could get a gaggle of devs in the same room and they would not be able to arrive at a decision about what the code is doing or even what it should be doing.
Messy brownfield code requires a lot more meta-analysis from previous sessions, just to get all of the spaghetti code into a manageable set of rules and documentation.
Don’t just send the agent blindly into the code. Every session, the agent is just going in fresh without any context on how the code works. Break the codebase down to a 10,000 foot view as a summary, and then get more documentation and agent rules for each of its major sections. Good tests help a great deal, too, and if there aren’t tests, start building them. Start creating skills just for the codebase, or any common tasks you’re asked to do against it.
Also remember that you don’t have to write any of this stuff. Have it write the docs, rules, skills, tests, and just review it to make sure it looks right.
Are you using api or subscription?
Technically neither - I use it via Microsoft Co-pilot subscription, and it’s a la carte for model usage from every provider.
Ah co pilot was much worse than claude code when I used it. Subscription is far far better the api prices are outrageous
Microsoft’s copilot branding is all over the place. Copilot CLI and GitHub copilot (IDE plugin) both let you choose which model you want to use.
Yes but not using claude code as a harness results in worse use out of their models from my testing
Let’s not forget Copilot is also an office suite lol
Everything is copilot. No more windows, copilot is my operating system. Microsoft has changed its name to Copilot, Inc., and Satya Nadella has updated his CopilotedIn profile name to Copilot Copilot. Copilot is my copilot.