• 0 Posts
  • 308 Comments
Joined 3 years ago
cake
Cake day: June 30th, 2023

help-circle
  • My point isn’t AI is good or bad, but that the difference is how much it gets leaned on.

    In this case, it’s how AI is (assumed to be) used at MS vs how it was used in the OP.

    MS appears to be heavily leaning on generative AI producing code. In my own experience, that is pretty good these days at responding to a prompt with a series of actions that achieves the desire of the prompt, but is bad at creating an overall cohesion between prompts. It’s like it’s pretty good at making lego blocks but if you try putting it all together, it looks like you built something from 50 different sets, plus the connections between the blocks are flawed enough that it’s liable to collapse the more you put it together.

    In the OP, AI is being used to submit bug reports. This one can be thought of as using an AI to write a book report instead of using an AI to write the book in the first place. If the AI writes a shitty report, it has zero effect on the book itself. But the AI might just include a list of all the typos in its report, which is useful for correcting the errors in the book.

    Also, game studios forgetting to replace placeholders is yet another issue more on the process itself, though it can also show a lack of attention to detail and maybe indicate that an AI was handling more of the process. A decent system would flag all assets for whether they are placeholders or final and then include a review of all flags before publishing to catch something like this.

    So this isn’t a general defense of using AI, I’m just saying that it’s possible to use it without everything it touches turning to slop, but that it often isn’t used like that, resulting in slop.

    And it’ll be easy to fall into the slop trap, what with how it’s always making leaps and bounds inprovements that help with instances of it fucking up but don’t resolve the fundamental issues that will probably mean LLMs will always produce some sort of slop (because everything boils down to some sort of word association, just with a massive set of conditional probabilities encoded into it that gives it the illusion of understanding).


  • The same reason any personal projects (and not using it to diminish what linux projects are but to say that the people working on them do it because they want the project to progress, not because of any financial incentive) can do better then commercial projects: where the passion is at.

    Someone just looking to get paid is more likely to say “ok this is good enough” and move on to the next thing. They are more likely to have managers breathing down their necks to get something done by some arbitrary deadline, too.

    It’s why indie games have been able to compete with AAA games. The latter are following a formula to get paid, plus are more willing to make compromises in the name of either saving costs or increasing revenue. The former just want to make their fun idea reality.

    Also, MS has invested a ton of money into AI and seem to be getting desperate for a return on that. Which means there’s a certain amount of denial about the quality. It’s not just a tool to them, but a tool they desperately need to work and prove it’s worth throwing a ton of money at.

    But for anyone that it’s simply a tool for, it can be useful. They are great rubber duckies. Like my last interaction with one was a case where it did horribly and was completely wrong about what “we were discussing”, but I still got to the right conclusion despite it because going through the conversation helped me think it through.

    And though it makes a lot of mistakes, its feedback isn’t always wrong. The fact that it can rehash previous things from its history means its good at spotting new instances of problema that have already been solved. So accepting bug reports should be fine, just with the understanding that they each need to be looked at and some reports will need to be rejected because they are wrong.


  • Sorry aliens but you’ll need to go back to your science labs because we have since discovered how to compact discs themselves. No more data discs the size of records, we can fit an entire 70 minutes worth of full fidelity (to our ears) digital audio and then surpassed even that and managed to get it up to 74 minutes! 700 times 2 to the power of 23 bits of arbitrary data (or maybe it’s just 700 times 8,000,000, we never did figure out the concept of honestly describing things marketers want to sell), all within our outstretched fingers or around a single extended finger.


  • It’s about 300 samples for an estimate of the distribution with a 95% confidence iirc. That’s assuming the samples are representative (unbiased) and 95% confidence doesn’t mean it’s within 95% of reality, but that 5% of tests run in such a way would be expected to be inaccurate (and there’s no way of knowing for sure which one this particular sample is because even a meta study will have such an error rate, though you can increase the confidence with more samples or studies, just never to 100% unless you study every possible sample, including future ones).
















  • Most of the AI industry is currently stuck in a kind of uncanny valley where it’s close enough to fool people who don’t care about details or who so desperately want to make money that they deny the reality that these AIs aren’t actually good at very much.

    But there’s been so much money invested in it that they are desperate to make it generate some revenue and profit and keep shoving it into things, hoping that their thing will be the one the public finally latches on to.

    It’s also management types that really bought in to it. The kind of managers that don’t know shit and will make impossible requests, or think something simple is hard and something hard is simple because they don’t actually know much about the jobs they are managing. But they do have the power to direct those under them to use the AIs as well as get of or dismiss the opinions of those pointing out the emperor has no clothes.

    Right now, they are hoping to find that substance that will keep the AI bubble from popping. But IMO the problem is fundamental to the big data approach to AI of “throw a ton of data at a generic correlation engine and hope that it ends up smart”.