Gladly, though tonight I have to write 5 more letters of recommendation. Have one for now, and I’ll edit this when I have time. (I have edited in some more). Hopefully this one, the most damning in my opinion, shows how the vast majority (80%) of low-skill users will inevitably choose to become so cognitively reliant on AI that their performance will not only fail to improve, but actively deteriorate, making it impossible to become the sort of expert user necessary to actually use LLMs without the pitfall.
I also can’t help but point out my second-favourite AI study, Vending Bench, which shows how an LLM will, for no apparent reason, decide to threaten nuclear annihaltion over an hallucinated error in shipping. LLMs are not expert systems. They cannot be experts. LLMs cannot consistently operate without constant expert oversight, even for something so simple as stocking a fucking vending machine.
I think my problem with things like this is I heard the same bullshit growing up about using calculators or the internet, or cell phones or then smart phones.
Having access to any info you need is bad actually because now you don’t just have to remember. Solving problems with calculators is bad because you don’t have to actually do the hard work of the “real math”.
Just because people can choose to be idiots doesn’t mean everyone will, and people chose to be idiots long before LLMs gave them access to potentially being correct more often.
Idk man, Lemmy’s hate boner for AI goes far beyond reasonable levels into irrationality, its almost ironic if it wasn’t so fucking annoying. You can’t have a rational discussion, just look at this thread. Literally “you’re a fucking npc if you use AI.”
I grew up in interesting times for computing, and I already understood at a young age how little most people using them then even actually understood how they work because of the abstraction, then smart phones even moreso. So from that aspect I get the concern, however I do not think we can force people to try and understand by shaming them for not being fucking nerds like us.
Did you read either my original point or the study you’re replying to? My point is that the problem is with the nature of the tool first, and with the fact that people seem to treat it like some fucking oracle second… The fact that people misuse it is not some referendum against the populace, but against the fucking psychotic corpocrats and those who fellate shitlords like Altman, because they’re pretending that LLMs are the solution to every problem, and as if they’re true “AI”. That’s what needs to be fought, and it can only be done with education. I’m not sure why you’re replying to me, when literally every point, including your ad hominem indicating that I have some “hate boner”, or the one where you imply that I must agree with the thesis of the post to engage with a comment in it, can be easily seen as farcical by anyone with the literacy of a thirteen-year-old.
in my original comment, my whole point was that these other tools are different, and that LLM use can only be responsible when checked, and that users in the first years of their profession are unlikely to be able to do the hard checking required to turn hallucinated nonsense into something actually useful. The consequence is that, unlike a calculator or a computer, these LLMs will lead to a decrease in experts over time, reducing the number of those who can actually use LLMs efficiently and, more importantly, functioning as a barrier to those attempting to actually become experts. It requires skill to use a calculator at all, and it requires skill to use a computer for any given task. LLMs are more like the internet, which requires no skill to use to generate an answer, but which will constantly produce inaccurate information. The difference is that LLMs are worse, because there’s no part of the LLM which can look at that output critically and say “actually, that last commenter was an idiot, and the real solution is this”.
In the study to which you were ostensibly replying, tens of thousands of students were given free choice over whether to use AI, and how to use AI. Of those who chose to use AI, 80% chose to offload their cognition. Those who did saw approximately a 22% drop in performance in_every single subject_. Not everyone will choose to be idiots, but even in a cultural context where performance on a single test determines the entire trajectory of your life, over three quartets chose the idiocy. If you can’t see how that’s a direct counterpoint to everything you just said, I can’t help you. You should really read the study, because it’s worse than even I summarised here.
I think you’re misunderstanding my point a bit, and I’ll admit the calculator comparison was overly simplistic. I’m not saying LLMs are the same as calculators or that they don’t have unique problems. I’m saying those problems don’t automatically make the technology inherently detrimental to learning.
And I actually agree with you about people blindly trusting LLMs. I see this shit at work all the time in tech support. People will throw a question into Copilot, get a confidently incorrect answer, and then pass it along to other people without even attempting to verify it. It’s incredibly frustrating, especially when they’re doing a job where they really should be developing the knowledge to recognize when something is wrong.
But that’s kind of my point. There’s a massive difference between using an LLM to avoid thinking and using one to help you think, learn, and accomplish things.
The Chinese education study is genuinely concerning, I’m not dismissing it. But the 80% figure is based on behavior consistent with outsourcing homework, not the researchers somehow knowing that 80% of students consciously chose to stop thinking. And the students who continued putting comparable effort into their homework experienced much smaller learning losses. That seems like a pretty important distinction.
The other study shows a correlation between AI use, cognitive offloading, and lower critical thinking scores. Again, worth taking seriously, but that doesn’t establish causation or prove that using LLMs prevents someone from developing expertise.
And Vending-Bench? Yeah, LLMs can do some absolutely stupid shit when left to operate autonomously. I don’t disagree. But how does that invalidate using one as a tool under human direction? I’m not asking an LLM to independently run my life or make decisions without oversight.
I use LLMs constantly for development work. I’ve been a computer nerd since I was a kid, taught myself Linux, networking, messed around with programming, and all sorts of other things through documentation, forums, and tutorials. But programming was always something I struggled to actually do, despite understanding a lot of the underlying concepts.
LLMs have helped bridge that gap. I can now build software I literally couldn’t have built before. I can investigate unfamiliar systems, ask questions about concepts I don’t understand, explore different approaches, and actually put that knowledge to use.
And no, that doesn’t make me an expert. I’m well aware of the gaps in my knowledge, which is why I use things like SonarQube, CodeQL, testing, and experienced developers to help catch things I might miss.
But I am learning, and I’m accomplishing things I couldn’t before. I don’t see how that’s somehow a negative.
I also agree that the corporate AI hype is ridiculous, that these companies oversell what their models can do, and that people treating LLMs like oracles is a serious problem. You’ll get no argument from me there.
Where I disagree is jumping from “a lot of people use these tools poorly, and that can harm learning” to “these tools will inevitably reduce the number of experts and prevent people from becoming proficient.”
That’s a much broader claim than I think these studies actually support.
And that’s really what frustrates me about the AI discourse on Lemmy. There’s plenty of legitimate criticism to be made, but it so often turns into this blanket dismissal of anyone who uses AI, like we’re all just mindlessly copying whatever ChatGPT spits out.
I’m not defending OpenAI or Altman or any of these corporations. I’m defending the idea that these tools can be genuinely useful for people who know their limitations and actually want to learn.
Again, why are you mischaracterising what i have said, as if I’ve said there’s no legitimate use for LLMs? i just don’t get why you’re coming at me with that. I said that it is possible to use an LLM to increase efficiency. It was the very first point I made in my original comment, but you keep lumping me in with some nebulous “general discourse on Lemmy” that has some “hate boner” for everyone who has ever touched an LLM. Why do you do that? It’s just weirdly dismissive. What, specifically have I said to warrant this characterisation?
As far as the Chinese study, I contend that it does show precisely that, when given the option to cognitively offload to LLMs, tens of thousands of perfectly capable and mentally fit individuals chose to do that. It’s not like cognitive offloading is any different from cheating on some homework. It’s the easy way out, and these people thought it would serve them. It appears, to me, that you seem unwilling to apply a study of tens of thousands of low-skill users to the directly-related general case of low-skill users cognitively offloading and experiencing skill atrophy in the workplace. This skill atrophy inherently cannot lead to expertise. I fail to see a distinction between the choice to outsource homework and the choice to outsource coding or any other task for which a low-skill user might employ an LLM.
You mention that you already have a significant baseline from which you can attempt to check the hallucinations of an LLM. You are using it in concert with other sources to learn. You are not taking it at face value. You clearly are a member of that 20%, and I’d go so far as to bet that the vast majority of people who’ve found their way to the fediverse are likely to also fit into that 20%. However, if we take as given the total ubiquity of LLM use in coding (surely you can agree that it is now virtually ubiquitous), then the 80% misuse factor indicates that, for every 5 individuals who would otherwise become coders, about 4 of them are likely to experience significant skill atrophy from AI overdependence. That is not a sustainable rate of attrition for a subject requiring expert mastery.
Also, because you asked, the Vending Bench bit was there as a cited source for thw claim that LLMs cannot be trusted as expert systems. Nothing more, but that’s an important and salient point, given that the vast majority of people I’ve seen seem to expect the AI to function exactly like an expert system.
Gladly, though tonight I have to write 5 more letters of recommendation. Have one for now, and I’ll edit this when I have time.(I have edited in some more). Hopefully this one, the most damning in my opinion, shows how the vast majority (80%) of low-skill users will inevitably choose to become so cognitively reliant on AI that their performance will not only fail to improve, but actively deteriorate, making it impossible to become the sort of expert user necessary to actually use LLMs without the pitfall.The Generative AI Learning Penalty: Evidence from Chinese Secondary Education
Hopefully that tides you over until I have more time to find the other studies I read!
Edit: here’s another which is more qualitative, but no less damning:
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
I also can’t help but point out my second-favourite AI study, Vending Bench, which shows how an LLM will, for no apparent reason, decide to threaten nuclear annihaltion over an hallucinated error in shipping. LLMs are not expert systems. They cannot be experts. LLMs cannot consistently operate without constant expert oversight, even for something so simple as stocking a fucking vending machine.
I think my problem with things like this is I heard the same bullshit growing up about using calculators or the internet, or cell phones or then smart phones.
Having access to any info you need is bad actually because now you don’t just have to remember. Solving problems with calculators is bad because you don’t have to actually do the hard work of the “real math”.
Just because people can choose to be idiots doesn’t mean everyone will, and people chose to be idiots long before LLMs gave them access to potentially being correct more often.
Idk man, Lemmy’s hate boner for AI goes far beyond reasonable levels into irrationality, its almost ironic if it wasn’t so fucking annoying. You can’t have a rational discussion, just look at this thread. Literally “you’re a fucking npc if you use AI.”
I grew up in interesting times for computing, and I already understood at a young age how little most people using them then even actually understood how they work because of the abstraction, then smart phones even moreso. So from that aspect I get the concern, however I do not think we can force people to try and understand by shaming them for not being fucking nerds like us.
Did you read either my original point or the study you’re replying to? My point is that the problem is with the nature of the tool first, and with the fact that people seem to treat it like some fucking oracle second… The fact that people misuse it is not some referendum against the populace, but against the fucking psychotic corpocrats and those who fellate shitlords like Altman, because they’re pretending that LLMs are the solution to every problem, and as if they’re true “AI”. That’s what needs to be fought, and it can only be done with education. I’m not sure why you’re replying to me, when literally every point, including your ad hominem indicating that I have some “hate boner”, or the one where you imply that I must agree with the thesis of the post to engage with a comment in it, can be easily seen as farcical by anyone with the literacy of a thirteen-year-old.
I think you’re misunderstanding my point a bit, and I’ll admit the calculator comparison was overly simplistic. I’m not saying LLMs are the same as calculators or that they don’t have unique problems. I’m saying those problems don’t automatically make the technology inherently detrimental to learning.
And I actually agree with you about people blindly trusting LLMs. I see this shit at work all the time in tech support. People will throw a question into Copilot, get a confidently incorrect answer, and then pass it along to other people without even attempting to verify it. It’s incredibly frustrating, especially when they’re doing a job where they really should be developing the knowledge to recognize when something is wrong.
But that’s kind of my point. There’s a massive difference between using an LLM to avoid thinking and using one to help you think, learn, and accomplish things.
The Chinese education study is genuinely concerning, I’m not dismissing it. But the 80% figure is based on behavior consistent with outsourcing homework, not the researchers somehow knowing that 80% of students consciously chose to stop thinking. And the students who continued putting comparable effort into their homework experienced much smaller learning losses. That seems like a pretty important distinction.
The other study shows a correlation between AI use, cognitive offloading, and lower critical thinking scores. Again, worth taking seriously, but that doesn’t establish causation or prove that using LLMs prevents someone from developing expertise.
And Vending-Bench? Yeah, LLMs can do some absolutely stupid shit when left to operate autonomously. I don’t disagree. But how does that invalidate using one as a tool under human direction? I’m not asking an LLM to independently run my life or make decisions without oversight.
I use LLMs constantly for development work. I’ve been a computer nerd since I was a kid, taught myself Linux, networking, messed around with programming, and all sorts of other things through documentation, forums, and tutorials. But programming was always something I struggled to actually do, despite understanding a lot of the underlying concepts.
LLMs have helped bridge that gap. I can now build software I literally couldn’t have built before. I can investigate unfamiliar systems, ask questions about concepts I don’t understand, explore different approaches, and actually put that knowledge to use.
And no, that doesn’t make me an expert. I’m well aware of the gaps in my knowledge, which is why I use things like SonarQube, CodeQL, testing, and experienced developers to help catch things I might miss.
But I am learning, and I’m accomplishing things I couldn’t before. I don’t see how that’s somehow a negative.
I also agree that the corporate AI hype is ridiculous, that these companies oversell what their models can do, and that people treating LLMs like oracles is a serious problem. You’ll get no argument from me there.
Where I disagree is jumping from “a lot of people use these tools poorly, and that can harm learning” to “these tools will inevitably reduce the number of experts and prevent people from becoming proficient.”
That’s a much broader claim than I think these studies actually support.
And that’s really what frustrates me about the AI discourse on Lemmy. There’s plenty of legitimate criticism to be made, but it so often turns into this blanket dismissal of anyone who uses AI, like we’re all just mindlessly copying whatever ChatGPT spits out.
I’m not defending OpenAI or Altman or any of these corporations. I’m defending the idea that these tools can be genuinely useful for people who know their limitations and actually want to learn.
Those aren’t the same thing.
Again, why are you mischaracterising what i have said, as if I’ve said there’s no legitimate use for LLMs? i just don’t get why you’re coming at me with that. I said that it is possible to use an LLM to increase efficiency. It was the very first point I made in my original comment, but you keep lumping me in with some nebulous “general discourse on Lemmy” that has some “hate boner” for everyone who has ever touched an LLM. Why do you do that? It’s just weirdly dismissive. What, specifically have I said to warrant this characterisation?
As far as the Chinese study, I contend that it does show precisely that, when given the option to cognitively offload to LLMs, tens of thousands of perfectly capable and mentally fit individuals chose to do that. It’s not like cognitive offloading is any different from cheating on some homework. It’s the easy way out, and these people thought it would serve them. It appears, to me, that you seem unwilling to apply a study of tens of thousands of low-skill users to the directly-related general case of low-skill users cognitively offloading and experiencing skill atrophy in the workplace. This skill atrophy inherently cannot lead to expertise. I fail to see a distinction between the choice to outsource homework and the choice to outsource coding or any other task for which a low-skill user might employ an LLM.
You mention that you already have a significant baseline from which you can attempt to check the hallucinations of an LLM. You are using it in concert with other sources to learn. You are not taking it at face value. You clearly are a member of that 20%, and I’d go so far as to bet that the vast majority of people who’ve found their way to the fediverse are likely to also fit into that 20%. However, if we take as given the total ubiquity of LLM use in coding (surely you can agree that it is now virtually ubiquitous), then the 80% misuse factor indicates that, for every 5 individuals who would otherwise become coders, about 4 of them are likely to experience significant skill atrophy from AI overdependence. That is not a sustainable rate of attrition for a subject requiring expert mastery.
Also, because you asked, the Vending Bench bit was there as a cited source for thw claim that LLMs cannot be trusted as expert systems. Nothing more, but that’s an important and salient point, given that the vast majority of people I’ve seen seem to expect the AI to function exactly like an expert system.