cross-posted from: https://discuss.online/post/41098657
http://www.smbc-comics.com/comic/huh-2
Alt text
Once I realized this, all those inept AI laundry-folding videos became hilarious.
Bonus panel
cross-posted from: https://discuss.online/post/41098657
http://www.smbc-comics.com/comic/huh-2
Alt text
Once I realized this, all those inept AI laundry-folding videos became hilarious.
Bonus panel
It can’t do math.
Anyone who thinks it can has never asked it to do math.
Most big LLMs will pass the math off to a more typical service that will solve the math problem deterministically and then pass the result back to the LLM to include in its response.
So yeah LLMs can’tactually do math but from a user perspective they can.
It can’t do art or writing either. It’s like a genius suffering from the advanced stages of Mad Cow Disease. You have to badger it for hours and lower your expectations before it produces something acceptable.
It can’t even count.
AI loves to present information in fancy formats like tables and lists, but it can’t even enumerate a table’s rows reliably.
It can do maths. It cannot do arithmetic. And actually nowadays it seems ok at arithmetic. (It farms arithmetic out to dedicated, non-language based sub agents)
Two weeks ago I was testing a draft of a puzzle to see how ai would do on it, with a page of about fifteen maths problems ranging from basic arithmetic up to easy integrals. It got all but one correct. (So that puzzle needs adjusting…)
I think people who say this remember how it couldn’t count letters three years ago and think that’s the end of the story.
It still can’t to math or arithmetic, they have it set up so it will use tools like Wolfram or just straight python for calculation.
By their very nature they suck ass at math
It can do it better than I thought. Normally it’d spin Python, as you said, so I tried asking it to not use tools:
You:
Is there an echo in here?
Even if I interpret you charitably, you are using the word “it” to mean “LLMs”, instead of “AI”. That isn’t valid or useful. Nobody cares very much whether specific classes of neural network are good at maths; they care whether ChatGPT is going to give them the right answer. Go waste some of OpenAI’s money right now by asking it a question on mathematics, and see whether “AI can do maths”.
This gets even more ludicrous when you look at what AI is enabling in real mathematics, rather than in arithmetic.
What an unfortunate formatting error you’ve made. Add an extra line break after your quote blocks to separate them.
Huh, doesn’t render that way on my instance!
It’s a little computer of course it can do maths. Where it struggles is working out what math to do, but so do humans.
If I need to work out what 12 out of 294 is as a percentage I just Google it. I can never remember the formula, the actual mathematics is not complicated, it’s the formula I can’t remember, and it’s the same with the AI.
I get your point, I often search for equations, but really 12/294*100 is not so difficult to remember.
You’ve got two numbers and a division, putting the in the wrong order will give you a clearly wrong result.
The problem with LLMs is that they are not reliable at numerical calculations even if they got the correct equation. But that is now easily fixed by giving them access to computation software.