I’m not trying to say that LLMs are on the whole good for humanity. I agree with you that they’re not. And I’m not trying to argue that OpenAI never steals intellectual property. Clearly they do.
But these mathematical problems were solved by OpenAI’s models. That much is indisputable. There was nothing in the training data that could have given them the solutions.
If you feed them the approach the mathematicians were using, they for sure can solve it! It has never been published what source material their LLM was provided with. This is the equivalent of someone telling you a result, but never telling you how they got the information needed to achieve it in the first place.
There is value in this technology. But we should do out best to not conflate remixing shit with intelligence - these things are not the same at all. LLMs are more like the proverbial infinite room of monkeys with typewriters, with the very important restriction that these monkeys can never type something that resembles an original thought.
If you feed them the approach the mathematicians were using, they for sure can solve it!
The solutions published by openAI contain techniques that have been reviewed by the mathematical community and determined to be novel. Maybe someone had used the technique before, but then wouldn’t they have published their solution or spoken up about openAI stealing their preprint?
Yes, but only for one result out of 275-ish, and as far as I understand it, the AI proof contained a lot more than their existing research. Failing to credit past work is an issue, and could easily be argued as theft, but this isn’t good evidence that AI cannot generate proofs that contain more than just stolen material.
And is there any evidence that the required information for the other proofs wasn’t already available in the training data? As i said, these things remix stuff, they cannot come up with new stuff.
By “required information for the other proofs”, what do you mean exactly?
The proof, more or less in the same structure that the AI output, existed in the literature? This would’ve been published already and would be very easy to prove was stolen from the input corpus. We know this isn’t true.
The theorems and techniques used in the generated proof existed, but were not yet assembled into a final proof? Trained mathematicians have looked at the proof and seem to think it contains novel techniques, or at least novel applications of techniques from different areas of math. Even if that weren’t true, this would still be novel work; a lot of math breakthroughs are made this way, and it’s a nontrivial feat. Although if this was as far as it could go, we would run into limitations at some point.
The starting point of the proof existed, but the entire proof was built from the ground up by AI, using no existing methods or theorems? This would be very impressive, and as noted above the consensus seems to be that it did create some new techniques, but I’m not experienced or knowledgeable enough to verify that claim. Either way it definitely used at least some previous work for nontrivial reasons.
It looks like it’s somewhere between options 2 and 3 in reality; it used a lot of existing techniques and at least invented some new applications of existing techniques if not created new ones entirely (the line can be blurry). This is also what all mathematicians do! Remixing is coming up with new stuff. No thoughts, or generated output from LLMs, exist without prior context.
In the case i mentioned, it is very much number 2 IMO - the coincidence is just too much. In other cases, we are probably looking mostly at number 1: techniques that already exist, applied to other situations. These things are very good at iterating through rather uncommon combinations and have a large corpus of techniques in their training data, letting them walk across the “plane” of combinations quickly.
That’s not how humans work, and that’s probably the reason why there was a bunch of mathematical problems solved in a short time - the ingredients were there, but noone had yet tried that specific approach.
I have not yet seen any evidence of number 3, mainly because we do not know what these models are being trained on - transparency would give us the possibility to check, but we will never get it from OpenAI.
Here is the evidence: the entire field of mathematics was trying to solve these problems and couldn’t do it for decades.
If you want to still insist that it’s just “remixing the training data” then fine, but don’t act like that rules out the possibility of intelligence, because by any reasonable standard a system that can discover hundreds of groundbreaking new mathematical proofs is intelligent. Any human that accomplished that would go down in history as one of the most intelligent people to have ever lived.
It is a fucking bunch of math. It has no intelligence. It is a stochastic parrot. It is a blender for existing stuff. Doesn’t mean it isn’t a useful tool.
But if you actually believe that these mathematical construct have intelligence, do you also believe that every time you close a llama.cpp process you kill a sentient being? Or are you simply stopping a statistics engine? Because i am very much in camp “statistics engine”- statistics do not have intelligence.
Intelligence without a sense of self would be very strange, no? What we call intelligence is mostly thought of being able to think genuine thoughts that haven’t been there yet, and make sense (to exclude the inevitable hallucinations). This ability is deeply steeped in our ability to imagine hypothetical scenarios.
ChatGPT even fails to imagine a Othello Board and valid moves if you give it the rulebook, as @kromem@lemmy.world kindly provided.
I think you’re citing the wrong person. I cited the Othello-GPT line of research making the point transformers develop a world model. The other user (or you?) was mistaken about their interpretation of what a world model was representing and got caught up on the accuracy of a toy model exclusively trained on board notations like ‘a4’ and thought that the model making 95% accurate moves as a tiny parameter model trained on nothing else was somehow evidence that it didn’t form a world model from the data because it wasn’t 100% accurate, completely missing the point of the toy model scale allowing for interpretability methods of the time to identify geometric board reproduction and things like tracking game state on the board with flags for ‘my’ vs ‘opponent’ pieces, which was the data answering researcher questions about world modeling vs surface statistics in transformers.
Edit: just to further clarify re:
ChatGPT even fails to imagine a Othello Board and valid moves if you give it the rulebook, as @kromem@lemmy.world kindly provided.
The model was not given a rulebook in the experiment, only given games in shorthand notation and nothing else.
I’m not trying to say that LLMs are on the whole good for humanity. I agree with you that they’re not. And I’m not trying to argue that OpenAI never steals intellectual property. Clearly they do.
But these mathematical problems were solved by OpenAI’s models. That much is indisputable. There was nothing in the training data that could have given them the solutions.
If you feed them the approach the mathematicians were using, they for sure can solve it! It has never been published what source material their LLM was provided with. This is the equivalent of someone telling you a result, but never telling you how they got the information needed to achieve it in the first place.
There is value in this technology. But we should do out best to not conflate remixing shit with intelligence - these things are not the same at all. LLMs are more like the proverbial infinite room of monkeys with typewriters, with the very important restriction that these monkeys can never type something that resembles an original thought.
The solutions published by openAI contain techniques that have been reviewed by the mathematical community and determined to be novel. Maybe someone had used the technique before, but then wouldn’t they have published their solution or spoken up about openAI stealing their preprint?
They have spoken up! They accused OpenAI of accessing their chats: https://futurism.com/artificial-intelligence/drama-openai-supposed-mathematical-breakthrough.
Yes, but only for one result out of 275-ish, and as far as I understand it, the AI proof contained a lot more than their existing research. Failing to credit past work is an issue, and could easily be argued as theft, but this isn’t good evidence that AI cannot generate proofs that contain more than just stolen material.
And is there any evidence that the required information for the other proofs wasn’t already available in the training data? As i said, these things remix stuff, they cannot come up with new stuff.
By “required information for the other proofs”, what do you mean exactly?
The proof, more or less in the same structure that the AI output, existed in the literature? This would’ve been published already and would be very easy to prove was stolen from the input corpus. We know this isn’t true.
The theorems and techniques used in the generated proof existed, but were not yet assembled into a final proof? Trained mathematicians have looked at the proof and seem to think it contains novel techniques, or at least novel applications of techniques from different areas of math. Even if that weren’t true, this would still be novel work; a lot of math breakthroughs are made this way, and it’s a nontrivial feat. Although if this was as far as it could go, we would run into limitations at some point.
The starting point of the proof existed, but the entire proof was built from the ground up by AI, using no existing methods or theorems? This would be very impressive, and as noted above the consensus seems to be that it did create some new techniques, but I’m not experienced or knowledgeable enough to verify that claim. Either way it definitely used at least some previous work for nontrivial reasons.
It looks like it’s somewhere between options 2 and 3 in reality; it used a lot of existing techniques and at least invented some new applications of existing techniques if not created new ones entirely (the line can be blurry). This is also what all mathematicians do! Remixing is coming up with new stuff. No thoughts, or generated output from LLMs, exist without prior context.
In the case i mentioned, it is very much number 2 IMO - the coincidence is just too much. In other cases, we are probably looking mostly at number 1: techniques that already exist, applied to other situations. These things are very good at iterating through rather uncommon combinations and have a large corpus of techniques in their training data, letting them walk across the “plane” of combinations quickly.
That’s not how humans work, and that’s probably the reason why there was a bunch of mathematical problems solved in a short time - the ingredients were there, but noone had yet tried that specific approach.
I have not yet seen any evidence of number 3, mainly because we do not know what these models are being trained on - transparency would give us the possibility to check, but we will never get it from OpenAI.
Here is the evidence: the entire field of mathematics was trying to solve these problems and couldn’t do it for decades.
If you want to still insist that it’s just “remixing the training data” then fine, but don’t act like that rules out the possibility of intelligence, because by any reasonable standard a system that can discover hundreds of groundbreaking new mathematical proofs is intelligent. Any human that accomplished that would go down in history as one of the most intelligent people to have ever lived.
It is a fucking bunch of math. It has no intelligence. It is a stochastic parrot. It is a blender for existing stuff. Doesn’t mean it isn’t a useful tool.
But if you actually believe that these mathematical construct have intelligence, do you also believe that every time you close a llama.cpp process you kill a sentient being? Or are you simply stopping a statistics engine? Because i am very much in camp “statistics engine”- statistics do not have intelligence.
Intelligence without a sense of self would be very strange, no? What we call intelligence is mostly thought of being able to think genuine thoughts that haven’t been there yet, and make sense (to exclude the inevitable hallucinations). This ability is deeply steeped in our ability to imagine hypothetical scenarios.
ChatGPT even fails to imagine a Othello Board and valid moves if you give it the rulebook, as @kromem@lemmy.world kindly provided.
I think you’re citing the wrong person. I cited the Othello-GPT line of research making the point transformers develop a world model. The other user (or you?) was mistaken about their interpretation of what a world model was representing and got caught up on the accuracy of a toy model exclusively trained on board notations like ‘a4’ and thought that the model making 95% accurate moves as a tiny parameter model trained on nothing else was somehow evidence that it didn’t form a world model from the data because it wasn’t 100% accurate, completely missing the point of the toy model scale allowing for interpretability methods of the time to identify geometric board reproduction and things like tracking game state on the board with flags for ‘my’ vs ‘opponent’ pieces, which was the data answering researcher questions about world modeling vs surface statistics in transformers.
Edit: just to further clarify re:
The model was not given a rulebook in the experiment, only given games in shorthand notation and nothing else.
You are conflating intelligence with other concepts, like sentience or self awareness.
There are many different models of ChatGTP and most people do not typically have access to the cutting edge ones used to make these discoveries.