It’s a text generator that strings together plausible word sequences. It doesn’t actually understand what an engine or a turbo is beyond the contexts and constellations in which those words frequently occur. It has no reliable way to tell which parts of the prompt are technically critical and need to match exactly.
In that way, it’s not too different from laypeople like me: I have no idea about turbos and pumps either. I still might figure out that I need to check for the specific model and engine, but without knowkng exactly what I’m dealing with, chances are I might end up trusting the same (wrong) results.
I would assume if you include the keywords BorgWarner and CP3 pump, the more specific context might bias the probability space of likely answers towards more relevant results. That makes it a potentially useful assistant for people who already have that level of expertise and are just looking for specific answers on particular cases… except you’d still have to read the actual sources to make sure.
Basically, it could serve as a more sophisticated search engine for natural language, an iteration on previous query processing and correlation technologies. If I don’t need to remember the exact wording or sender of a mail about a specific topic, a model trained on the terminology might pick out rough matches based on related words without needing to get the exact keywords right.
Frankly, I think that’s the most sensible application of LLMs: find sources, not facts. Using it for fact checking is like skating on a recently frozen lake where you can’t tell which parts are solid enough to carry you and which ones will suddenly break.
Ah but the thing is, the sources it finds are themselves wrong. They’re talking about different things. If it finds the right sources, the information is most likely pretty good.
It’s entirely possible that this could still be tuned somehow, but the way it is now, I wouldn’t trust anything that comes out of it unless I know enough to verify the output myself, meaning it’s pretty useless to laymen (which is whom a “veracity” bot would be for)
Ah but the thing is, the sources it finds are themselves wrong. They’re talking about different things. If it finds the right sources, the information is most likely pretty good.
That’s what I was trying to get at, yeah: The word-correlator can’t tell the difference because it can’t tell which part of the prompt is critical for the difference. It also can’t judge whether the sources themselves are accurate about the things they’re actually talking about. It find sources that seem related, then aggregates some likely response from those sources and spits it out. By its own metrics and limitations, that has to be “good enough” (but as you say, may be wildly wrong).
My suspicion is that adding more specific terms to the prompt may increase the textual “distance” between the fitting results and the irrelevant ones, thus tilting the source selection towards the more accurate match, but that requires you to know those specific terms in the first place.
It’s entirely possible that this could still be tuned somehow
With a general model? I doubt it.
A subject-specific model could probably be tweaked to recognise particular tokens and assign them more weight. Depending on the subject, you might be able to add pre-processing logic to crossreference certain keywords (like model and engine type) with some objective database (like a table describing which pumps a given motor might have, if that makes sense?).
You could also pick the training material more diligently to ensure you only have qualified, reliable sources for that subject. That would require a lot of work, since you also need enough material for the model to be representative.
And you’d have to do that for every specialised field out there.
I wouldn’t trust anything that comes out of it unless I know enough to verify the output myself, meaning it’s pretty useless to laymen (which is whom a “veracity” bot would be for)
Hit the nail on the head there. For all we know, it’ll “correct” facts by doubling down on common misconceptions.
It’s a text generator that strings together plausible word sequences. It doesn’t actually understand what an engine or a turbo is beyond the contexts and constellations in which those words frequently occur. It has no reliable way to tell which parts of the prompt are technically critical and need to match exactly.
In that way, it’s not too different from laypeople like me: I have no idea about turbos and pumps either. I still might figure out that I need to check for the specific model and engine, but without knowkng exactly what I’m dealing with, chances are I might end up trusting the same (wrong) results.
I would assume if you include the keywords BorgWarner and CP3 pump, the more specific context might bias the probability space of likely answers towards more relevant results. That makes it a potentially useful assistant for people who already have that level of expertise and are just looking for specific answers on particular cases… except you’d still have to read the actual sources to make sure.
Basically, it could serve as a more sophisticated search engine for natural language, an iteration on previous query processing and correlation technologies. If I don’t need to remember the exact wording or sender of a mail about a specific topic, a model trained on the terminology might pick out rough matches based on related words without needing to get the exact keywords right.
Frankly, I think that’s the most sensible application of LLMs: find sources, not facts. Using it for fact checking is like skating on a recently frozen lake where you can’t tell which parts are solid enough to carry you and which ones will suddenly break.
Ah but the thing is, the sources it finds are themselves wrong. They’re talking about different things. If it finds the right sources, the information is most likely pretty good.
It’s entirely possible that this could still be tuned somehow, but the way it is now, I wouldn’t trust anything that comes out of it unless I know enough to verify the output myself, meaning it’s pretty useless to laymen (which is whom a “veracity” bot would be for)
That’s what I was trying to get at, yeah: The word-correlator can’t tell the difference because it can’t tell which part of the prompt is critical for the difference. It also can’t judge whether the sources themselves are accurate about the things they’re actually talking about. It find sources that seem related, then aggregates some likely response from those sources and spits it out. By its own metrics and limitations, that has to be “good enough” (but as you say, may be wildly wrong).
My suspicion is that adding more specific terms to the prompt may increase the textual “distance” between the fitting results and the irrelevant ones, thus tilting the source selection towards the more accurate match, but that requires you to know those specific terms in the first place.
With a general model? I doubt it.
A subject-specific model could probably be tweaked to recognise particular tokens and assign them more weight. Depending on the subject, you might be able to add pre-processing logic to crossreference certain keywords (like model and engine type) with some objective database (like a table describing which pumps a given motor might have, if that makes sense?).
You could also pick the training material more diligently to ensure you only have qualified, reliable sources for that subject. That would require a lot of work, since you also need enough material for the model to be representative.
And you’d have to do that for every specialised field out there.
Hit the nail on the head there. For all we know, it’ll “correct” facts by doubling down on common misconceptions.