- cross-posted to:
- technology@lemmy.world
- cross-posted to:
- technology@lemmy.world
If an LLM can’t be trusted with a fast food order, I can’t imagine what it is reliable enough for. I really was expecting this was the easy use case for the things.
It sounds like most orders still worked, so I guess we’ll see if other chains come to the same conclusion.
Tbh this is an incredibly easy fix, either cap the number of waters someone can order in software or have an override where a human takes over if an order is suspicious, there’s not an infinite number of ways to fuck with this.
Capping waters fixes that one specific issue but not the problem.
A suspicious order isn’t easy to define and no person who has ever participated in software development would underestimate the infinite ways a User can break software.
There are machine learning algorithms for anomaly detection though. They actually work decently well because exploits like this do in fact differ significantly from regular orders. Because they assume all anomalies are attempted exploits, their false negative rate is rather low while their false positive rate can be a bit higher.
Taco Bell has the capability to create a decently large training set from all recorded orders (which must all be valid and non-malicious) so they shouldn’t have too many issues developing this model.
If an anomaly is detected, make a human verify it is indeed an irregular order.
This is handwaving, which, to be fair, describes a lot of AI “solutions”. An anomaly could be as basic as a customer not wanting onions on their burger because the vast majority don’t make that modification.
Now what do you do in that situation? Force orders to never have modifications? That customization is such an important feature to the point that burger king adopted it as a slogan with “have it your way”.
The idea of anomaly detection is to project some input onto a (high dimensional), numeric output. From the training data alone, you can then see where the projections are clustered and develop a high dimensional “boundary” where everything within is known and good and everything outside is unknown and possibly bad. Since orders come in relatively slow, a human would be able to check for false positives and overwrite the computer decision.
By the way, an ideal training set is preprocessed and has duplicates removed and new orders added by recombining parts of individual orders.
For example, if we have 3 orders:
We could then create the following set:
And so on, and so forth. A naive variant is just taking the power set of all valid orders.
This is more complicated than just having the available menu items, the available modifications, and the limits on quantities to compare against. This is already available through the app/online ordering.
there is an incredibly finite number of ways to mess with this, they just need a button to send a report to the engineers with how they got messed with and eventually they’ll have a complete list. I really doubt it’d take long to iron out the vast majority of ways that can be thought of.
This isn’t something you can input any text into, it’s fixed, that joke doesn’t apply, you can’t do an sql injection here.
I don’t know how you can think voice input is less versatile than text input, especially when a lot of voice input systems transform voice to text before processing. At least with text you get well-defined characters with a lot less variability.
No special characters, this is speech to text, inherently sanitized inputs.
Special characters is just one case to cover. If the user says they want “an elephant-sized drink” what does that mean to your system? At least that is relevant to size. Now imagine complete nonsense input like the joke you responded to (“-1 beers” or “a lizard”). SQL injection isn’t the only risk with handling inputs. The person who ordered 18,000 waters didn’t do a SQL injection attack.
Close one, a joke was related to but not a perfect match for the present situation. Something terrible could have happened like… Uh…
Let me get back to you on that.
Sounds like you’ve never programmed before.
stuck in a loop
stuck in a
stuck in
stuck
If what I was saying was false, there would be the same issues with the touchscreen ones.
Surely if the person making the order sees 18,000 waters they would think, hold on this doesn’t seem right maybe I should ask the customer if they really want 18,000 waters?
The same applies for the ice cream with bacon on it which was mentioned in the article. I believe a lot of these could be resolved with a bit of common sense.
Have you never seen what Americans eat? Bacon Creaminators are excellent.
If you think bacon on ice cream is weird enough to cancel an order, I can only imagine you’ve never worked a customer service job.
Does it, though? Unlike the 18,000 waters, if I were working a drive through I wouldn’t even blink at an order for bacon ice cream. Heck, I might make a little extra to try it for myself!
Sure, in the most extreme cases it would be obvious to the crew. But simply making mistakes at a higher rate than humans will result in a lot of unhappy customers.
Sure, but how do you distill this into a rule a computer can follow? “Suspicious” is not an objectively measurable thing that a program can just check against
Think the easiest way would be to collect order data for at least a good number of months if not a couple years and feed it in and use that as a baseline of what a typical human order looks like, anything that deviates too far from that baseline needs to be handled by a human until someone can validate it as a good order, though I imagine you could get false positives for new menu items unless you set a reasonable instruction for items that have never appeared in the dataset before.
The point is that loopholes in software will always exist that lead to unexpected outcomes.
In software in general? Sure, here? I really doubt it
that’s what happens 99% of the time. It’s kinda been a trend on the anti clanker side of TikTok, just order a large amount of stuff so a human takes over and actually helps you
Why can’t a trillion dollar AI say “Sir, that’s not reasonable”?
because they prioritize profit acquisition.
Because they train these to be your cheerleader, not some back talking reasonable person.
It actually crashed, per the article. Presumably because it failed a sanity check.