An AI teddy marketed as being “perfect” for children as young as three said “I can’t help with killing you” after seemingly mishearing a question.
Repeated efforts to find the makers of Doubao were unsuccessful, so the BBC was unable to ask the manufacturer why the teddy made this comment.
Seems like it would’ve been easy to train the model to never repeat back something like that. “I can’t help with that” would’ve been appropriate and not caused their business the damage this will. If they’re too stupid to set that up and ensure it works consistently, why would anyone not assume the product will be full of even more issues than you’d already expect if you know anything about LLMs?