An AI teddy marketed as being “perfect” for children as young as three said “I can’t help with killing you” after seemingly mishearing a question.
Repeated efforts to find the makers of Doubao were unsuccessful, so the BBC was unable to ask the manufacturer why the teddy made this comment.
I think what they want is confirmation that there are system or model instructions that explicitly mention dangerous or adult topics and that priming any LLM with things it can’t say is only going to make sure these words and concepts are within its context when it invariably starts to hallucinate.
Look, parents are cautious about who they let interact with their kids. The truth is, you wouldn’t leave your kid to hang out with a rando. And an LLM isn’t even a rando. It doesn’t have the reasoning, the common sense or the decency of any random human being. You should not let your kid just talk to an LLM, no matter how well designed it is, no matter what safeguards are in place. That LLM is not a person you trust, it has no accountability.
That toy probably shouldn’t be on the market, but the parent definitely should not have bought it, that was the most significant example of poor judgment here.
I agree. I was on a bit of a different track, but letting your kids have access to an unaccountable statement generator is a horrible idea and should have been stopped at the board room it was suggested in, before it even made it to prototype