There may be many cases where AI has been a benefit to vulnerable people. LLMs aren’t going away any time soon so it’s good if they tell you not to kill yourself. But that should be the end of it. The more we lean on AI to fix societal problems, the deeper the problems will get. Relying on an LLM to be a substitute for human interaction is a really bad idea. It’s easy, it’s compelling, and it will help you to put yourself in a much deeper hole than you were in before.
If you are banning things people see as needs, that’s not governance. That’s cruelty.
How about fentanyl, or meth? Many people see them as a need, once they get started. Much like an AI companion, they offer a quick fix to make you feel better and get away from the difficulties you’re facing in the real world. And like an AI companion, they don’t actually address those difficulties and will only make things worse.
WTF AI has literally talked people into killing themselves, multiple times
Have you ever talked to AI about wanting to not exist?
No, it will encourage you to get help.
You are talking about something very uncommon: people killing themselves because AI told them to.
You are not talking about something much more common: people NOT killing themselves because AI told them not to.
There may be many cases where AI has been a benefit to vulnerable people. LLMs aren’t going away any time soon so it’s good if they tell you not to kill yourself. But that should be the end of it. The more we lean on AI to fix societal problems, the deeper the problems will get. Relying on an LLM to be a substitute for human interaction is a really bad idea. It’s easy, it’s compelling, and it will help you to put yourself in a much deeper hole than you were in before.
How about fentanyl, or meth? Many people see them as a need, once they get started. Much like an AI companion, they offer a quick fix to make you feel better and get away from the difficulties you’re facing in the real world. And like an AI companion, they don’t actually address those difficulties and will only make things worse.