I get that hating on anything AI-related is trendy these days - and I especially understand the pain of a grieving mother. However, interpreting this as a chatbot encouraging someone to kill themselves is extremely dishonest when you actually look at the logs of what was said.

You can’t simultaneously argue that LLMs lack genuine understanding, empathy, and moral reasoning - and therefore shouldn’t be trusted - while also saying they should have understood that “coming home” was a reference to suicide. That’s holding it to a human-level standard of emotional awareness and contextual understanding while denying it the cognitive capacities that such standards assume.

“I promise I will come home to you. I love you so much, Dany,” Sewell Setzer III wrote to Daenerys, the Character AI chatbot named after Game of Thrones.

The bot replied that it loved the teenager too: “Please come home to me as soon as possible, my love.”

“What if I told you I could come home right now?” Sewell wrote, to which Daenerys responded: “Please do, my sweet king.”

It was the last exchange Sewell ever had. He took his own life seconds later…

Source