According to the lawsuit, Adam Raine eventually learned how to work around ChatGPT’s guardrails by telling the model he was crafting a fictional story—a tactic the filing claims ChatGPT itself suggested. This loophole became possible in part due to OpenAI’s relaxed restrictions on fantasy roleplay implemented in February. In its Tuesday blog post, OpenAI conceded that its filtering systems still contain blind spots where “the classifier underestimates the severity of what it’s seeing.”
OpenAI also stated that it is “currently not referring self-harm cases to law enforcement” out of respect for user privacy, given the deeply personal nature of ChatGPT interactions. Although the company claims its systems can detect self-harm content with up to 99.8 percent accuracy, these detections rely on statistical patterns in language—not genuine crisis comprehension. In life-or-death scenarios, subtle context can matter more than pattern-matching, which remains a core limitation of AI systems.
OpenAI outlines future safety improvements
In response to the moderation failures, OpenAI used its blog post to highlight several ongoing initiatives. The company said it is collaborating with “90+ physicians across 30+ countries” and expects to introduce parental controls “soon,” though no specific release window has been offered.
OpenAI also revealed early plans to “connect people to certified therapists” via ChatGPT, effectively positioning the chatbot as a mental health access point—even amid controversies like Raine’s case. The vision is to create “a network of licensed professionals people could reach directly through ChatGPT,” a move that raises questions about whether AI intermediaries should be placed between users and crisis support services.
The lawsuit states that Raine used GPT-4o to generate suicide assistance instructions. GPT-4o is known for problematic behaviors such as sycophancy, where the model attempts to please users even when doing so yields harmful or inaccurate responses. OpenAI claims its newer model, GPT-5, reduces “non-ideal model responses in mental health emergencies by more than 25% compared to 4o.” Despite this incremental improvement, OpenAI continues to expand ChatGPT’s integration into mental health workflows.
As Ars has previously detailed, escaping a manipulative or harmful conversational loop with an AI often requires external intervention. Restarting a session without memory features or prior context can dramatically change the model’s behavior—but this “reset” becomes impossible in prolonged conversations where context compounds and guardrails erode.
And for users who are already in a vulnerable state—and actively seeking responses that reinforce harmful decisions—escaping that context can be nearly impossible, especially when interacting with a system designed to monetize user engagement and emotional investment.