Character.AI, an artificial intelligence chat platform, is banning under-18 users due to the negative effects of chatbots on teenage mental health, which were alleged in a September lawsuit filed by concerned parents, some of whom have lost children to suicide after engagement with the platform.
The lawsuit, filed against Character.AI’s parent company Character Technologies, alleges the platform, is “causing serious harms to youth causing serious harms to thousands of kids, including suicide, self-mutilation, sexual solicitation, isolation, depression, anxiety, and harm towards others.”
In response to the backlash and accusations addressed in the lawsuit, Character.AI announced Thursday it will make certain changes to its platform concerning teen use.
“We have seen recent news reports raising questions, and have received questions from regulators, about the content teens may encounter when chatting with AI and about how open-ended AI chat in general might affect teens, even when content controls work perfectly,” the AI company wrote. “As part of this, we are creating a fundamentally different experience for teen users from what is available to adults. This includes a model specifically for teens that reduces the likelihood of encountering sensitive or suggestive content while preserving their ability to use the platform.”
Character.AI will finalize the ban on November 25. It aims to re-invent its under-18 experience, declaring certain initiatives for that end.
“This change will take effect no later than November 25. Between now and then, we will be working to build an under-18 experience that still gives our teen users ways to be creative – for example, by creating videos, stories, and streams with Characters,” Character.AI said.
The platform will also roll out an AI Safety Lab, described as “an independent non-profit dedicated to innovating safety alignment for next-generation AI entertainment features.”
Character.AI’s change comes as the Federal Trade commission conducts a study on the effects of AI chatbots on the mental health of children.
