Meta has introduced new features to notify parents when their teenagers discuss suicide or self-harm with the company’s AI chatbot, Meta AI. Parents who have activated the supervision feature for their teens on Instagram will receive alerts on their own devices if a teenage user displays signs of crisis in conversations with Meta AI on any Meta platform. These platforms include Instagram, WhatsApp, Facebook Messenger, and the Meta.ai website.
This enhancement expands Meta’s existing safety measures. Previously, when a teenager indicated thoughts of self-harm or suicide, the AI chatbot would direct them to crisis helplines and suggest contacting a parent or another trusted individual. Moreover, if teen users on Instagram repeatedly search for content related to self-harm and suicide using the regular search function, it automatically triggers notifications to parents.
The parent company of Instagram and Facebook, Meta, has made this new feature available to users in Canada, the U.K., the U.S., and Australia. Meta initially announced the development of a flagging feature for Meta AI in February.
Additionally, Meta is developing a tool to alert emergency services when conversations with the chatbot, involving both adults and teens, indicate potential self-harm. While stakeholders appreciate the initiative, they emphasize the need for further improvements.
Experts stress that this tool is just one method to assist parents in monitoring their children’s online activities. Despite its benefits, they caution that it is not foolproof and may produce false positives or fail to flag genuine risks. Meta plans to err on the side of caution initially, even if a teen’s intentions are unclear.
Furthermore, challenges may arise as young users may attempt to circumvent the system. Advocates call for stricter regulations to protect young individuals, suggesting that voluntary measures are insufficient. In line with this, Canada introduced the Safe Social Media Act in June, aiming to mandate safety requirements for AI chatbots in response to expressions of self-harm or suicide.
The introduction of Meta’s new tool coincides with increased scrutiny on AI’s role in mental health crises among youth. Instances where AI failed to alert authorities in critical situations have raised concerns. OpenAI, for instance, has faced legal action for its chatbot’s alleged involvement in tragic events.
In conclusion, while Meta’s new parental alert system is a step forward in safeguarding young users, experts and advocates underscore the need for comprehensive measures and regulatory frameworks to ensure online safety for children.