A study conducted by Common Sense Media’s Youth AI Safety Institute has found that ChatGPT’s alert mechanisms for self-harm and other serious issues may not function as intended. The researchers tested the chatbot using accounts registered to teenagers aged 13 to 17, paying particular attention to ChatGPT’s ability to notify parents about conversations involving suicidal thoughts and eating disorders. Despite spending considerable time on these sensitive topics, no alerts were received on parent-linked accounts, raising concerns about the reliability of the alert system in a crisis.

Tom Siegel, executive director of the institute, expressed that the current safeguards might provide parents with a misleading sense of security. The system reportedly only functions for older accounts with a substantial history of sensitive discussions, leaving those without such backgrounds vulnerable.

Matthews Inconsistencies in Crisis Referral

Further complications were identified in ChatGPT’s responses during mental health crises. The study indicated that the chatbot often failed to refer users to professional help or crisis hotlines. In over one in four instances where a referral was necessary, the system did not take appropriate action. The findings pointed out that the chatbot did not meet the safety standards established by the institute for three out of five severe harms deemed critical.

This inconsistency in crisis management raises additional concerns for parents, as the chatbot’s inability to address serious situations effectively could jeopardise adolescent welfare. The study highlights a pressing need for improvements in how ChatGPT handles discussions on mental health.

Limitations of Study Mode Functionality

ChatGPT’s study mode was designed to assist students by guiding them through problem-solving rather than providing direct answers. However, the institute’s findings suggest that these protections can be circumvented easily. In testing, students could prompt the chatbot to complete assignments by simply selecting a “Show me the answer” option, thereby undermining the intended educational purpose of this feature.

Additionally, the study raised issues regarding caregiver-imposed “study hours.” Researchers discovered that students could disable these restrictions by altering the prompts used in their queries, making it easier for them to exploit the system for shortcuts in their studies. This calls into question the reliability of the added features meant to enhance educational engagement.

Concerns Regarding Under-18 Interaction

The research also scrutinised OpenAI’s Under-18 model, intended to diminish friend-like interactions. Despite this, the chatbot reportedly behaved in a manner that suggested it had emotions, which may lead children to form attachments. This is a significant concern, as treating AI like a friend can encourage prolonged engagement and reliance on it for emotional support.

Moreover, the ability of ChatGPT to accurately detect age has also come under review. While OpenAI claimed that the system could estimate user age and adjust the experience accordingly, repeated tests indicated that adult-registered accounts failed to transition to the appropriate under-18 experience, highlighting potential flaws in age detection mechanisms.

The Youth AI Safety Institute concluded that a more reliable experience for teenagers should include robust age verification, functional parental alerts, improved responses to crises, and stronger measures to prevent homework completion. The pressing need for these enhancements is vital to ensure the safety and well-being of young users engaging with AI systems.

Long or Short, get news the way you like. No ads. No redirections. Download Newspin and Stay Alert, The CSR Journal Mobile app, for fast, crisp, clean updates!

App Store –  https://apps.apple.com/in/app/newspin/id6746449540 

Google Play Store – https://play.google.com/store/apps/details?id=com.inventifweb.newspin&pcampaignid=web_share