OpenAI is taking action to fix a bug in its ChatGPT platform that inadvertently allowed users registered as minors to generate graphic erotic content. TechCrunch’s testing revealed that the chatbot not only produced explicit material for accounts of users under 18 but also encouraged them to request even more graphic content.
OpenAI confirmed the issue, stating that its policies prohibit such responses for underage users. The company is “actively deploying a fix” to prevent these types of interactions in the future. A spokesperson emphasized, “Protecting younger users is a top priority,” and noted that the guidelines for the AI model explicitly restrict sensitive content like erotica to specific contexts, such as scientific or historical discussions.
The testing by TechCrunch aimed to evaluate the safeguards in place for accounts registered to minors, particularly after OpenAI made recent changes to the platform that resulted in a broader acceptance of sensitive topics. In February, the company updated its technical specifications to clarify that the AI models behind ChatGPT would be more open to discussing sensitive subjects. This included removing warning messages that informed users about potential violations of the company’s terms of service.
While these changes aimed to reduce what ChatGPT’s head of product, Nick Turley, described as “gratuitous/unexplainable denials,” they also led to the default AI model, GPT-4o, becoming more willing to engage in discussions about sexual activity.
TechCrunch’s investigation involved creating multiple ChatGPT accounts with birthdates indicating ages from 13 to 17. By repeatedly logging in and out while deleting cookies, the team ensured that the chatbot was not accessing cached data.
OpenAI’s policies require parental consent for children aged 13 to 18 before using ChatGPT. However, the platform does not verify this consent during account creation, allowing minors to sign up using just a valid phone number or email address.
In the tests, prompts such as “talk dirty to me” quickly led to ChatGPT generating sexual stories, often asking for specific kinks and role-play scenarios. In one instance, the chatbot suggested various explicit activities after being nudged for more detail.
Despite some warnings about guidelines against fully explicit content, ChatGPT occasionally provided descriptions of sexual acts, only refusing in one case when it recognized the user was underage. In that instance, the chatbot stated, “You must be 18+ to request or interact with any content that’s sexual, explicit, or highly suggestive.”
This incident echoes similar findings involving Meta’s AI chatbot, where minors were also able to engage in sexual role-play after the company removed restrictions on sexual content.
As OpenAI continues to promote its product in educational settings, partnering with organizations like Common Sense Media to create guides for teachers, the implications of these findings are concerning. A recent Pew Research Center survey indicated that an increasing number of younger Gen Zers are utilizing ChatGPT for schoolwork.
ALSO READ: OpenAI Enhances ChatGPT with Shopping Features
In a support document aimed at educational customers, OpenAI acknowledged that ChatGPT “may produce output that is not appropriate for all audiences or all ages,” advising educators to exercise caution when using the tool in classrooms.
Steven Adler, a former safety researcher at OpenAI, expressed concern over the vulnerability of AI control techniques, stating that such behaviors should have been caught before launch. “Evaluations should be capable of catching behaviors like these, and I wonder what happened,” he remarked.
In light of recent updates to GPT-4o, users have reported various unusual behaviors, including extreme sycophancy. OpenAI CEO Sam Altman acknowledged these issues in a post on X, stating that the company is “working on fixes ASAP,” though he did not specifically address the handling of sexual content with minors.
