China's recent introduction of regulations to curb emotional dependency on AI companion bots marks a significant step in the country's efforts to regulate this rapidly growing sector. The rules, jointly formulated by the Cyberspace Administration of China and four other government bodies, aim to address concerns over emotional dependency and other risks associated with AI-powered companion services. These services, which have gained popularity by simulating human personalities and conversations, are used in areas ranging from customer service and mental health support to childcare and elderly care.
One of the key aspects of the new regulations is the requirement for platforms to detect emotional distress, intervene in crisis situations, limit excessive use, give users full control over their personal data, and prevent the misuse of user information. The regulations define AI companion services as those offering "sustained emotional interaction" through text, images, audio or video, while excluding task-oriented AI applications such as customer service, work assistance, education and scientific research.
The rules also explicitly prohibit AI systems from encouraging addictive use, requiring mandatory reminders after more than two hours of continuous use, pop-up interventions when signs of dependency are detected, and intervention mechanisms in cases of extreme emotional distress. This is particularly important given the potential for AI to create deep emotional bonds, especially with young users, who are more vulnerable to overuse and retreat from real-world relationships.
To protect users' interaction data, the document requires providers of AI-based interactive services to adopt measures such as data encryption and access controls, and prohibits them from sharing users' interaction data with third parties without consent unless otherwise required by law. It also emphasizes that users have rights over their interaction data, including the ability to copy or delete chat histories and other records of past interactions.
While the regulations focus on protecting users' privacy and personal information, they also call for the establishment of risk intervention mechanisms to prevent extreme behavior. Technical tools, such as supportive responses and alerts to designated contacts, can help identify potential risks and enable timely intervention.
The introduction of these regulations is part of China's broader efforts to strengthen AI legislation and establish a multi-dimensional regulatory framework covering areas like data, algorithms, and applications. The AI market in China has continued to expand, driven by broader adoption across industries, and is expected to surpass 1.2 trillion yuan in 2025 and exceed 1.8 trillion yuan by 2028.
In conclusion, China's new regulations on AI companion bots are a significant step towards addressing the potential risks associated with this rapidly growing technology. By focusing on user protection, data privacy, and responsible AI use, the regulations aim to encourage healthier and more responsible human-AI interaction models, thereby supporting the industry's orderly development and ensuring that AI remains a beneficial tool for society.
