China's Cyber Regulator Cracks Down on AI Misuse, Removes 5.6 Million Items
China's cyberspace watchdog is turning up the heat on AI-related chaos. In the second phase of its "Clear and Bright" special campaign, the Cyberspace Administration of China (CAC) has removed over 5.61 million pieces of illegal and improper information, investigated 49,000 accounts, and taken action against more than 2,400 websites and apps.
The focus? Curbing the misuse of AI to spread false information, peddle violent or vulgar content, impersonate others, and harm minors. Local regulators have been told to prioritize these issues, and the results are already rolling in.
Platforms Step Up Their Game
Tech giants aren't sitting idle. Platforms like Douyin, Kuaishou, Weibo, Tencent, Baidu, Bilibili, Xiaohongshu, Zhihu, Douban, and Taobao have been upgrading their AI content governance. They've rolled out smarter multimodal recognition models, expanded their face and voiceprint libraries, and even introduced real-time face governance systems. So far, they've issued 46 governance announcements.
Meanwhile, AI service providers—think Tongyi, Yuanbao, Qianwen, and Wenyiyan—are tightening their own pipelines. They're reviewing training data more strictly, limiting the output of harmful content, and making sure AI-generated content is clearly labeled. App stores, including Huawei, Xiaomi, OPPO, and Vivo, have also raised the bar for developers, with stricter entry checks and spot audits on existing apps.
Seven Cases That Raised Eyebrows
The campaign has also exposed some pretty wild examples of AI gone wrong. Here are seven typical cases:
Classic works get "remixed": Some users used AI to twist classics like Romance of the Three Kingdoms and Journey to the West, turning beloved characters into clickbait fodder. Think low-quality, sensational plots designed purely for traffic—what many call "digital garbage." These popped up across Weibo, Douyin, Kuaishou, and WeChat.
Fake disasters and scams: AI was used to fabricate disaster scenes and rescue footage, invent social hot topics, push feudal superstitions, and even deploy AI avatars to hawk products deceptively—often targeting the elderly.
Deepfake impersonation: Face-swapping and voice-cloning tech was used to impersonate public figures, generating vulgar dance clips and fake intimate scenes for profit.
Horror and gore: Some accounts mass-produced AI-generated images filled with terrifying, weird, violent, and vulgar content.
Toxic content for kids: AI-generated material depicting underage pregnancy and fighting, or promoting "toxic animations" and disturbing videos aimed at children, was being churned out and spread widely.
Fake engagement: AI "management" software was used to simulate real users, posting comments and interactions to boost engagement—basically, a high-tech version of astroturfing.
Inappropriate AI agents: Some websites and platforms had AI agents with covers, descriptions, or dialogues that were obscene or harmful to minors.
What's Next?
The CAC says all involved accounts have been dealt with according to the law. For websites and platforms that refuse to clean up their act, penalties could include warnings, fines, removal, or even shutdown.
Looking ahead, the regulator promises to keep a close eye on new risks from AI-generated content. The message is clear: AI is a powerful tool, but it can't be a free pass for chaos. Expect more oversight and swift action as the campaign continues.
Key Points
- The CAC's second phase of the "Clear and Bright" campaign removed over 5.61 million pieces of illegal content and investigated 49,000 accounts.
- Platforms like Douyin, Weibo, and Baidu have upgraded AI governance with better recognition models and labeling requirements.
- Seven typical cases were exposed, including AI-modified classics, deepfake impersonation, and toxic content targeting minors.
- Penalties for non-compliant platforms range from warnings to shutdowns, and the regulator vows to keep monitoring AI risks.