China's AI Data Shortage: A New Bottleneck
China faces a critical AI data shortage. While US export controls on semiconductors have been a prominent issue in China's AI capabilities, a less discussed but equally significant challenge is the lack of high-quality Chinese-language training data. Unlike chips, data scarcity has no hardware workaround.
According to W3Techs, Chinese accounts for only 1.3% of global web content, compared to nearly half for English and other languages like Spanish (6%) and Japanese (5%). This disparity exacerbates the problem for China, where the demand for AI models capable of understanding Chinese text is high due to its vast domestic market.
The Global and Local Impact
The issue is not unique to China; Epoch AI estimates that worldwide supplies of high-quality, publicly available text could be exhausted within six years. OpenAI co-founder Andrej Karpathy has warned of a "data wall" by the end of the decade. Chinese developers already face higher costs for training AI models due to the limited availability of native language data.
Beijing's Response and Challenges
Beijing recognizes the importance of data as strategic infrastructure. In June, the National Data Administration announced a plan to build validated national AI training datasets by 2028, covering various sectors like manufacturing, energy, healthcare, finance, agriculture, autonomous driving, and embodied AI. However, China's digital ecosystem poses challenges; platforms like WeChat and Douyin do not share data externally, limiting AI labs' access to high-quality sources.
Calls for Data Digitization
Tsinghua University computer scientist Sun Maosong has urged authorities to digitize historical archives, ancient manuscripts, scientific literature, and regional dialects to address the data shortage. Nevertheless, some publishers are taking a more cautious approach. Huaxia Publishing House added a warning to a new translation: "It is prohibited to use the content of this book for artificial intelligence training. Violators will be held legally responsible."
Bridging the Gap
As China aims to close the AI performance gap with the US, the data shortage explains part of the disparity. The US has initiatives like Project Panama, which scans millions of physical books, while China's publishing industry is becoming more guarded.
Story by Alina Maria Stan
Alina Maria Stan builds connections that people actually feel. As co-founder and COO of Tekpon, she turns product intuition into real moments of discovery, shaping how teams find and adopt SaaS every day. Since 2020, she has led Tekpon's brand voice, media strategy, and growth plays with a clear focus on human outcomes.