State Media Control Shapes LLM Training Data, Princeton Study Finds

Large language models do not learn from a neutral internet. A new study published in Nature reveals that state-coordinated media content infiltrates AI training data at rates that measurably shift model behavior toward government-preferred narratives. The research team, led by Brandon M. Stewart of Princeton University, demonstrates that institutional influence over information environments translates directly into algorithmic outputs.

The paper, titled "State Media Control Influences Large Language Models," appears in Princeton SPIA's Research Record series. Stewart, an associate professor of sociology, serves as the corresponding author alongside co-first authors Hannah Waight (University of Oregon) and Eddie Yang (Purdue University). The study combines evaluations across 37 countries with a detailed case study examining Chinese state media.

Here is the physical reality of the problem: when researchers compared two sources of Chinese state-coordinated media against a major open-source training dataset derived from Common Crawl, they found over 3.1 million Chinese-language documents with substantial phrasing overlap. That represents approximately 1.64% of the dataset's Chinese-language subset. For documents mentioning Chinese political leaders or institutions, the share climbed to 23%.

State-scripted news showed up in LLM training data at a rate 41 times greater than Chinese-language Wikipedia entries. Only about 12% of the matched documents came from known government or news domains. The rest had spread across ordinary webpages, apps, and reposts until the messaging looked like part of the broader information environment (which is precisely how propaganda becomes invisible).

The researchers tested whether this content could actually shift model behavior. They took a small, open-weight model and added additional documents to the training process. The results were stark: adding scripted news to training data made the models produce more favorable answers nearly 80% of the time compared to an unmodified model. This held true even when compared to other non-scripted Chinese media.

Commercial models present a different challenge. Since companies do not provide access to their training data, the team used a within-model, cross-language comparison. They reasoned that if states have strong real-world influence over pretraining data, it should appear most clearly in the state's primary language. When human raters evaluated responses to political questions about China, the Chinese-prompted answer was more favorable to China 75.3% of the time. For prompts not about China, the rate was no different from chance.

Joshua Tucker, co-author and co-Director of the NYU Center for Social Media, AI, and Politics, noted that the public debate has focused on what AI can generate. This study points upstream. Before AI systems can influence politics, politics can influence AI.

The team calls this phenomenon "institutional influence." The mechanism is straightforward: state-coordinated content recirculates through newspapers, apps, reposts, and ordinary webpages until it looks like part of the broader information environment. Once state-coordinated content is in the training data, the model can launder it into what looks and sounds like neutral, objective information. Large language models separate the message from the messenger.

What began as a strategic narrative from a powerful government in a state media outlet can reappear as informed commentary from a highly knowledgeable intelligent agent. With no visible source reputation, people lack any signal about the interests that shaped that answer. This is not a theoretical concern. It is measurable, reproducible, and already happening in deployed systems.

The researchers recommend greater transparency from AI companies so that their models' training data is evident to all. They also suggest extending their study from text-only models to image and video models. The bottom line, as the team concluded, is that training data does not just fall from the sky. It is produced in a context mitigated by the existence of socio-political institutions.

Understanding these institutions can and should in the future be harnessed to produce better understanding of the outputs of LLMs. Whether AI companies will voluntarily disclose their training data sources remains an open question. The incentive structure does not currently favor transparency.

The study's findings raise concerns about AI, democracy, censorship, and training-data transparency. Users asking political questions in different languages may receive systematically different answers without knowing why. The technology works exactly as designed. The design itself is the problem.

Artūras Malašauskas is an AI Systems Integrator with 20+ years of production-grade web engineering experience. He has designed, shipped, and scaled enterprise Python/PHP systems for logistics, SaaS, and public-sector clients. For the past year, he has focused exclusively on AI integrations: deploying open-source LLMs, building generative media pipelines (image, audio, video), and engineering multi-agent workflows for real production environments. His standard: reproducibility, security, cost-efficient inference—no vaporware. He documents and evaluates emerging AI tooling, separating verified capabilities from marketing noise. Technical editor at: muza-ai.eu, ai-verslas.lt, ai-naujinos.lt Connect on LinkedIn