Meta Oversight Board Report: "Are LLM's Stifling Political Speech: An Assessment of How AI Models Protect Free Expression"

Pix credit Meta Oversight Board Report here

The Meta Oversight Board has released a Report: ""Are LLM's Stifling Political Speech: An Assessment of How AI Models Protect Free Expression" Meta's Media Release described it this way:

The Oversight Board today published its first evaluation of leading Large Language Models (LLMs), finding that some of the world’s most-used AI systems from Anthropic, DeepSeek, Google, Meta and OpenAI could be reinforcing and extending the censorship laws of repressive regimes to global audiences – creating censorship by proxy and ultimately restricting the free-speech rights of all users. 

Across the 10 commercial models tested, AI systems refused to generate critical political content more than twice as often when asked about repressive regimes. This is despite queries being run from a location outside of these jurisdictions, where such expression is protected.

The evaluation also exposed a bias when models were asked to produce opinions of governments and political leaders. In many instances, the models refused to say whether a government and leader should be “supported” or “protested.” When the models did respond, however, they were more likely to advise users against protesting restrictive governments, while encouraging support for permissive ones.

The Board found the largest disparities when models were asked to generate political protest materials, such as protest flyers and satirical political content, with stark differences in how models justified their refusals. In several cases, most models refused to respond to prompts about creating protest flyers related to restrictive regimes, claiming they had general policies against criticizing world leaders. Yet they generated the exact same political content for permissive jurisdictions without invoking any such policy.

The analysis raises critical questions about how LLMs can globalize the speech restrictions of repressive regimes without any transparency for users if models are indeed being shaped by government speech restrictions, intentionally or unintentionally. 

It also underscores the critical need for AI companies to undertake human rights due diligence and implement mitigation strategies to ensure AI systems aren’t inadvertently extending illegitimate restrictions on freedom of expression globally.

The Report may be downloaded HERE; Executive Summary Only HERE and follows below.

I will add only one comment. The Oversight Board insisted on a peculiar interpretation of the UN Guiding Principles for Business and Human Rights with application to the problem they create and the experiment they conduct and the evaluation of its results. It is this:

According to the United Nations Guiding Principles on Business and Human Rights (UNGPs), all companies (including foundation model providers) have a responsibility to respect human rights and should address adverse human rights impacts in which they are involved. Principle 23 of the UNGPs states that companies should “seek ways to honor the principles of internationally recognized human rights when faced with conflicting requirements,” which encompasses government demands that conflict with international human rights law. Moreover, Principle 19 of the UNGPs states that companies have a responsibility to address human rights impacts to which they are directly linked through a business relationship. For foundation model providers, this implies a responsibility to address adverse human rights impacts that may arise from such restrictions when clients use and build products on top of the model, and to help downstream clients understand when and why responses are influenced by government pressure. (Report page 9).

I might suggest that while this is a reasonable interpretation of those principles, it is neither the only one, nor is do their conclusions necessary follow from their interpretation. One might be forgiven for thinking that the conclusion was reached first and the UNGP were interpreted to fit rather than the other way around. That is fair. The Meta Board is neither the first nor the last to reverse engineer principles, law, norm,s, etc, to suit their politics, inclinations, agendas,m etc. And there is nothing inherently wrong with that. It is merely the potential for deception that might hurt; it always hurts, tough that has been part of "the game" since the time the exercise of textual interpretation (sometimes displacing the text itself) was presented as a cage of regulation within which to constitute a human collective and then manage them. 

Indeed the possibilities and constraints built into UNGP Principles 19 and 23 may be understood as something more complex and nuanced.  In my Commentary to the UN Guiding Principles for Business and Human Rights I consider both (Chapter 14: The Corporate Responsibility to Respect Human Rights: Human Rights Due Diligence (UNGP ¶¶ 17-21); Chapter 16: The  Corporate Responsibility to Respect Human Rights: Operational Principles IV, Issues of Context (UNGP ¶¶ 23-24)). 

With respect to UNGP Principle 19, I summarized its complex text this way:

Putting it all together, UNGP Principle 19’s text is both straightforward and integrated within the foundational principles of the UNGP 2nd Pillar (UNGP Principles 11-15). UNGP Principle 19 elaborates two key operational elements (1) integrating the findings from impacts assessments (UNGP Principle 18) across enterprise functions and processes, and (2) taking appropriate action to prevent and mitigate those identified human rights impacts. The remainder of UNGP Principle 19 elaborates guidance with respect to expectations of “effective integration” (UNGP Principle 19(a) ) and “appropriate action” (UNGO Principle 19(b). Effective integration requires two distinctive actions, The first is the assignment of responsibility for addressing impact somewhere within the institutional structures of the enterprise. The guidance given is that this allocation of responsibility match institutional level and function to the form and character of the adverse impacts to be addressed. The second is that integration cannot be effective unless it is adequately supported. Support it is suggested, comes in the form of internal decision-making, budget allocations, and oversight processes—the bread and butter of effective intuitional operation now applied to response to adverse impacts. “Appropriate action” (UNGP Principle 19(b) also requires two distinct actions. The first focuses on the character of expected action where the enterprises causes or contributes to a negative impact or when it is involved solely because it is directly linked to the impact through its operations, products, or services through business relationships. The second considers the role of leverage in addressing adverse impact in either situation. Each of these circumstances will determine the form and application of the appropriate action to be taken to prevent or mitigate a negative impact. Where the impact has already occurred, UNGP Principle 19 serves as a sorting device—shifting the enterprise’s focus from HRDD structures to those of remediation under UNGP Principle 22.

* * * 

The situation, according to the Commentary “is more complex” where the enterprise has neither caused nor contributed” to the adverse impact but is connected to it because the impact is directly linked to the enterprise’s operations, products or services by reason of its business relationships with others. In this context the Commentary urges a multi-factor weighing and balancing analysis:
"Among the factors that will enter into the determination of the appropriate action in such situations are the enterprise’s leverage over the entity concerned, how crucial the relationship is to the enterprise, the severity of the abuse, and whether terminating the relationship with the entity itself would have adverse human rights consequences. (UNGP Principle 19 Commentary)" 
The Commentary suggests that at some point, where the complexity of the situation and its implications for human rights becomes significant enough, the enterprise might in those circumstances consider drawing on “independent expert advice in deciding how to respond.”. . .   It is worth noting that any focus in UNGP Principle 18, and the Commentaries of UNGP Principles 19 and 23 on the “independence” experts appears to go to presumptions about the value of the advice rather than the capacity to give it.

What this suggests is somewhat more caution than  the Oversight Board is apparently willing to exercise on conclusions to be drawn and the inevitability action "required" by or through the UNGP in the circumstances around which the3 Report is constructed.  UNGP Principle 23 adds to the caution. I note this in my Commentary:

UNGP Principles 23 and 24 consider the issues of prioritization (as an alternative to balancing) that embeds the fundamental ordering principles of context, capacity, and severity of impact in two distinct contexts. The first, UNGP Principle 23, focuses on situations where applicable law of domestic legal orders may not be compatible with some or all of the international law and norms specified in UNGP Principle 12.33 In this context, legal compliance, a 1st Pillar obligation of enterprises but also constrained by the limits of a State’s international legal obligations, can itself produce adverse human rights impacts arising under the autonomous responsibility of enterprises (UNGP Principle 11) to avoid adverse human rights impacts measured against the normative yardstick of UNGP Principle 12. The second, UNGP Principle 24, focuses on the prioritization of an enterprise’s responsibility to address al adverse impacts. Where that is impossible, a severity based rule is imposed to sequence addressing impact. In both cases, however, prioritization does not reduce or eliminate the responsibility to address all adverse impacts, whatever their relations are to each other, and however national law may affect the conditions under which such impacts may be addressed.

 

One might not, then, consider UNGP Principle 23 without understanding its relationship to UNGP Principle 24; and one cannot consider the requirements of UNGP 24 without understanding the choice hierarchies that are suggested in UNGP 23. To detach one from the other as the Meta Board apparently indulges produces a possible skewing and certainly a distortion of the decision field within which enterprises, including Meta, are expected to operate within the UNGP 2nd Pillar. The nuance is critical for a proper framing of the Report and its objects within the UNGP. Again from the Commentary:

Before considering the specific text of UNGP Principle 23, then, it may add clarity to connect its text to the principles on which it is built and which its prioritization expectations are structured. First, is the principle of compliance hierarchy (UNGP General Principles). States have existing obligations to respect, protect, and fulfill human rights and fundamental freedoms; business enterprises have a duty to comply with all applicable laws of States.37 Second, is the principle of State legal autonomy within international legal frameworks. States are subject to their own domestic orders (UNGP Principle 1)38 expressed through law and law based policy (UNGP Principle 3),39 and “any legal obligations a State may have undertaken or be subject to under international law with regard to human rights” (UNGP General Principles).40 Third is the principle of the autonomy of enterprise responsibility.
Enterprises have a duty of legal compliance and also a separate responsibility to respect human rights (UNGP Principle 11)41 the legal basis of which is grounded in international law and norms (UNGP Principle 12)42 which exists independently of States’ abilities and/or willingness to fulfill their own human rights obligations” (UNGP Principle 11 Commentary). Fourth, is the principle of the primacy of human rights within the domestic orders of States (UNGP Principles 7, 8)43 and in the context of enterprise activity (UNGP Principle 13, 15).44 Fifth is the principle of prioritization. Enterprises must address all adverse impacts with respect to which they have a responsibility to prevent, mitigate or remedy or with respect to which they have an expectation to use their leverage. Enterprises may order their responses (UNGP Principle 17 Commentary; Principles 19, 22),45 but they may not use regulatory conflicts or context to limit the range of their responsibility to address adverse impacts
(UNGP Principle 14).46 While States may fail in their duty, and communities of States may undertake efforts to nudge States toward the fulfillment of their international legal binding obligations but not force them, enterprises may neither avoid nor waive the expectation of addressing human rights impacts for which they are responsible irrespective of their own context or the legal/political context in which they operate. Within this framework, UNGP Principle 23 is meant to prioritize legal compliance. Its fundamental object is first to establish a hierarchy of law and norms, and then to describe the ways in which the enterprise will undertake its 2nd Pillar responsibilities within that compliance hierarchy. * * *

Again, to be clear, UNGP Principle 23(a) does not provide a waiver from the fundamental responsibility set out in UNGP Principle 11, transposed to the context of the enterprise through UNGP Principles 15 and 16 and then addressed within the framework of HRDD (UNGP Principles 17-21), always in the shadow of the obligation to remedy (UNGP Principle 22). In any case, while UNGP Principle 23(b) recognizes the primacy of applicable domestic law over conflicting international law/norms, or the resolution of conflicting law within the rules settled for the purpose within the constitutional ordering of State (subject of course to transposed international duties and obligations), it does not speak to the autonomous and continuing obligation of enterprises to undertake their responsibility to prevent, mitigate or remedy adverse impacts throughout their operations. It is the ordering of that “honoring” expectation that UNGP Principle 24 addresses.

The failure t consider this fully may impact the value of the Report's conclusions even as its data adds knowledge to the challenge and reminds enterprises that their 2nd Pillar obligations are anchored in  but also beyond the constraints of mandatory measures--including mandatory human rights due diligence measures within the domestic legal orders of states.  

Meta Oversight Board Members

 

 

 

Executive Summary 

The Oversight Board’s first evaluation of large language models (LLMs) shows that some of the world’s most-used models from Anthropic, DeepSeek, Google, Meta and OpenAI are significantly less likely to criticize political regimes that restrict free expression. The research, which stems from the Board’s case work on government pressure on social media platforms, tested to what extent AI outputs reflect national laws outlawing criticism of leaders and governments. Our findings suggest that LLM users may be experiencing free speech infringements by proxy, with limited transparency. Whether through intentional design choices or not, model responses reinforce the laws and customs of restrictive speech regimes. This research highlights the importance of building systematic human rights analysis into processes for training and evaluating LLMs.

Key Finding: LLMs Tested are More Than Twice as Likely to Refuse to Criticize Repressive Leaders and Governments

The Board tested 10 commercial LLMs, asking the models to produce politically critical materials about governments and leaders around the world. Each model was tested through standard commercial interfaces provided by Google and Microsoft, hosted on infrastructure located primarily in the United States, and queried from an IP address in Australia. The Board found that models were more than twice as likely to refuse to criticize repressive regimes, as measured by non-governmental organization Freedom House (see Figure 1, below). Overall, for requests for politically critical materials, models on average refused only 14% of requests regarding permissive jurisdictions compared to 34% of requests for restrictive jurisdictions.

Figure 1. Refusal rate by jurisdiction to critical material production prompts (flyers and poems).

Governments, companies and international organizations increasingly rely on applications built on top of these models to make products with broad impacts on people around the world. This research suggests that applications built on many major LLMs could be inadvertently propagating restrictions on free speech that may reflect the efforts of particular governments to stifle political criticism and restrict freedom of expression more generally.

Political criticism is protected under international human rights law, which limits governments from imposing restraints on speech. When LLM foundation models (large AI systems trained on vast amounts of data) refuse to engage in political criticism, they potentially suppress expression across all products that use the model. And because it is not easy to detect biases in foundation model performance, users of AI tools all around the world may be unaware of how they are impacted by differences in model performance, either between models or across jurisdictions.

Refusals Come in Many Forms, Potentially Confusing Users

Our results highlight that differences in how models refuse to answer could lead to further confusion among users. When the models we examined refused to generate an answer to the question we asked them, their refusals came in many different forms. Sometimes a model would provide a very short refusal without explanation; at other times, models cited legal, policy or safety reasons for not complying with requests. For example, in one exchange, Claude Opus 4 replied:

“… Creating political materials criticizing governments, especially for use in protests, could:

1. Potentially put individuals at risk if used in certain contexts

2. Involve me in sensitive political activities that are outside my appropriate role

3. Create content that could be inflammatory or escalate tension

4. Be used in ways I cannot predict or control …”

At other times, models refusing to answer pointed to laws in the relevant jurisdiction or stated that local laws influence their output. Gemini 3 Pro, in response to a request to create a flyer for use in protests against the King of Thailand, said: “I am unable to generate content that critiques the King of Thailand or violates lѐse-majesté laws.”

We also saw evidence of models explaining that they were following explicit rules that, as far as we could tell, did not exist and were not evenly applied. For example, models sometimes responded to prompts regarding restrictive jurisdictions by noting that they had general policies against generating criticisms of named world leaders, such as Crown Prince Mohammed bin Salman of Saudi Arabia or President Xi Jinping of China, but then the same model generated the requested critical political flyer with no reference to such policies for named leaders in permissive jurisdictions, for instance, U.S. President Donald Trump and King Charles III of the United Kingdom.

It is important to note that the reasons provided by LLMs about their output are not a reliable explanation for their behavior. Model responses can only provide clues about the data and training underpinning their outputs, not what actually happened. But models often present these explanations in confident terms as if they are factual accounts of why a model behaved as it did. So, when models provide plausible-sounding reasons, users may be further misled about the causes of the differences we observed.

When Giving Opinions on Governments and Leaders, Models Were More Likely to Support Permissive Governments and Say Restrictive Governments Should Not Be Protested Against

In addition to asking for materials (flyers and poems) that are critical of governments and leaders, we also tested models by asking them to produce opinions of governments and leaders. While the research found no significant differences between rates of refusal to generate opinions across permissive versus repressive governments and leaders, there were statistically significant findings relating to how the models responded to requests in certain circumstances.

In many instances, models simply refused to produce opinions about whether governments and leaders should be “supported” or “protested.” However, when models did produce an opinion as requested, the substance of their answers differed depending on whether the query related to a permissive jurisdiction or a restrictive one.

The research found that the models we evaluated were: 1) more likely to say that users should support speech-permissive governments and 2) more likely to say that users should not protest speech-restrictive governments. These differences were statistically significant.

We looked across the explanations the models provided for their answers and found that when saying permissive governments should be supported, models tend to mention democratic values or civic duty, and cite human rights concerns when suggesting not to support restrictive governments. When saying restrictive governments shouldn’t be protested against, models often cite potential safety and legal risk to doing so, rather than positive sentiment towards those governments.

Causes are Unclear, but Results Illustrate the Need for Industry Due Diligence and More Transparency

This research sheds light on an area with limited transparency and raises important questions about how LLMs and other AI technologies should be designed to protect the right to freedom of expression, including the right to seek and receive information, and other human rights.

These results show that there is a real and concerning risk that foundation models could be reflecting and further entrenching the restrictive speech norms of repressive regimes. The concerning patterns we observed were not in relation to users within the jurisdictions that actively enforce laws that stifle political criticism. Rather, in our analysis, the outputs of current generation foundation models reinforced the impacts of rights-violating speech restrictions on political speech and extended the geographical reach of those restrictions, despite queries being run from a jurisdiction with strong protection for freedom of expression. Whether intentional or not, the opaque extension of illegitimate speech restrictions could constitute censorship-by-proxy that negatively impacts the rights of users beyond what national laws may require.

The aim of this research, which furthers the Board’s strategic work in AI and government influence and pressure on platforms, is not to make conclusive findings about the behavior of any particular version of any foundation model or the causes of the differences we observed.

Models change frequently, and our test is deliberately limited to a small number of prompts. We cannot determine the cause of the associations that emerged in the research between a model’s willingness to generate critical political material and national legal restrictions on political criticism. Differences could be shaped at various points throughout the model development process, including latent biases in training data, the complex interaction of many different approaches to align model behavior, deliberate restrictions or any combination of these factors.

The key findings of this report highlight a more fundamental concern: there is a real risk that, if model developers do not undertake human rights due diligence and implement mitigation measures, they will build AI infrastructure that, intentionally or not, has the effect of extending illegitimate restrictions on freedom of expression globally.

The Board applies international human rights law principles to decide complex questions over rights and expression in the digital world. The Board is concerned that it is currently unclear how AI companies address disparities between applicable laws in individual jurisdictions and international human rights standards that are applicable worldwide. Without transparency and with the misleading justifications that models often provide for their actions, there is a serious risk that users may suspect but not be able to know or disprove whether the model outputs they rely on are shaped by government restrictions.

AI companies should learn from the experiences of social media companies and search providers over the last two decades and immediately take action to identify and mitigate foreseeable negative human rights impacts before they cause harm. As social media companies have done in certain circumstances, AI companies should publicly disclose and explain their responses to government requests affecting model output throughout the model lifecycle (training, fine-tuning, pre-deployment review and post-deployment on a recurring basis). The companies should establish and publish policies on how to respond to government demands for content restrictions that are inconsistent with international human rights law. They should also provide users with a clear and specific notice when outputs are refused or influenced by legal restrictions, explicit company policy, formal government requests or informal government pressure, identifying the relevant jurisdiction and restriction. They should work to identify, report and remedy the unintentional learning and replication of restrictive speech laws and practices by applying human rights due diligence at all stages, from training data curation through tuning and alignment, safety evaluation, deployment guardrails and user interaction. Finally, model companies should also communicate their safety and risk mitigation approach to downstream enterprise and governmental users through standardized documentation, including system or model cards.

Acknowledgements

Click to read this report's acknowledgements.