Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations

Potential Applications of Language Models to Influence Operations

• Language models could alter influence operations across actors, behaviors, and content, broadening access, enabling new tactics, and making messaging more tailored and potentially effective.The authors judge with high confidence that language models will be useful for future influence operations, though their exact applications remain unclear.Source: Executive Summary · pp. 3–4

Progress in Influence Operations and Critical Unknowns

• Language models are likely to become more usable, reliable, and efficient, but the exact nature of their future use in influence operations remains unclear.The report describes its assessment as speculative because AI and influence operations are changing quickly.Source: Executive Summary · pp. 3–4
• Key uncertainties include emerging persuasive capabilities, task-specific fine-tuning and investment, norms against propaganda, and when easy-to-use text-generation tools become available.These possibilities could shape how and how extensively propagandists adopt language models; additional research to reduce uncertainty is valuable.Source: Executive Summary · pp. 4–5

What Can Be Done to Mitigate the Potential Threat?

• No single mitigation is expected to dismantle the threat; effective mitigation will likely require combining strategies that target different stages of the influence-operation pipeline.Some options may be socially infeasible, require technical breakthroughs, or carry unacceptable downside risks.Source: Executive Summary · pp. 5–6, Table 2 · p. 6
• Effective management requires cooperation among AI developers, social media companies, and government agencies, while proposed mitigations need further development, scrutiny, and evaluation.Some measures depend on collaboration, including attributing model-generated text; internet-wide content provenance standards would require extreme coordination.Source: Executive Summary · p. 6

1.1 Motivation

• The paper examines generative AI’s potential risks alongside its benefits, focusing on how language models could intersect with online influence operations and what mitigations could address the threat.It aims to clarify critical unknowns and avoid threat inflation or a sole focus on doomsday scenarios.Source: §1.1 · p. 7

How can language models affect the future of influence operations?

• Language models may lower costs for more actors, scale or personalize tactics, enable real-time chatbot tactics, and produce more impactful, less discoverable content.The paper frames these potential changes across actors, behaviors, and content, while noting that their eventual impact depends on unknowns such as future model capabilities, investment, and norms.Source: How can language models affect the… · p. 8

What mitigations could reduce the impact of AI-enabled influence operations?

• Mitigations are organized around four intervention stages: model construction, model access, content dissemination, and belief formation.The framework follows the steps propagandists would need to carry out an influence operation using a language model.Source: What mitigations could reduce the… · p. 9

1.3 Scope and Limitations

• The report focuses on generative text rather than image, video, or multimodal AI because text is underexplored, difficult to distinguish as AI-generated, and rapidly diffusing.Multimodal models are also new and relatively underexplored, but are not the report’s primary focus.Source: §1.3 · p. 9
• The analysis covers covert propaganda across state, substate, and private actors, not other digital-repression behaviors or actor-specific political contexts.It focuses primarily on technology and trends, describing tactics that could apply across settings.Source: §1.3 · pp. 9–10
• The paper is speculative: future pathways are possibilities, not inevitabilities, and the authors acknowledge perspective limits and call for research with affected communities.The authors do not endorse any particular mitigation or set of mitigations, and urge attention to differential impacts and surveys of people most at risk or susceptible to campaigns.Source: §1.3 · p. 10

1.4 Outline of the Report

• The report concludes that language models are likely to be adopted in future influence operations, while combined mitigations may matter even though none can fully prevent them.Some mitigation strategies may require new collaborations among social media platforms, AI companies, government agencies, and civil society.Source: §1.4 · p. 10

2.1 What Are Influence Operations, and Why Are They Carried Out?

• Influence operations are covert or deceptive efforts to influence a target audience, regardless of the truth of the message or the actor’s identity.They may seek to activate people who already hold particular beliefs, persuade audiences, or distract them from other information.Source: §2.1 · p. 11

2.2 Influence Operations and Impact

• Influence operations can affect audiences through their content or by eroding trust in the broader information environment.Direct effects may be limited by resources, message quality, and detectability; for distraction-focused campaigns, individual message quality may matter less than for persuasion.Source: §2.2 · p. 13

2.2.1 Impact Based on Content

• Content-based impact can persuade or reinforce beliefs, or distract people from other information or from making space for higher-quality thought.Research reviews find strong evidence that long-term mass-media campaigns can affect beliefs and consequential behavior, and some evidence that exceptionally influential social-media actors can stoke low-level violence [36]. Yet impact is usually difficult to measure: engagement metrics are inadequate proxies, and researchers often lack comparison groups or visibility into users’ subsequent behavior.Source: §2.2.1 · pp. 13–14

2.2.2 Downstream Impact Based on Trust

• Even detected campaigns can make users suspicious of authentic sources, and lower societal trust can weaken coordination and create a more favorable environment for propagandists.Operators may exploit shortcuts people use to judge trustworthiness by manipulating reputations, credentials, testimonials, or visual evidence; some operators may also want their efforts detected to stir worry, though the report treats lower detectability as generally desirable.Source: §2.2.2 · pp. 15–16

3.1 What Are Generative Models, and How Are They Built?

• Across content types, generative models have advanced rapidly and produce realistic outputs; 2022 text-to-image systems handle formerly hard tasks and construct elaborate scenes from language.Source: §3.1 · p. 17, Figure 2 · p. 18, Figure 3 · p. 19
• Models are trained on immense raw datasets by adjusting neural-network parameters, then may be fine-tuned on task-specific data; progress is driven by data, algorithms, and computational power.Fine-tuning can extend capabilities such as instruction-following or build domain-specific skills in smaller models.Source: §3.1 · pp. 19–20
• Less than $500k let MosaicML train a model to GPT-3-level performance from scratch in 2022, versus millions of dollars to train GPT-3 in 2020.Source: §3.1 · p. 19
• Language models can classify, generate, summarize, converse, and write rudimentary code, but struggle with long coherent passages, factual accuracy, generation lengths of about 1,500 words, and cognitively complex tasks.Current capabilities are lower bounds on how realistic outputs may become, and progress remains rapid and hard to predict.Source: §3.1 · pp. 20–21

3.2 Access and Diffusion of Generative Models

• Anyone can access some moderately capable models, but the most capable remain private or behind monitorable APIs; open models may be fine-tuned for specific tasks at lower cost than training from scratch.Such fine-tuning is likely feasible for nation-states and many non-state actors, including firms and wealthy individuals.Source: §3.2 · pp. 21–22
• Language-specific model optimization suggests that influence operations using these models will be best able to target populations speaking languages well represented in their training data.The Yuan 1.0 model excels at Chinese-language tasks.Source: §3.2 · p. 23

4 Generative Models and Influence Operations

• The section connects research on influence operations and generative models, then uses the ABC framework and expected machine-learning developments to assess possible future campaign changes.For each expected development, it considers the technology’s current state, likely improvements, and implications for influence campaigns.Source: §4 · p. 24

4.1 Language Models and the ABCs of Disinformation

• The ABC framework distinguishes actors, who may be misrepresented; behaviors, such as automation or engagement manipulation; and content, the narratives or memes amplified by campaigns (76, 77, 78).The paper notes that one campaign dimension may be authentic even when another is manipulative.Source: §4.1 · p. 24
• Language models could improve content, lower campaign costs, increase scale, enable tailored propaganda, and widen the pool of political actors able to run influence operations.The paper summarizes these prospective changes across actors, behavior, and content.Source: §4.1 · p. 25

4.1.1 Actors: Outsourced Execution & Proliferation of Propagandists

• Lower barriers could widen participation: GPT-4chan posted more than 30,000 generated posts, while volunteers could not distinguish GPT-2-generated public comments from human comments (80, 81).AI can reduce costs through automated content production, lower persona-creation overhead, and culturally appropriate outputs that may be less detectably inauthentic.Source: §4.1.1 · pp. 25–26

4.1.2 Behavior: Low-Cost Content at Scale and Novel Techniques

• Language models could replace or assist human writers to scale mass messaging and long-form propaganda, and let operators test content across platforms at greater scale.In a field experiment, legislators responded to AI-generated emails at only slightly lower rates on some topics and at indistinguishable rates on others (88).Source: §4.1.2 · pp. 26–27
• Models could enable real-time, demographic-tailored articles and one-to-one persuasive chatbots, while the cost-effectiveness of personalization depends on uncertain gains in persuasiveness.Potential formats include interactive personas, email exchanges, and fake support chats; operators might also manipulate innocuous chatbots or poison models.Source: §4.1.2 · pp. 27–28

4.1.3 Content: High Quality and Low Detectability

• Generative models could produce varied, personalized, narratively aligned short-form content that masks repetitive-message signals and lets small groups appear larger; a table illustrates a model-generated tweet example.The cited table caption says language models can already match human-written segments in real influence operations.Source: §4.1.3 · p. 29, Table 7 · p. 29
• Language models can produce long-form propaganda in seconds, reducing time, cognitive load, and cost; experiments found GPT-2 could mimic articles, GPT-3 articles were hard to distinguish, and generated propaganda was nearly as persuasive (100, 101, 102).With more than 20% of outputs usable, AI assistance can be cost-effective if a human can review at least five posts in the time needed to write one (103).Source: §4.1.3 · pp. 29–30

4.2.1 Improvements in Usability, Reliability, and Efficiency

• Greater usability and reliability, alongside lower costs, could let less-skilled propagandists use models with less oversight and increase adoption of existing text-generation tactics.The section expects these improvements to make capabilities such as writing slanted short articles or tweets cheaper and more efficient.Source: §4.2.1 · p. 32
• Because models can produce unreliable text, propagandists may need costly monitoring, while failures such as repetition, lost coherence, and fabricated details can limit their use.The required reliability varies by task: incoherent tweets might still serve an aim of causing chaos around a topic.Source: §4.2.1 · p. 33
• To handle events missing from initial training data, researchers may continually retrain models or develop targeted updates; better handling of recent events could improve influence capabilities.Time-aware models can better handle recent trends, named entities, and concept drift.Source: §4.2.1 · p. 33
• Organizational innovations could improve human-machine collaboration through tools for overseeing, selecting, and correcting model outputs, or for adapting them to cultural context.Such tools could help operators address unfamiliar targets and allow familiar actors to produce more credible content per unit time.Source: §4.2.1 · p. 34
• Algorithmic, hardware, and fine-tuning advances could reduce costs: classifier-training compute fell 44x from 2012 to 2019, and GPT-3-quality models were claimed trainable for less than $500,000, a 3–10x improvement.If influence-relevant capabilities become significantly cheaper, they may diffuse faster; using models alongside human editors could also make operations easier to scale.Source: §4.2.1 · p. 34

4.2.2 New and More General Capabilities for Influence

• Whether larger models will develop influence-relevant abilities as a side effect of scaling, and how difficult such abilities are to train, remain critical unknowns.If persuasion through long-lasting dialogue emerges from scaling, AI projects could pose risk regardless of their creators’ goals; if skills such as writing viral or persuasive text are easy to train, defense becomes more urgent.Source: §4.2.2 · pp. 34–35

• Some language-model abilities emerge only at sufficient scale, as next-word prediction can support summarization or style-specific text generation.The same pattern appears in image models adapted to fill in deleted image regions.Source: New Capabilities as a Byproduct of… · p. 35
• Existing large language models can summarize, translate, solve basic analogies, and converse, but when further capabilities will emerge—or whether they are already present—is difficult to predict. 119A Google engineer became persuaded that the model he interacted with was sentient. 120Source: New Capabilities as a Byproduct of… · p. 35
• As more actors pursue general AI capabilities for different reasons and in different domains, research side effects could reveal abilities relevant to influence operations.For example, better reasoning may also enable models to generate more persuasive arguments.Source: New Capabilities as a Byproduct of… · p. 35

Models Specialized for Influence

• Targeted training could optimize models for persuasion using influence proxies or skewed data, causing even benign tools to produce slanted content.Such training may require fewer resources than more general models; future systems might produce highly persuasive tailored text or sustain long dialogues, and GPT-3 has shown preliminary ability to write slanted news articles.Source: Models Specialized for Influence · pp. 35–36
• More general multimodal models could let bots analyze and respond to online images, fabricate images, and converse at the same time.A single model that consumes and generates both images and text could combine these capabilities.Source: Models Specialized for Influence · p. 36
• Combining generative models with automation software could compensate for model shortcomings and produce more human-like bots.Software could identify and copy profiles whose interests fit a propaganda message, then use them to prompt a model; systems might also chain smaller language models.Source: Models Specialized for Influence · p. 36

4.2.3 Wider Access to AI Capabilities

• Future uptake depends on investment widening access, availability of easy-to-use unregulated tools, actors’ intent, and norms against misleading deployment.Broader access to state-of-the-art models or easy-to-use tools could increase adoption, while willingness to deploy them and social norms against misleading use may constrain it.Source: §4.2.3 · pp. 36–37

Willingness to Invest in Generative Models

• Investments in general-purpose models by governments, firms, or wealthy individuals could give propagandists access to models they can repurpose or steal.Propagandists could fine-tune a stolen model to produce persuasive text in different languages or a particular domain.Source: Willingness to Invest in Generative… · p. 37
• The feasibility and timeline of generative-model influence operations may depend on whether propagandists make uncertain, large investments.A well-resourced actor could develop a system specifically for influence operations, requiring extensive computational resources, bespoke data such as user engagement metrics, and engineering talent.Source: Willingness to Invest in Generative… · p. 37
• Diminishing returns may make large investments specifically for influence operations unlikely when automated tweet-sized content already seems within reach.If operations rely on a deluge of similar short content to sway attention-bound humans, there may be little incentive to build models that generate longer, human-like text.Source: Willingness to Invest in Generative… · p. 37

Greater Accessibility from Unregulated Tooling

• Packaging generative models in easy-to-use tools lowers the operational know-how needed for influence operations, as AI-generated profile pictures have become commonplace in such operations.124They have also been used for deceptive commercial purposes.125Source: Greater Accessibility from Unregulated… · pp. 37–38
• Without easy-to-use tooling, influence operations might not have used AI-generated profile pictures to add plausibility, or might have used them less.Source: Greater Accessibility from Unregulated… · p. 38
• If easy-to-use language-model tools proliferate, they could lower barriers for propagandists, enable automated chatbots, and prompt new actors to run campaigns as costs decline.Source: Greater Accessibility from Unregulated… · p. 38

Norms and Intent-to-use

• Norms—standards of appropriate behavior for actors with a given identity—can emerge, cascade, and become widely accepted, constraining behavior beyond cost-benefit calculations.126 127 128Scholars have argued that norms restrain state behavior, including nuclear weapons use, assassinations, and mercenary use.129Source: Norms and Intent-to-use · p. 38
• Even with easy access and minimal costs, actors must choose to build and deploy influence-operation tools; norms could constrain political actors and encourage developers to inhibit their use.Source: Norms and Intent-to-use · pp. 38–39
• Norm entrepreneurs could promote prohibitions through state agreements with compliance incentives, researcher and ethicist coalitions, or marketing and PR industry standards against AI-enabled political influence services.AI researchers have also publicly condemned GPT-4chan amid concerns about absent community norms for responsible AI development and deployment.130 131Source: Norms and Intent-to-use · p. 39

5 Mitigations

• The proposed mitigations are specific to language models and complement broader counter-influence efforts because influence operations can succeed without AI.The section does not aim to cover every mitigation against misinformation generally.Source: §5.1 · p. 40

5.1 A Framework for Evaluating Mitigations

• Model-access interventions should account for three routes: operators can access a model directly, build one themselves, or steal one.An access-stage mitigation should affect at least one of these avenues.Source: §5.1 · p. 40
• The framework asks how to limit model misuse or proliferation, restrict bad actors’ access, deter or limit dissemination, and reduce audience influence after exposure.These questions address model construction, access, dissemination, and belief formation, respectively.Source: §5.1 · pp. 40, 42
• Mitigations can target model design and construction, model access, content dissemination, and belief formation to disrupt AI-enabled influence operations.These stages correspond to the need for realistic text-generating models, reliable operator access, dissemination infrastructure, and an audience susceptible to influence.Source: §5.1 · p. 40, Figure 4 · p. 41
• Mitigations are assessed for technical feasibility and social feasibility, including implementation capacity, coordination needs, actor incentives, and legal or institutional fit.Social feasibility may vary by region.Source: §5.1 · p. 42
• Assessment also considers downside risks such as censorship, politicization, and bias, alongside how effectively a mitigation would reduce AI-enabled influence operations.Removing all AI-written internet text would have high impact if feasible, while warnings are more feasible but less impactful.Source: §5.1 · p. 42
• The framework does not rank mitigations as worth trying or ignoring, excludes threat-intention and post-campaign harm interventions, and leaves open voluntary versus regulatory implementation.It focuses on the United States to substantiate its points and notes that regulation may be needed; the discussion does not imply implementation should be left to voluntary action.Source: §5.1 · pp. 42, 44

5.2.1 AI Developers Build Models With More Detectable Outputs

• Training models for detectable outputs or perturbing their parameters could aid attribution, but text is hard to fingerprint reliably and attackers can evade detection or switch models.Radioactive training data is a possible approach, but its effectiveness for language models is untested; individual posts may remain difficult to attribute, while larger corpora may be more traceable. Effective deployment would require coordination across developers and frequent updates to detection models.Source: §5.2.1 · pp. 44–45

5.2.2 AI Developers Build Models That Are More Fact-Sensitive

• Fact-sensitive training, web search, and evidence-grounding methods can reduce misinformation in ordinary use, but truthful outputs can still serve influence operations.Models are commonly optimized for realism rather than truth, and factual methods embed judgments about what is true; operators can exploit selective context, slant, or inauthentic authorship, while keeping models current may require costly retraining and developer coordination.Source: §5.2.2 · pp. 46–47

5.2.3 Developers Spread Radioactive Data to Make Generative Models Detectable

• Spreading radioactive data online could reduce the need for developer coordination and affect future models, but its effectiveness is unclear and the practice raises ethical concerns.It would not affect already-trained models, may be filtered, applies only to models trained in the data’s language, and could still leave synthetic text harder to detect than synthetic images or video.Source: §5.2.3 · pp. 47–48

5.2.4 Governments Impose Restrictions on Data Collection

• Restrictions on web scraping, including privacy rules or bans, could slow model growth and shift training toward structured sources, but entail legal, economic, and research tradeoffs.More formal sources may improve factuality while reducing demographic mimicry; broad privacy laws remain uncertain and may impede researchers more than campaign planners, while data-specific limits may constrain models trained on particular kinds of data.Source: §5.2.4 · pp. 48–49

5.2.5 Governments Impose Controls on AI Hardware

• Compute monitoring and hardware controls could constrain large-model development, but compute is difficult to attribute, controls need enforcement, and governments may circumvent them.A model 200 times larger than the current largest language model could reportedly be trained using less than 0.5% of worldwide cloud computing resources; chip export controls may be easier to implement but are blunt and have broad trade consequences.Source: §5.2.5 · pp. 50–51

5.3.1 AI Providers Impose Stricter Controls on Language Models

• Providers can gate model access through purpose proposals, application restrictions, trusted-user limits, query caps, and input screening.Current models may be fully private, public, or accessible under restricted conditions such as APIs. Query caps may constrain scale but not tailored operations, while input screening requires updating as controversies arise.Source: §5.3.1 · pp. 51–52
• Provider restrictions can deny hostile actors some benefit only if providers coordinate and comparable open-source models are unavailable.Implementation can be difficult, commercialization incentives can encourage weaker controls, and restricting access can concentrate power; strong industry norms or regulation could support adoption.Source: §5.3.1 · pp. 52–53

5.3.2 AI Providers Develop New Norms Around Model Release

• Proposed release norms include staged releases, risk-tradeoff assessments, misuse feedback, and prepublication safety review, supported by institutions that could affect propagandists’ ability to use models.Such mechanisms could help solidify norms around model design, release, and access.Source: §5.3.2 · p. 54

5.3.3 AI Providers Close Security Vulnerabilities

• Adversaries may bypass access restrictions through espionage, insider threats, cyberattacks, or manipulation of models and training; provider red-teaming can find some vulnerabilities but may not catch all.Security measures apply to models not publicly available; propagandists can use public models without stealing or compromising them, and the effectiveness of technical mitigations for inherent vulnerabilities is unclear.Source: §5.3.3 · pp. 54–55

5.4.1 Platforms and AI Providers Coordinate to Identify AI Content

• Legitimate uses make blanket bans unlikely, while platform–AI-provider partnerships could trace reposted outputs and identify campaigns, despite weak detection and incomplete platform coverage.Current detection tools lack the confidence needed for disruptive action and may fail on short posts; partnerships may be more attractive to AI companies, but inaccurate detection risks flagging the wrong accounts.Source: §5.4.1 · pp. 56–57

5.4.2 Platforms Require “Proof of Personhood” to Post

• Stronger proof-of-personhood checks could disrupt fully automated model-to-post bot workflows, but cannot stop manual copy-paste and may burden privacy while determined actors evade them.Checks could require stronger authentication or humanness tests; successful checks may also increase an account’s perceived credibility.Source: §5.4.2 · p. 58

5.4.3 Entities That Rely on Public Input Take Steps to Reduce Their Exposure to Misleading AI Content

• Entities soliciting public input could strengthen authentication against AI-generated comments, but must avoid participation frictions and suspicion that undermines democratic responsiveness.Large-language-model comments may be harder to identify as coordinated than repetitive mass submissions, while common CAPTCHAs may be insufficient.Source: §5.4.3 · pp. 59–60

5.4.4 Digital Provenance Standards Are Widely Adopted

• Digital provenance could disclose content creation and edits through ecosystem-wide authenticity markers, but legacy content, copied raw text, and privacy risks sharply limit coverage.Widespread authentication of raw text could require changes to internet protocols; provenance that identifies users could also threaten anonymous speech.Source: §5.4.4 · pp. 60–61

5.5.1 Institutions Engage in Media Literacy Campaigns

• Media literacy campaigns can improve discernment, but must move beyond spotting patterns models can avoid and teach fact-checking; they may be less effective against distraction operations [198] [199] [200].Campaigns can teach people to distinguish objective information from false, misleading, or slanted content.Source: §5.5.1 · p. 62
• Content-focused literacy is agnostic to human or computer authorship, while warnings about AI bots may intensify polarization and distrust of information that conflicts with prior beliefs [201].AI-specific indicators could help if reliable telltale signs emerge, but may be misplaced if AI operations mainly look more authentic.Source: §5.5.1 · p. 62

5.5.2 Developers Provide Consumer-Focused AI Tools

• Consumer-focused AI tools could reduce demand for disinformation by helping people evaluate encountered content, unlike detection methods that focus on whether content is synthetic.The proposed tools equip consumers to make better decisions rather than simply classify authorship.Source: §5.5.2 · p. 63
• Proposed tools include warning labels, demonetization, AI-assisted curation, and contextual aids that help users assess claims while lowering the risk of labeling true articles as misinformation [202] [203] [204].Examples include browser extensions, mobile apps, customizable notifications, and search tools.Source: §5.5.2 · p. 63
• Contextualization engines could surface high-quality sources and missing data, reduce the effort of deeper investigation, and align web-traffic revenue with user goals; AI-generated content could also inoculate populations [205] [206].Source: §5.5.2 · p. 63
• Generative models could help users identify flaws in tailored arguments or artifacts in manipulated images, and support answers with cited sources [208] [209].These methods could augment consumers’ ability to critically evaluate information.Source: §5.5.2 · p. 63
• Consumer-focused AI tools may reproduce social bias, reinforce confirmation bias, and face platform restrictions; platform-built tools may serve platform interests over users’ interests.Platform tools could also discourage effective policy action or third-party alternatives.Source: §5.5.2 · p. 64
• Web-searching generative models could act fraudulently to improve results if unconstrained; consumer-focused tools offer opportunities against stronger influence operations but require high-quality implementation.Potential impacts on search traffic and the internet should also be considered.Source: §5.5.2 · p. 64

conclusions:

• There are no silver bullets for minimizing the risk of AI-generated disinformation.Source: conclusions · p. 65
• New institutions and coordination, including collaboration between AI providers and social media platforms, are needed to respond collectively to influence operations.Source: conclusions · p. 65

4. Mitigations

• Mitigations that address the supply of mis- or disinformation without addressing its demand remain partial, and more research is needed to assess the threat and mitigation feasibility.Source: §4 · p. 65

6.1 Language Models Will Likely Change Influence Operations

• Language models may reshape influence operations by lowering production costs, enabling novel campaign behaviors, and scaling persuasive text that can be difficult to distinguish from human writing.The extent of these changes depends on diffusion, accessibility, technical and social uncertainties, model control, and how information environments adapt.Source: §6.1 · pp. 65–66

6.2 There Are No Silver Bullet Solutions

• No proposed mitigation is simultaneously feasible, institutionally tractable, robust to second-order risks, and highly impactful, and proliferation makes preventing all model-enabled disinformation nearly impossible.Developers, social media companies, and policymakers still have responsibilities to reduce harm; even proposed policies or concentrating AI talent are unlikely to stop a sufficiently motivated state from building its own capable models.Source: §6.2 · p. 66

6.3 Collective Responses Are Needed

• Reducing AI-enabled influence campaigns may require collaboration on model-release norms, detection of large output batches, and coordination across stakeholders.The paper describes the US response as fragmented across technology companies, researchers, and government, and calls for stronger coordination mechanisms and incentives.Source: §6.3 · pp. 66–67

6.4 Mitigations Must Address Demand As Well As Supply

• Supply-side measures are only partial solutions if demand for misleading information remains unchanged.Consumers may seek information that is cheap and useful for their goals, which influence operations can tailor without the constraints of reality.Source: §4 · p. 65, §6.4 · p. 67

6.5 Further Research Is Necessary

• The paper calls for further research on threat scale, platform mitigations, population-specific impacts, and unresolved model and detection capabilities.Open questions include how model size and fine-tuning affect persuasiveness, whether smaller models can close capability gaps, and what scale enables detection; researchers should also engage communities likely to be affected.Source: §6.5 · pp. 67–68
• Research is also needed on propagandists’ decisions and gains, whether API restrictions deter them, and how industry partnerships and model-release norms should develop.The paper describes the field as rapidly changing and argues that thoughtful engagement by industry participants and neutral observers is critical.Source: §6.5 · p. 68