Research from OpenAI, Georgetown, and Stanford shows that LLMs can drive public opinion manipulation

Advances in AI-powered large language models promise new applications in the near and distant future, where programmers, writers, marketers, and other professionals can take advantage of advanced LLMs. But one new study by scientists from Stanford University, Georgetown University and OpenAI highlight the impact LLMs can have on the work of actors seeking to manipulate public opinion through the dissemination of content online.

The study finds that LLMs can boost political influence operations by facilitating content creation at scale, reducing labor costs and making it more difficult to detect bot activity.

The research was conducted after Georgetown University’s Center for Security and Emerging Technology (CSET), OpenAI, and Stanford Internet Observatory (SIO) co-hosted a workshop in 2021 to explore the potential misuse of LLMs for propaganda purposes. And as LLMs continue to improve, there are fears that malicious actors will have more incentive to use them for nefarious purposes.

Research shows that LLMs influence actors, behaviors and content

Influencing operations are determined by three key elements: Actors, behaviors and content. The study by Stanford, Georgetown and OpenAI shows that LLMs can influence all three aspects.

Now that LLMs make it easy to generate long stretches of coherent text, more actors will find it attractive to use them for influence operations. Content creation previously required human writers, which is costly, poorly scalable, and can be risky when actors try to hide their activities. LLMs are not perfect and can make stupid mistakes when generating text. But a writer in conjunction with an LLM can become much more productive by editing computer-generated text rather than writing from scratch. This makes the writers much more productive and reduces labor costs.

“We argue that for propagandists, language generation tools are likely to be useful: they can reduce content generation costs and reduce the number of people required to create the same volume of content,” Dr. Josh A. Goldstein, co-author of the paper and research fellow on the CyberAI project at CSET, told VentureBeat.

In terms of behavior, LLMs can not only drive current influence operations, but also enable new tactics. For example, adversaries can use LLMs to create dynamic, personalized content at scale or create conversational interfaces such as chatbots that can communicate directly with many people at once. LLMs ability to produce original content also makes it easier for actors to hide their influence campaigns.

“Because text generation tools create original output every time they are run, campaigns that rely on them can be harder to spot for independent researchers because they don’t rely on so-called ‘copypasta’ (or copied and pasted text that is everywhere). is repeated on the Internet). bills),’ Goldstein said.

We don’t know much yet

Despite their impressive performance, LLMs are limited in many critical ways. For example, even the most sophisticated LLMs tend to make absurd statements and lose coherence as their text grows beyond a few pages.

They also lack context for events not included in their training data, and retraining them is a complicated and costly process. This makes it difficult to use them for political influence campaigns that require commentary on real-time events.

But these limitations don’t necessarily apply to all types of influence operations, Goldstein said.

“For operations that use longer text and try to convince people of a particular story, they may be more important. For operations that mainly try to ‘flood the zone’ or distract people, they may be less important ,’ he said.

And as technology matures, some of these barriers may be lifted. For example, Goldstein said the report was primarily prepared before the release of ChatGPT, which showed how new data collection techniques and training techniques can improve the performance of LLMs.

In the paper, the researchers predict how some of the expected developments could remove some of these barriers. For example, LLMs will become more reliable and useful as scientists develop new techniques to reduce their errors and adapt them to new tasks. This may encourage more actors to use them for influence operations.

The authors of the article also warn of “critical unknowns.” For example, scientists have found that as LLMs get bigger, they show emerging capabilities. As the industry continues to push for larger-scale models, new use cases may emerge that can benefit propagandists and influence campaigns.

And with more commercial interest in LLMs, the field is bound to move much faster in the coming months and years. For example, the development of publicly available tools to train, run and refine language models will further reduce the technical barriers to using LLMs for influence campaigns.

Implementing a kill chain

The authors of the paper propose a “kill chain” framework for the types of mitigation strategies that can prevent the misuse of LLMs for propaganda campaigns.

“We can begin to address what it takes to counter abuse by asking a simple question: What does a propagandist need to successfully conduct a language model influence operation? From this perspective, we identified four points for intervention: model construction, model access, content dissemination, and belief formation. A range of possible measures exists at each stage,” Goldstein said.

For example, in the construction phase, developers can use watermarking techniques to make data created by generative models discoverable. At the same time, governments can impose access controls on AI hardware.

In the access phase, LLM providers can impose stricter usage restrictions on hosted models and develop new standards around model release.

In terms of content distribution, platforms offering publishing services (e.g. social media platforms, forums, e-commerce websites with rating features, etc.) may impose restrictions such as “proof of personality”, which will make it difficult for an AI-powered system to submit content at scale.

While the paper provides several such examples of mitigation techniques, Goldstein stressed that the work is not complete.

“Just because a mitigation is possible doesn’t mean it should be implemented. Those in a position to implement — whether in technology companies, in government or researchers — must assess its desirability,” he said.

Some questions that should be asked are: Is a mitigation technically feasible? Socially feasible? What is the downside risk? What impact will it have?

“We need more research, analysis and testing to better address what mitigations are desirable and to highlight mitigations that we have overlooked,” Goldstein said. “We don’t have a miracle solution.”