The best text-generating AI models could turbocharge disinformation campaigns

·2 min read

A new report lays out the ways that cutting-edge text-generating AI models could be used to aid disinformation campaigns.

Why it matters: In the wrong hands text-generating systems could be used to scale up state-sponsored disinformation efforts — and humans would struggle to know when they're being lied to.

Stay on top of the latest market trends and economic insights with Axios Markets. Subscribe for free

How it works: Text-generating models like OpenAI's leading GPT-3 are trained on vast volumes of internet data, and learn to write eerily life-like text off human prompts.

  • In their new report released this morning, researchers from Georgetown's Center for Security and Emerging Technology (CSET) examined how GPT-3 might be used to turbocharge disinformation campaigns like the one carried out by Russia's Internet Research Agency (IRA) during the 2016 election.

What they found: While "no currently existing autonomous system could replace the entirety of the IRA," algorithmically based tech paired with experienced human operators produces results that are nothing less than frightening.

  • Like many other automation and AI technologies, GPT-3's real power is in its ability to scale, says Ben Buchanan, director of the CyberAI Project at CSET and a co-author of the report.

  • GPT-3 "lets operators try a bunch of variants on a message and see what sticks," he says. "The scale might lead to more effective feedback loops and iterations."

  • "A future disinformation campaign may, for example, involve senior-level managers giving instructions to a machine instead of overseeing teams of human content creators," the authors write. "The managers would review the system’s outputs and select the most promising results for distribution."

What to watch: While OpenAI has tightly restricted access to GPT-3, Buchanan notes that it's "likely that open source versions of GPT-3 will eventually emerge, greatly complicating any efforts to lock the technology down."

  • Researchers at Huawei have already created a Chinese-language at the scale of GPT-3, and plan to provide it freely to all.

  • Because identifying the latest computer-generated text is difficult, Buchanan says the best defense is for platforms to "crack down on the fake accounts" used to disseminate misinformation.

The bottom line: Like much of social media more broadly, the report's authors write that systems like GPT-3 seem "more adept as fabulists than as staid truth-tellers."

Like this article? Get more from Axios and subscribe to Axios Markets for free.

Our goal is to create a safe and engaging place for users to connect over interests and passions. In order to improve our community experience, we are temporarily suspending article commenting