OpenAI confirmed that a now-banned X account which went viral in June for appearing to be a Russian troll using ChatGPT was a hoax that likely originated in the U.S.
The account in question had frequently posted in support of President Donald Trump before spitting out an error message partially in Russian that indicated its ChatGPT credits expired.
Many social media users were quick to point to the incident as evidence of Russian interference ahead of the 2024 presidential election—though others argued the viral post did not seem quite right.
Now, OpenAI—in its October report on influence and cyber operations—has dismissed concerns about the account serving as a Russian troll.
“Our investigation showed that this post was a hoax which could not have come from our models,” the report states. “However, earlier posts made by the same X account were generated using our models, apparently in an attempt to bait controversy. This activity likely originated in the United States.”
While the post in question was not AI-generated, OpenAI reported that the troll account used its models to generate adversarial counter-argument replies for X on topics ranging from “fantasy gaming through motorcycles to arguments about whether the world is flat.”
“This was an unusual situation, and the reverse of the other cases discussed in this report,” OpenAI concluded. “Rather than our models being used in an attempt to deceive people, likely non-AI activity was used to deceive people about the use of our models.”
OpenAI banned the user’s account, it said. The user’s X account was also banned.
While that particular account may not have been part of a Russian influence campaign, OpenAI did say that its tech was recently used in Russian influence operations.
The latest report makes one reference to a Russian-origin influence campaign dubbed Stop News, which used its programs to generate comments, articles, and images. OpenAI does not believe the operation was aimed at spreading fake news, but rather drawing attention and generating clicks.
OpenAI has previously highlighted multiple instances of Russian-origin accounts using its programs with nefarious intentions.
“In this year of global elections, we know it is particularly important to build robust, multi-layered defenses against state-linked cyber actors and covert influence operations that may attempt to use our models in furtherance of deceptive campaigns on social media and other internet platforms,” OpenAI said in its latest report.
Internet culture is chaotic—but we’ll break it down for you in one daily email. Sign up for the Daily Dot’s web_crawlr newsletter here. You’ll get the best (and worst) of the internet straight into your inbox.
