[Explainer] US journal Science corrects “Persuasive AI” paper; main conclusions remain unchanged

IT

The U.S. science journal Science has announced a correction to a research paper published in September 2024 concerning “Persuasion AI.” While the correction was due to some errors in the data aggregation process, the main conclusion—that facts and evidence change people’s opinions—remains unchanged.

Background and Key Points of the Paper Correction by the US Journal Science

On October 22, 2026, the U.S. science journal Science published a correction notice regarding a research paper on “persuasive AI” published in September 2024. This paper was presented by a research team including American University and demonstrated that dialogue with AI utilizing large language models can reduce people’s confidence in conspiracy theories. This correction was made to rectify errors in data aggregation processing and some numerical notations that were discovered after publication. As a result of recalculating the data and conducting detailed verification by the research team and the editorial department of Science, it was confirmed that although some statistical values were modified, there were no changes to the main conclusion or the effect sizes themselves, which form the core of the paper: “AI can persuade people out of their conspiratorial beliefs and alter their opinions by engaging in fact- and evidence-based dialogue.” As shown in the figure below, research on persuasive AI has been attracting significant attention in recent years.

Figure 1

It is concluded that this is a voluntary correction to ensure the validity of the research methodology and the transparency of the analysis, and that the scientific value of the main findings remains unchanged.

The Astonishing Mechanism of Persuasive AI Revealed by a Study

In the original research paper published in September 2024, dialogue experiments with generative AI (such as ChatGPT) were conducted involving over 3,000 participants. In the experiments, participants had a conversation of several minutes with the AI about a conspiracy theory they believed in, during which the AI provided personalized, specific, and objective facts and refutation data against the participants’ claims. As a result, it was demonstrated that even for deeply rooted conspiracy theories, which were previously considered extremely difficult to persuade people out of, participants’ beliefs significantly diminished after interacting with the AI. More importantly, it was confirmed that this persuasive effect lasted not just immediately after the conversation, but over the course of several months. The study analyzed that a logical persuasion approach—presenting facts and logical grounds rather than mere emotional appeals—proved successful. Compared to persuasion by human experts, the persuasiveness of generative AI, which can construct precise information in real time tailored to an individual’s interests and questions, was shown to be equal to or greater than that of humans, sending a major shock through the scientific community.

Background of the Re-verification Process and Data Correction

The process by which errors in data descriptions or flaws in analysis are pointed out after the publication of an academic paper and corrections are made is positioned as part of the self-cleaning mechanism in science. In the case of this persuasive AI paper as well, re-verification proceeded after receiving comments from external researchers and the community following its publication. During the re-verification process, erroneous descriptions were found in the sample sizes and some statistical analysis figures presented in the charts and graphs, but it was revealed that these were not due to fraudulent acts such as data falsification or fabrication, but rather technical mistakes during the aggregation stage. In recent years, with the increase in papers using generative AI in the academic community, peer review standards for the accuracy and reproducibility of data processing have been tightening. The following figure illustrates the paper’s re-verification and academic publishing check system.

Figure 2

Science promptly issued a correction notice, explicitly stating that the paper’s overall reasoning and major conclusions remain fully valid even based on the corrected data. This transparent response of correcting errors and disclosing information has served as an important precedent for ensuring the credibility of research.

The Challenges of Persuasiveness and Academic Ethics in the Generative AI Era

The Impact of Persuasive AI on Society and Public Opinion Formation

While persuasive AI technology holds promise for positive applications such as correcting misinformation and educational activities, it also carries significant risks, including public opinion manipulation and the erosion of democratic processes. Research has shown that large language models are capable of wielding advanced grammatical structures and ethical language with ease, allowing them to exert persuasive power equal to or greater than that of humans. In particular, because they can generate massive volumes of messages optimized for individual interests and political preferences, there is concern that if abused for shaping public opinion in elections or spreading disinformation, they could exert a social impact on an unprecedented scale. Furthermore, concerns have been pointed out that AI might gain the trust of humans and then lure them into making inappropriate judgments or decisions involving security risks. The European AI Act and regulatory agencies in various countries are advancing stricter regulations on manipulative and誘導的 (inductive/manipulative) AI use that exploits human cognitive vulnerabilities, making where to draw the ethical boundaries of persuasion technology a focus of international debate.

The Rapid Expansion of Generative AI in Academic Writing and Publishing

Even in research and academic writing, the use of generative AI is rapidly expanding. While contributing to efficiency in literature searches, proofreading, and code generation, problems caused by inappropriate AI use are also becoming apparent. For example, a study has reported that AI-generated paper summaries are about 4.85 times more likely to overgeneralize the original data compared to those written by humans. In addition, issues such as mass production by paper mills and hallucinations citing non-existent fictitious literature are increasing, with a growing trend of cases leading to paper retractions and corrections. Major academic publishers (such as AAAS, which publishes the journal Science, Elsevier, and Springer Nature) strictly prohibit listing AI as an author and have established rules requiring the disclosure of its use in manuscript writing and peer-review processes. To maintain the reliability of scientific papers, strict human fact-checking of generative AI outputs and the ensuring of transparency are essential.

Maintaining Academic Credibility and Future Points to Watch

This recent series of corrections surrounding persuasive AI has once again highlighted the challenges and prospects facing the academic community in correctly evaluating advanced technology research results and returning them to society. Moving forward, there is a need not only to simply measure the persuasiveness and capabilities of AI, but also to establish technical and theoretical frameworks that clearly distinguish between persuasion based on rational grounds and unfair manipulation that exploits cognitive biases. Additionally, in academic publishing, the construction of ecosystems to maintain research integrity is underway, including the introduction of AI-generated content detection technology, the strengthening of peer-review systems, and the development of paper retraction databases. For readers and society as a whole, it is also extremely important to enhance information literacy to correctly understand the characteristics and limitations of information generated by AI. As AI technology continues to evolve by leaps and bounds, the central focus going forward will be on how to build a desirable relationship between humans and AI while balancing the pursuit of scientific truth with ethical governance.

Reference page

  • 【Durably reducing conspiracy beliefs through dialogues with AI】[suspicious link removed]

[#PersuasiveAI #ScienceMagazine #PaperCorrection #GenerativeAI #ArtificialIntelligence #AIEthics]

コメント

Copied title and URL