FEATURED
Scholars say situation adds to larger concerns about AI usage producing high volume, low quality articles
Scholars are raising questions about AI usage and academic quality after University of Chicago statistician Nicholas Polson wrote more than 200 academic papers this year using AI.
“The question needs to be: do these add anything of value and should we rely on them or not?” Ivan Oransky told The College Fix in a recent interview. Oransky is a co-founder of Retraction Watch, a website that tracks academic retractions, and a New York University journalism professor.
A tenured professor, Polson’s papers primarily focus on advanced math and statistics such as statistical parameterization and financial markets. Some of his research highlighted on the University of Chicago’s website includes measuring the “energy cost” of using AI and analyzing the odds of cheating.
Recently, however, concerns about the large number of articles Polson was producing prompted SSRN, a pre-print server for academic papers, to retract 253 papers that he wrote or co-wrote within the past year, The Washington Post reported.
In comparison, he wrote 10 published papers in 2018, 14 in 2017, and 12 in 2016, according to the “Research” page on his website.
When asked why it removed his papers, SSRN told the Post, “The most significant issue was the unusual submission activity.”
Polson’s name also no longer appears in search results on SSRN, The Fix found.
The Fix contacted Polson and the University of Chicago’s media relations office for comments twice via email asking about the concerns with his publication work over the past year, but neither responded.
On SSRN, papers do not have to be peer reviewed or published in academic journals to appear on the server. The site is designed as a platform to “rapidly share preprints and other early-stage research” and garner feedback from other scholars. The publisher does require scholars to disclose if they used AI.
Polson acknowledged using AI in an interview with the Post, saying it makes “the grunt work” easier.
“Clearly AI makes a productive researcher far more productive,” he said.
However, the statistician did not say if “he disclosed the use of AI in any of the papers he published” on SSRN this year, according to the newspaper.
Polson also defended the quality of his writings, saying he works hard to produce his research.
He told the Post that academics who have raised complaints against him might be motivated by “envy and a desire to poach my process.”
But scholars who spoke to The Fix and the Post said the situation highlights a larger concern about AI being used to produce an overwhelming amount of papers that journals cannot adequately review.
Responding to Polson, Andrew Gelman, a statistician at Columbia University, told The College Fix that he is “not sure what people are envious of.”
“Just about anyone could get a bot to write 258 papers and post them on a preprint server,” he said in an interview last week.
Gelman was one of the first scholars who took notice of Polson’s unusual activity and wrote about it in a Columbia University statistics blog on Aug. 27.
He said he grew concerned about the quality of Polson’s work while reviewing one of his many recent papers.
In the blog, Gelman wondered if Polson’s papers possibly could be an experiment or even a joke.
Not opposed to the use of AI, Gelman told The Fix that he knows colleagues who use chatbot-generated code and produce excellent work. He said the issue is quality research.
“It’s not just that a chatbot can produce slop, but that it enables the human production and promotion of slop,” he said to The Fix.
On the broader issue, an editor of Organization Science said her journal has received an overwhelming number of academic papers since AI became widely available.
“There’s going to be a flood of poor-quality research that the system is not equipped right now to handle,” senior editor Claudine Gartenberg told the Washington Post.
Gartenberg said the number of submissions to her publication has doubled since 2022.
Similarly, data from Elsevier, an academic publisher that runs SSRN and several academic journals, shows that submissions to its publications have increased by 1.5 million in the same time period, according to the newspaper.
Meanwhile, Oransky of Retraction Watch said it would be better to look at the system than Polson’s specific case.
“The question is why did he feel the need to publish 200 something papers? That’s about publish or perish. It’s always more useful to look at the fundamental reasons of why something is happening,” he told The Fix.
Publish or perish refers to the pressure scholars say academia puts on them to publish a certain amount of academic research and journal articles.
Polson is teaching two classes this fall at the University of Chicago’s Booth School of Business, “Bayes, AI and Deep Learning” and “Business Statistics.”
MORE: ‘Zombie’ retracted papers continue to infect future research, scholars warn
