Pages Navigation Menu

SHOWFUN - Show & Fun & More!

Algorithm-generated nonsense reviews fool actual human peer reviewers one in four times

Every so often, scientists need a little picking on, too. Especially when the well-respected peer reviewing process goes horribly wrong.

About a decade ago, some grad students from MIT came up with a project that started as a shot over the bow at computer scientists and their ever-increasing use of buzzwords and jargon. They made a program called SCIgen that uses a Markov chain to stochastically (randomly) sample out clauses from a body of text called a “corpus.” It then arranges the words in a syntactically correct way, which nevertheless reads like a name-dropping version of The Jabberwocky, crossed with Mad Libs. “One useful purpose for such a program,” they suggest, “is to auto-generate submissions to conferences that you suspect might have very low submission standards.”

Feeding their Markov generator a whole bunch of peer-reviewed computer science papers resulted in an article that eventually got past enough reviewers that the students got one such conference to accept it for presentation. “Our plan was to go there and give a completely randomly generated talk, delivered entirely with a straight face,” they remark on their website, but someone got wind of their plot and disinvited them from presenting by refunding their fee. The reason given, and it’s a lousy one, is that their paper was fake so they shouldn’t be allowed to present it. (They went anyway, and delivered the aforementioned talks.)

Building on SCIgen and programs like it, researchers have now taken another shot at the peer reviewing process. They found that feeding a Markov chain a corpus of publications from the biosciences resulted in faux reviews that fooled actual reviewers so often it’s scary. The researchers made randomly generated reviews of actual published articles using their iteration of a Markov chain, and then gave them to reviewers, slipped into the stack like ringers between actual reviews.

While their algorithm “cannot possibly deceive any rigorous editorial procedure,” the authors remark, it sure does work if all the reader does is skim.  Their frankly damning results are published in a paper entitled “Your Paper Has Been Accepted, Rejected or Whatever: Automatic Generation of Scientific Paper Reviews.” Canned, generic phrasing can join forces with time pressure, and end up with a reviewer’s bleary eyes glossing over the content like a TA grading homework at 4AM. Reviews generated by the authors’ algorithm could fool actual humans one time in four.

Influence over the peer review process is real power, and not one that should be used lightly or for reasons of financial gain. But there are hundreds of “predatory publishers” that do only loose review of the papers they charge to publish, and sometimes make no review at all. The veneer of credibility that an author gets by being published in a peer-reviewed journal starts to evaporate as soon as the peer review process stops being trustworthy.

What does it say to skeptics of the reviewing process, when the process can be thrown by an algorithm — and not even something done on a supercomputer, just something three grad students hacked together as a joke? Come on, guys. This is one field where we’re really not ready for the computers to beat us.

Leave a Comment

Captcha image