Medical research fraud is alarming.

BEN MOL was a Monash University professor of obstetrics, in Melbourne. He came across a notice to retract a study about uterine fibroids, and infertility that had been published in Egypt in 2011. The journal which had published it was retracting it because it contained identical numbers to those in an earlier Spanish study—except that that one had been on uterine polyps. It was discovered that the author had just copied some parts of the paper from the polyps and modified the disease.

“From that moment I was alert,” says Dr Mol. His alertness did not limit to his reading of published papers. At the time, he was editor of European Journal of Obstetrics and Gynaecology and was often also a peer-reviewer for papers submitted in other journals. Two papers with allegedly fake data landed quickly on his desk. He refused to accept them. He found them again a year later. However, the data was skewed and published in another journal.

He has since teamed up to research groups of papers from authors he identifies as data fabricators. Fire was everywhere he looked. There were tables on patients’ characteristics that contained only even numbers. Clinically impossible values were found. The sex ratio of babies was an unbelievable 40:60, when the mothers-to be had been chosen randomly. It was not uncommon for clinical trials to be completed at such incredible speeds.

Unreality is created

Dr Mol and his fellows have written to journals with concerns over more than 750 papers. Unfortunately, the investigations can take many years and sometimes nothing is done. Only 80 of the flagged studies have been retracted. Worse, many have been included in systematic reviews—the sort of research round-ups that inform clinical practice.

Many millions of patients may be wrongly treated. One example is the steroid injections that were given to elective Caesarean section women to give birth to their babies. These injections are designed to prevent newborns from having breathing problems. There is a worry that they might cause damage to a baby’s brain, but the practice was supported by a review, published in 2018, by Cochrane, a charity for the promotion of evidence-based medicine. When Dr Mol and his coworkers reviewed the review, they discovered three studies they considered unreliable. These three studies were not included in the revised review published in 2021.

Due to the diligence of Dr Mol, increasingly large numbers of partially or fully fabricated papers are being discovered. Retraction Watch, an online repository that tracks papers on biomedical science topics, has nearly 19,000 papers that have been removed (see chart 1). In 2022 there were about 2,600 retractions in this area—more than twice the number in 2018. While some were honest errors, the vast majority of retractions in this area were caused by misconduct (see chart 2).

John Carlisle is a British journal editor and anaesthetist who has discovered hundreds of problematic papers. He reckons “almost all [clinical] guidelines will be affected in some way by false data that is currently unrecognised”. False data can also lead to blindness for those trying to build on them, costing millions.

Journals can take many years to retract if they are ever successful. This means that one out of 1,000 papers will be retracted. It doesn’t sound so bad. However, Ivan Oransky, one of Retraction Watch’s founders, reckons, based on various studies of the matter and reports from sleuths, that something more like one in 50 papers has results which are unreliable because of fabrication, plagiarism or serious errors.

The majority of fabricated papers are from one or both of these sources. Some papers, particularly those that claim to report clinical trials are often the work of individual fraudsters, or groups of fraudsters. These are the ones Dr Mol found. Others, more often purporting to be about basic science, such as molecular biology, are written, for a fee, by outfits known as “paper mills”. These are often fabricated by copying published papers, and substituting the gene/disease a legitimate paper refers too with another.

Retraction Watch data shows that over 25% of 19,000 retractions are attributed to 200 authors. Senior scientists from large hospitals and universities are some of the most prolific fraudsters. Many have not spoken out about the motivations behind their research. However, the confessions of those who have made this disclosure speak volumes about the motivations for others to follow the same path.

Cheats ever prosper

Scientific research is a hard slog, and the results are often disappointingly equivocal answers to a researcher’s bold hypothesis. Diederik Stappel, a former psychology professor at Tilburg University (the Netherlands), had 58 papers pulled after revelations about his fabrications. He did so, he said in his confessional memoir, “Faking Science”, because they otherwise failed to provide a solution that was “simple, clear, beautiful and elegant”. He was driven by career opportunities, like many other fabulists. The aphorism “publish or perish” reflects an academic truth. For promotion or to switch institutions in order to obtain a better job, a long publication list is essential. But journals prefer studies that show strong, positive results—meaning much of a researcher’s effort is, in terms of career advancement, wasted.

Publishing lots of studies has enabled some fraudsters to be perceived as leading experts in their fields, with the perks of professional prestige which go with that—at least until things unravel. Dorothy Bishop, an Oxford University retired experimental psychologist who helps identify problematic studies, observes that fraudsters often lead research groups or collaborate with other research centers. These colleagues suffer the consequences of the fraud when it is revealed. “Quite often”, she says, “junior people get their careers completely derailed by this.”

It is particularly problematic in China. Many hospitals require unrealistic publishing quotas to be eligible for the highest jobs. Publishers who publish in top journals receive huge cash bonuses. China, a country with a strong paper-mill sector, is no surprise. Nearly all papers retracted from paper mills have Chinese authors. About a fifth of papers submitted to journals include at least one Chinese contributor, but this fifth account for almost half the papers later retracted.

The problem goes beyond China, as a review of submissions made to two journals targeted at paper mills shows. The fakes were submitted by authors from over 70 countries. Even though middle income countries (China included) are the most common, many of the most dangerous fabrications of clinical trial results have been discovered in America, Canada or Europe.

How much cheating is never caught is anyone’s guess. As Dr Bishop notes, “The only ones we know about are the ones that are not very good at it. If somebody is very good at fraud, you’re not going to detect it.” And it is not just a few bad apples. PLOS ONE published 18 surveys in 2009 that asked scientists about fraud. The majority of these were from the United States. Only 2% of the respondents admitted to falsifying data, but 14% claimed they had known someone who did. Similarly, a third of those asked confessed to other questionable research practices, such as dropping inconvenient data points based on “gut feeling”, or making important changes to a study’s protocol while it was in progress. However, 72% of the respondents pointed the finger at their colleagues.

This is not a rare occurrence. A 2016 survey of British academics found that nearly one in five had falsified data. In a survey conducted in the Netherlands, 10% reported that they had falsified data or fabricated data.

Fake papers can be used to support treatments that have been supported by other research. These are unlikely to influence clinical practice. However, some fraudsters inspired medical interventions that have proven to be ineffective or even harmful.

To boost blood pressure, patients in critical condition who were undergoing surgery received starch infusions. Joachim Boldt (a German anaesthesiologist) was part of the seven discredited studies that led to this conclusion. After his fabrications were exposed, a revised summary of the evidence was published in the Journal of the American Medical Association in 2013. It concluded that starch infusions given in such circumstances could cause kidney damage, and even lead to death.

Likewise, for more than a decade cardiac patients in Europe were given beta-blockers before surgery, with the intention of reducing heart attacks and strokes—a practice that rested on a study from 2009 which was eventually determined to have been based, at least in part, on fabricated data. This approach could have led to 10,000 deaths in Britain every year, according to one estimate. A systematic review showing that high-dose sugar solution infusion reduces mortality following head injury was discredited after an investigation failed find any evidence that any of its trials, all of which were attributed to the same researcher, actually took place.

Bricks without straw

It is not difficult to spot fake papers. You can catch one by accident and then look at the others with the same hand. Dr Mol did it. “A lot of the problems are only apparent when you look across publications by an author. So if you had one paper, you wouldn’t necessarily identify them,” says Alison Avenell of the University of Aberdeen, who is part of a team that has raised concerns about hundreds of clinical trials published by several research groups. Dodgy papers can be identified by unusually large effects of reported treatments, unusual dropout rates among participants or similar data appearing in different trial reports. “You see things that are virtually statistically impossible,” she says.

Dr Carlisle, along with others, has devised a variety of statistical checks to catch unusual number in single papers. These checks were used by Dr Carlisle as editor of Anaesthesia for all 526 clinical-trial papers submitted to Anaesthesia between 2017 and 2020. When he looked at papers for which the authors had not submitted individual patients’ underlying numbers, he found that 2% included false data. This jumped to 44% for those for which he could also examine such individual numbers, including 26% which he judged as “fatally flawed”.

This routine review could help stop many fake papers being published. Yet, though reviewers who vet journal papers for publication “get all of these supplemental files and links to data sets”, says Lisa Bero, of Cochrane, “most of them don’t look at it.” Moreover, though authors often say that the backup data for their papers will be made available on request, a study conducted in 2022 reported that 93% fail to provide them when so requested by other researchers. “You ask for the original data and then, my goodness, you realise we live in a dangerous world,” says Ian Roberts of the London School of Hygiene and Tropical Medicine. Floods, earthquakes, termites, stolen laptops—he has seen all sorts of excuses for lost data.

Blots all over the landscape

Cochrane keeps a list of retracted papers and systematic reviews are updated as necessary. This is a rare type of review, partly because retracted articles are not often flagged in online libraries as such and continue to be cited as valid research. Researchers are often reluctant to clarify things, even when they learn that the reviews were based on fabricated studies. The attitude is, “it was right at the time”, says Jack Wilkinson, a biostatistician at Manchester University.

Dr Avenell and her colleagues evaluated the impact of 27 retracted papers from clinical trials, which covered a variety of medical conditions. These papers were included within 88 systematic reviews as well as clinical guidelines. According to the researchers, half of these reviews had conclusions that could be changed if the retracted trials are removed. They notified all authors of reviews about the retractions. Only half of them responded. One year later, 39 of 44 reviews were affected by no action.

In recent years, similar depressing discoveries were made about paper-mill articles. Although many of these articles are so poor that any serious researcher would consider them a joke and are published in obscure periodicals, which will print anything for a fee. However, some appear solid enough to be accepted into leading scientific journals. Now, they are discovering that hundreds of them have been published. An analysis of 53,000 papers submitted for journals in various disciplines by six publishers revealed that between 2% to 46% of the papers were suspicious.

Molecular-biology articles, those paper-mill specialities, often include pictures of Western blots—a laboratory technique used to study proteins. These images are often found in papers that examine the effects of a drug on human cells. Because no two Western blots are the same, a sharp pair can spot duplicates. Elisabeth Bik (a Dutch microbiologist) is an outstanding detective who hunts for fake papers. Using her “pattern-matching eyes and lots of caffeine”, as she put it in a recent opinion piece, she has analysed more than 100,000 papers, and found blot-based evidence of error or cheating in 6,500 of them.

Studies using falsified Western blots might seem less significant than those involving fabricated clinical trial. But consider a recent controversy concerning a series of influential papers on the causes of Alzheimer’s disease of which the lead author is Sylvain Lesné of the University of Minnesota (which is investigating the matter).

Dr Bik and other researchers have found evidence of image manipulation, including images of blots in these studies. And other scientists have tried and failed to replicate the results of a landmark study Dr Lesné published in Nature in 2006, which appeared to provide crucial evidence concerning a hypothesis that links the disease with so-called amyloid plaques in the brain, and which is one of the most cited papers in Alzheimer’s research. This article, which pointed out one form of amyloid beta, a plaque-forming proteins, may have set off a wave of scientific inquiry that could lead to a complete reversal of the hypothesis. Nature published a concern statement in July 2022 while it investigates.

Similar patterns are emerging for genetics. In a study published last year Jennifer Byrne, a cancer researcher at the University of Sydney, led a team that screened nearly 12,000 papers using Seek & Blastn, a piece of software they developed themselves, to check details of substances called nucleotide-sequence reagents (NSRs).

NSRs are small pieces of DNA and RNA that make up part of many genetic studies. They can bind to certain bits of natural genetic material. Seek & Blastn extracts the sequences of these reagents from papers and compares them with those in Blastn, a public sequence database, to check whether they match their supposed genetic targets. The paper-mill production was found to be a factor in 6% of the errors discovered by the team.

Fake genetic studies like these can have as devastating consequences as the ones involving Western blots. Dr Byrne and her co-authors estimate that as many as 100,000 published papers about human genes may emanate from paper mills, and that a quarter of these are such that they “may misinform the future development of human therapies”.

The discipline should be studied

Science is often said to be self-correcting. True, science can be self-correcting. If a claim is sufficiently important, it is likely that inability or subsequent work to match it will eventually be noticed. It is possible to hide behind the scenes in the short-term. Even co-authors of a data-fabricating scientist—those, in other words, who are closest to him or her—may not notice what the culprit is up to. In complex studies of a particular disease, several types of researchers will be involved, who are, by definition, not experts in each other’s fields. As Dr Bishop observes, “You just tend to take on trust the bits of data that somebody else has given you.”

Self-correction is often not effective even when a paper has been flagged for containing false data. There is a “massive variation” in how journals and publishers react when alerted to problems, says Dr Wilkinson—whether they investigate at all, how long it takes to reach a decision, and what they do after that.

Dr Avenell says that it takes about two to three years for a publication to be published. “The only way we get retractions is to repeatedly badger the journals over and over and over again”, she says. “If you just send them a one-off email with details of all of your concerns, it’s very unlikely that anybody is ever going to do anything.” Dr Bero says that many Cochrane reviewers think it a waste of time to alert journal editors about problematic studies they have identified, so they don’t bother. “Many of our authors have written to editors and just gotten no response, or the editor doesn’t do anything to retract the article or doesn’t investigate at all,” she observes.

Journals are often lacking the necessary staff, such statisticians, to handle such matters. For their part, publishers make more by publishing more than they do from investigating retractions. They fear being sued for fraud by belligerent crooks. They often give the ball to institutions that employ suspected fraudsters.

Institutions that receive federal grant money in America (which makes up almost all of the serious ones), must follow government guidelines. They are required to investigate allegations of research misconduct within 60 calendar days. C.K. states that some institutions take much longer than two years to complete their investigations. Gunsalus, an University of Illinois ethicist, has carried out such investigations.

Do it!

There are sometimes good reasons to delay. One of the difficulties is finding competent experts to perform complex analyses. Another problem is the possibility that further suspicious publications may be discovered by investigating a paper. All too often, though, “there is very little sense of urgency”, says Dr Gunsalus. It is not about whether a researcher should lose their job, but rather to preserve the integrity of scientific literature. The university generally remains mum until an employment decision is made.

Dr Gunsalus says that even in America, there is often a lack of quality in these investigations. Many places don’t have the right regulations. Dr Bishop says that research misconduct can sometimes lead to people being fired in Western countries. Andrew Grey of University of Auckland in New Zealand and his colleagues noted in their concerns regarding 172 papers published at the Kashan University of Medical Sciences in Iran by Zatolleh asemi, a group that was led by Zatolleh.

Universities do not offer incentives to whistle-blowers, although that may stop fake studies from being made. The rules often contain a lot about the consequences for spurious allegations but not much about the duty to raise fraud alarms.

Few whistle-blowers have Dr Bik’s perseverance. After raising concerns about Didier Raoult’s 60 papers at Marseille’s university hospital, she was subject to online threats. A colleague posted her home address on Twitter. Dr Raoult also filed a legal complaint against her for attempted blackmail and harassment, an action the French National Centre for Scientific Research, a government agency, condemned as “judicialisation of scientific controversy and criticism”. In 2021 she won the John Maddox prize—awarded jointly by Sense about Science, a British charity, and Nature, of which Maddox was, for many years, editor—for “courage and integrity in standing up for sound science and evidence”.

There are signs that there is some hope. Cochrane has begun to verify papers for integrity. One in four studies that were included in a recent Cochrane review on research on drugs designed to prevent preterm labour exclusions was based on this reason.

In a similar vein, Britain’s National Institute of Health and Care Research pays for INSPECT-SR, a project run by Dr Wilkinson that is developing integrity-checking tools for systematic reviews like those conducted by Cochrane. STM, an association representing the publishing industry, is developing a similar system to detect fraud in submitted papers. Joris van Rossum is the leader of this project. The system will alert editors about suspicious features used by paper mills like duplicated images, papers being submitted to multiple journals simultaneously, and authors who are not experts in the subject.

Dr Wilkinson is concerned that there will be an arms race between the fabricators and those caught. Dr van Rossum is also worried. Such fears are amplified by the growing sophistication of artificial-intelligence (AI) tools like ChatGPT. In a recent study for which this was asked to generate 50 medical-research abstracts, both human reviewers and an AI-output detector failed to identify a third of the bot’s papers as such.

Trust. Verify!

Fake scientific records can be kept out of science if publishers are willing to invest more resources. Statistics checks for clinical trials papers often require tedious manual work such as entering data into spreadsheets. This would mean journals would need to hire dedicated staff, which could reduce profits.

Many academics who have struggled for years to retract fabricated papers are skeptical that newer methods of detecting fraud will make a significant difference. Dr Roberts and Mol believe journals should be regulated in the same way social media and news businesses are in certain countries. They should also have standards for what they publish. Peter Wilmshurst, an American cardiologist, has spoken out against research misconduct in his area and called for criminal penalties. Dr Gunsalus demands that universities make public reports from research-fraud investigations. All agree that publishing is the best way to end up in disaster.

None of these solutions are easy or quick. However, it is obvious that the choice to ignore patients is causing them serious harm. As Stuart Richie of King’s College, London, notes in his book, “Science Fictions”: “For the sake of the science, it might be time for scientists to start trusting each other a little less.”

© 2023 The Economist Newspaper Limited. All rights reserved.

Published under license by The Economist. You can find the original content at https://www.economist.com/science-and-technology/2023/02/22/there-is-a-worrying-amount-of-fraud-in-medical-research