Preprinting Does Not Meet Science’s Duty of Care Responsibility to Society

Abstract

While acknowledging that peer review is imperfect, we argue that preprint servers, which disseminate non-peer-reviewed research, represent a significant risk to society. Preprints can lead to the spread of misinformation, undermine public trust in science, and contribute to harmful public health outcomes, as occurred during the COVID-19 pandemic. We challenge the premise that preprints accelerate discovery and we argue that the absence of pre-publication peer review compromises the reliability of scientific findings. We contend that scientists have a duty of care to society and that the lack of pre-publication vetting, and the minimal oversight associated with preprinting, neglects this responsibility. Because of these harms, and to safeguard both public health and trust in science, we propose that journals should not publish manuscripts posted on preprint servers.

Keywords: peer review, post-publication review, preprint servers, scientific publishing, public trust in science

The past few centuries — a period over which the greatest improvements in human health, science, and technology have occurred — correspond with the development of journals that would look more or less “modern” today. This is no coincidence. A key element in these improvements has been expert, pre-publication peer-review, which helps editors to separate valid, helpful research from work that is uninformative or unreliable.

The three of us come from different vantage points — a fisheries biologist and journal editor in Norway, the editor-in-chief of a surgical journal, and an expert in publishing with three decades in the industry—and all of us agree that peer review is imperfect. In fact, we’ve all spent our careers working to improve it (Cooke et al., 2024).

All three of us are concerned about new approaches that seek to disrupt or abandon prepublication peer review, the most damaging of which is the proliferation of preprint servers in biomedical and natural science research.

After decades of use in other fields, preprint servers—platforms that publish non-peer-reviewed versions of manuscripts—entered the biomedical and natural science publishing landscape. During the COVID-19 pandemic, some even suggested that preprint platforms helped biomedical science bring COVID-19 under control more quickly (Fraser et al., 2021).

We disagree. We contend that preprint platforms do more harm than good by undermining expert editorial selection and peer review. This was especially damaging during the COVID-19 pandemic. We also believe that the fundamental structural flaws of preprint servers undermine the values that scientists espouse (Leopold, 2021a), limit the social good that science achieves (Leopold, 2021b), and risk the public’s and the environment’s health and safety (Leopold et al., 2019). They are also particularly prone to exploitation for the dissemination of misinformation, thereby eroding the credibility of one of civil society’s most trusted sources of information (scientific journals), and the expert communities that they serve.

To illustrate our qualms, we explore two purported benefits of preprints: the suggestion that preprints accelerate scientific discoveries and the argument that the incentives of preprinting help researchers as well as society.

The notion that preprints accelerate discovery is based on the idea that draft papers circulating widely could spark or speed insights by others. By extension, the idea is that researchers and society both benefit. Within a framework that we contend is applicable — namely, the concept known as duty of care — we can scrutinize those claims.

Preprints do not accelerate discovery

A common argument made to support preprinting posits that the early release of research on preprint platforms accelerates discovery (Maggio et al., 2018) and allows other scientists to build on work more quickly (Fraser et al., 2021). These claims do not withstand scrutiny.

The ability to report scientific results quickly (and often hurriedly) in a preprint is unnecessary in biomedical or natural science, and it can be harmful. High-quality research, and its reporting, takes time, reflection, and thoughtful feedback from experts who share the fundamental scientific norm of getting it right and who see the endeavor as a truth-seeking activity. In most contexts (including global pandemics), the imperative to be right is far more important than the desire—often mainly for the benefit of the authors—to go fast. In the rare situations in which days matter, such as a genuinely time-sensitive discovery, an unanticipated fatal complication associated with a commonly used drug, or (as has been the case more recently) the emergence of a frightening new pathogen, all leading journals offer fast-track publication. They have done so for a long time. For example, on January 21^st, 2020, the Washington Post reported on the first US fatality from a “mysterious virus that broke out last month in China” (Sun & Bernstein, 2020). A fully reviewed, well-presented, informative scholarly article on COVID-19 appeared in the New England Journal of Medicine only 3 days later (Zhu et al., 2020).

For preprints to accelerate discovery, they would need to create a measurable difference in discovery trends. These trends may be difficult to measure, especially in biomedicine, where numerous hurdles exist prior to study approval. Nonetheless, the plausibility of the argument can be scrutinized. During COVID-19, 17-30% of COVID-related scientific manuscripts were preprinted (Else, 2020), with an estimated 15% of these making it through peer-review to journal publication (Anderson, 2020). That means that among published COVID-related papers, 3-5% were preprints. If the journals system handled 95-97% of COVID-related research without preprints, it is hard to make the case that preprints accelerated discovery to any large degree. Further, a large percentage of preprints are posted at the same time or after submission to a journal, often weeks or months after (Anderson, 2020). Given that, it is difficult to accept the argument that preprints accelerate discovery. At the very least, this contention should not be accepted until there is compelling evidence to support it.

For preprints to accelerate discovery, future researchers must also have confidence that the findings in those preprints are robust and trustworthy. In the current preprint ecosystem, such confidence would be misplaced. With many preprints posted on multiple preprint servers, some persisting after withdrawal from or modification in other venues, preprints create uncertainty and a serious risk of “version confusion” (Anderson, 2021b). These problems can perpetuate error and harm to patients in clinical research (Leopold et al., 2019). They also waste the time of researchers, who already are strained by other demands. Preprints add noise and create a draining intellectual burden. They slow researchers down, rather than speeding things up.

These arguments suggest to us that preprinting does not accelerate discovery. Instead, numerous high-profile failures of preprinting—including news stories about preprints contributing to poisonings (ivermectin; Anderson 2021d) and preprints feeding anti-vaccine misinformation and conspiracy theories (Anderson 2021f; Anderson, 2022)—suggest to us that they can, in fact, be bad for our health.

Post-publication review of preprints is inadequate

Those failures aren’t accidental. One might predict that widely disseminating unreviewed findings can result in harm. In fact, we did predict it (Leopold et al., 2019; Leopold, 2021a).

There is just no reason to think that post-publication peer review, which is the generic term for the approach used by preprints, will safeguard our patients and ecosystems as well as pre-publication peer review does. There are many reasons for this, not least of which is the fact that post-publication peer review simply doesn’t happen on preprint servers. Only a small minority of preprints ever get reviewed, that percentage is declining (Wakeling et al., 2020), and most comments made about preprints don’t rise to the level that a typical scientist would consider a “peer review.” An assessment of post-publication commenting on preprints reported that one in 12 got any comments at all. Among those that received comments, the comments were less than a paragraph in length (on average 43 words; Carneiro et al., 2023). This cannot be considered in-depth peer review.

As editors, reviewers, and authors ourselves, we understand the desire to get one’s work out quickly. However, we also know that, for the safety and benefit of society, we must subordinate that personal desire to the need to get it right (Anderson, 2021c; Bauchner, 2017; Kronzucker et al., 2022; Leopold et al., 2019; Sheldon, 2018). The world’s leading organizations of medical writers have echoed these concerns (American Medical Writers Association et al., 2021). Arguments in favor of post-publication peer-review can’t avoid the “horse is out the barn” problem, where preliminary or problematic studies are released and quickly amplified, limiting or eliminating the potential for meaningful subsequent correction.

The built-in incentives to preprinting are harmful

Strong journals are incentivized to get it right (both in terms of editorial selection decisions and careful review and editing) as part of good brand stewardship, and, not insignificantly, because they need to convince individuals or libraries to pay for them. By contrast, preprints—and the social-media and journalistic outlets that amplify them—score when they are splashy. Investigative journalists have found controversial preprints being used to justify popular documentaries on major streaming services, despite the underlying science being unable to pass muster and being roundly and repeatedly condemned by experts in the domain (Anderson, 2021e; Anderson, 2024).

Journalists and independent investigators have also identified examples of preprints posted to boost a company’s stock (Anderson, 2021e), to promote a politically motivated conspiracy theory (West & Bergstrom, 2021), or to undermine faith in prudent scientific and public health measures (Vogel, 2020). Journalists cover preprints as news and promote them on social media, and often fail to distinguish preprints from peer-reviewed articles. Recurrent speculative claims, identified as such only after the fact, increase public cynicism about science and undermine the public’s trust in science and medicine.

The incentives for scientists to use preprints, and for journalists to cover them as news, are so strong that they’re unlikely to be easily overcome with preprint reforms or ethical guardrails. Preprints look like journal articles, can be referenced on patient- and public-facing websites, can be cited, and have been used to promote unvetted treatments to patients and inform public policy. Many researchers now routinely promote their work to journalists as “a preprint” to increase the appearance of breaking news and gain more coverage, while also making more extreme claims. This creates a landscape of perverse incentives—the worse the concept, the greater the incentive to use preprint servers to promote it.

Scientists owe a duty of care to society, not to themselves

Duty of care can be defined as the requirement to act toward others and the public with a reasonable level of watchfulness, attention, caution, and prudence (e.g. Law.com, 2022). This is a guiding principle for physicians, and for researchers whose work is supported by and relied upon by civil society. With that in mind, the focus must be on whose lives the biomedical and scientific enterprise was built to improve and not on the desires and needs of the individuals who conduct the research.

Despite all the purported flaws attributed to peer review (Cooke et al., 2024), it seems obvious that pre-review screening, requiring conflict-of-interest disclosures, peer-reviewing, editing by a subject-matter editor, and final copyediting is more likely to bring us closer to truth than the low-bar path to publication offered by preprint servers. In this sense, posting a preprint can be viewed as shirking the duty of care that researchers have to civil society.

We contend that the duty of care that researchers owe to society also includes being mindful of the signal-to-noise ratio in science. Preprints have increased the noise level and are drowning out the signal. A large study of 2,221 of the earliest COVID-related preprints found that all had fatal data flaws and concluded that the preprints produced nothing of clinical value (Roberts et al., 2021). Despite this scathing review, these “findings” remain discoverable online. To illustrate the issue, one early-pandemic preprint suggested that COVID-19 was bioengineered (Pradhan et al., 2020). This preprint achieved one of the highest Altmetric scores ever recorded (West & Bergstrom 2021). Despite being withdrawn quickly, the message accompanying that withdrawal says that the authors “intend to revise it in response to the comments received from the research community.” Two years on, no such revision has been posted. In fact, ultimately, some 55% of preprints are never published in a peer-reviewed journal (Abdill & Blekhman, 2019; Anderson, 2020).

Another duty of care is to not cause undue harm or distress to others. During the pandemic, preprints allowed click-seeking journalists and social media platforms to trigger a seemingly endless series of false alarms (Dooney, 2020). Sociologists speak of “alarm fatigue” (Agency for Healthcare Research and Quality, 2022) and we think that the risk of this is real, diminishes the public’s trust in science, and weakens science’s ability to intervene when society needs it most.

We recognize that there is a tension between the good of individual researchers and the greater good in a civil society (Anderson, 2021g), but we assert that this tension should be resolved in favor of the latter.

False equivalencies

The traditional journals system, based on prepublication peer review, isn’t perfect. Despite best efforts, authors, editors, reviewers, statisticians, and professional staff can make mistakes or fail to detect problems. In a system with major incentives for novel findings and for productivity, fraud will occur. Untangling instances of misconduct can take months or years.

But the false equivalence offered by preprinting—the idea that since journals and preprints can both err, there’s no inherent advantage to the former over the latter (Curry, 2015), or that post-publication peer review is equivalent to pre-publication peer review (Carneiro et al., 2020)—makes no sense and is not supported by the evidence. Publication in a strong journal involves clearing numerous high bars, including surviving initial screening by leaders in the field, scrutiny by several peer-reviewers, careful editing, statistical review, copyediting, and perhaps an iterative process of manuscript revisions. It is therefore difficult to argue that this system is not better suited to the task of truth-seeking than posting a preprint when the author thinks that it is ready for public consumption.

In a fast-moving information economy with many sources of rapid amplification, it is more important than ever to have reliable, trustworthy sources of information to inform public health and general science policy (Leopold, 2022).

Possible solutions

Ultimately, in a perfect world, we argue that preprint servers should only be available to scientists. Preprints encourage the wide, public dissemination of unreviewed and often self-interested claims, which has resulted in harm, exhaustion, and confusion. Second, the preprint system neglects the duty of care that scientists owe to the public and preprints do not deliver the benefits they are claimed to hold. If preprints were only accessible to scientists, this would allow preprinting to achieve its nominal purpose — to get feedback prior to submission to a peer-review process at a journal (after all, that is what the “pre” and “print” refer to, respectively). Unfortunately, there is no practical means by which to keep journalists and the public from viewing preprint server content, nor is there a way to keep those who post their work to preprint servers from amplifying their unvetted claims on social media and elsewhere.

We believe that pressure needs to be applied in the form of a disincentive that scientists will care about: Journals that publish content pertaining to human health, or to any science policy matter, should not consider for publication a manuscript that has been posted as a preprint. They should remove from consideration anything that is posted during the course of peer review. One of the authors of this article has brought together the leading journals of his surgical specialty, which have adopted and now apply this standard (Leopold et al., 2019). All journals that publish research relevant to public policy should follow suit. Anything less risks both public trust and public health.

References

Abdill, R. J., & Blekhman, R. (2019). Tracking the popularity and outcomes of all bioRxiv preprints. eLife, 8, Article e45133. https://doi.org/10.7554/eLife.45133

Agency for Healthcare Research and Quality (AHRQ). (2022, March 10). Alert fatigue. Patient Safety Network. https://psnet.ahrq.gov/primer/alert-fatigue

American Medical Writers Association, European Medical Writers Association, & International Society for Medical Publication Professionals. (2021). AMWA-EMWA-ISMPP joint position statement on medical publications, preprints, and peer review. Current Medical Research and Opinion, 37(5), 861–866. https://doi.org/10.1080/03007995.2021.1900365

Anderson, K. (2019, October 3). The problem of preprints in the press. The Geyser. https://www.the-geyser.com/biorxiv-preprints-in-the-press/

Anderson, K. R. (2020). bioRxiv: Trends and analysis of five years of preprints. Learned Publishing, 33(2), 104–109. https://doi.org/10.1002/leap.1265

Anderson, K. (2021b, January 7). Cleaning up our own mess. The Geyser. https://www.the-geyser.com/distilling-chaos-out-of-order/

Anderson, K. (2021c, July 16). A messy withdrawal of a sloppy preprint. The Geyser. https://www.the-geyser.com/a-messy-withdrawal-of-a-sloppy-preprint/

Anderson, K. (2021d, August 21). Ivermectin preprint, poison control calls. The Geyser. https://www.the-geyser.com/ivermectin-preprint-poison-control/

Anderson, K. (2021e, August 25). medRxiv gets played — again. The Geyser. https://www.the-geyser.com/medrxiv-get-played-again/

Anderson, K. (2021f, September 27). Another preprint bomb detonates. The Geyser. https://www.the-geyser.com/another-preprint-bomb-detonates/

Anderson, K. (2021g, December 13). Is Open Science all about “me”? The Geyser. https://www.the-geyser.com/is-open-science-all-about-me/

Anderson, K. (2022, January 17). Canada drops more preprint bombs. The Geyser. https://www.the-geyser.com/canadians-throw-more-preprint-bombs/

Anderson, K. (2024, August 7). eLife’s bad model defies burial. The Geyser. https://www.the-geyser.com/elifes-bad-model-defies-burial/

Bauchner, H. (2017). The rush to publication: An editorial and scientific mistake. JAMA, 318(12), 1109–1110. https://doi.org/10.1001/jama.2017.11816

Carneiro, C. F. D., Queiroz, V. G. S., Moulin, T. C., Carvalho, C. A. M., Haas, C. B., Rayêe, D., Henshall, D. E., De-Souza, E. A., Amorim, F. E., Zacouteguy Boos, F., Guercio, G. D., Costa, I. R., Hajdu, K. L., van Egmond, L., Modrák, M., Tan, P. B., Abdill, R. J., Burgess, S. J., Guerra, S. F. S., … & Amaral, O. B. (2020). Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature. Research Integrity and Peer Review, 5, Article 16. https://doi.org/10.1186/s41073-020-00101-3

Carneiro, C. F. D., da Costa, G. G., Neves, K., Boechat de Abreu, M., Tan, P. B., Rayêe, D., Zacouteguy Boos, F., Anderjew, R., Lubiana, T., Malički, M., & Amaral, O. B. (2023). Characterization of comments about bioRxiv and medRxiv preprints. JAMA Network Open, 6(8), Article e2331410. https://doi.org/10.1001/jamanetworkopen.2023.31410

Cooke, S. J., Young, N., Peiman, K., Roche, D. G., Clements, J., Kadykalo, A., Provencher, J., Raghavan, M., DeRosa, R., Lennox, R., Fayek, A. R., Cristescu, M., Murray, S., Quinn, J., Cobey, K. D., & Browman, H. I. (2024). A harm reduction approach to improving peer review by acknowledging its imperfections. Facets, 9, 1-14. https://doi.org/10.1139/facets-2024-0102

Curry, S. (2015, September 7). Peer review, preprints, and the speed of science. The Guardian. https://www.theguardian.com/science/occams-corner/2015/sep/07/peer-review-preprints-speed-science-journals

Dooney, E. (2020, April 9). New research suggests runners should be further than 2m apart. Runners World. https://www.runnersworld.com/uk/news/a32094750/coronavirus-runner-slipstream/

Else, H. (2020). How a torrent of COVID science changed research publishing — in seven charts. Nature, 588, Article 553. https://doi.org/10.1038/d41586-020-03564-y

Fox, J. (2020, May 5). A pandemic moves peer review to Twitter. Bloomberg Quint. https://www.bloombergquint.com/gadfly/coronavirus-research-moves-faster-than-medical-journals

Fraser, N., Brierly, L., Dey, G., Polka, J. K., Pálfy, M., Nanni, F., & Coates, J. A. (2021). The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLOS Biology, 19(4), Article e3000959. https://doi.org/10.1371/journal.pbio.3000959

Kodvanj, I., Homolak, J., Virag, D., & Trkulja, V. (2022). Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues. Scientometrics, 129, 1339-1352. https://doi.org/10.1007/s11192-021-04249-7

Kronzucker, H. J., Qiu, Q. S., & Sonnewald, U. (2022). The good and the bad of preprint servers in plant physiology. Journal of Plant Physiology, 271, Article 153661. https://doi.org/10.1016/j.jplph.2022.153661

Law.com. (2022, March 9). Duty of care. https://dictionary.law.com/Default.aspx?selected=59

Leopold, S. S. (2021a). Editorial: Can journals, as trusted intermediaries, cut through the signal-to-noise problem in medical publishing? Clinical Orthopaedics and Related Research, 479(7), 1409–1412. http://doi.org/10.1097/CORR.0000000000001845

Leopold, S. S. (2021b, November 19). The dangers of undercooked science and a hungry public. Seattle Times. https://www.seattletimes.com/opinion/the-dangers-of-undercooked-science-and-a-hungry-public/

Leopold, S. S. (2022). Editorial: The process is the outcome-And it all starts with CORR’s wonderful peer reviewers. Clinical Orthopaedics and Related Research, 480(12), 2281-2283. http://doi.org/10.1097/CORR.0000000000002454

Leopold, S. S., Haddad, F. S., Sandell, L. J., & Swiontkowski, M. (2019). Editorial: Clinical Orthopaedics and Related Research, The Bone & Joint Journal, The Journal of Orthopaedic Research, and The Journal of Bone and Joint Surgery will not accept clinical research manuscripts previously posted to preprint servers. Clinical Orthopaedics and Related Research, 477(1), 1–4. http://doi.org/10.1097/CORR.0000000000000565

Maggio, L. A., Artino Jr, A. R., & Driessen, E. W. (2018). Preprints: Facilitating early discovery, access, and feedback. Perspectives on Medical Education, 7, 287–289. https://doi.org/10.1007/s40037-018-0451-8

Pradhan, P., Pandey, A. K., Mishra, A., Gupta, P., Tripathi, P. K., Menon, M. B., Gomes, J., Vivekanandan, P., & Kundu, B. (2020). Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag. bioRxiv. https://doi.org/10.1101/2020.01.30.927871

Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., Weir-McCall, J. R., Teng, Z., Gkrania-Klotsas, E., AIX-COVNET, Rudd, J. H. F., Sala, E., & Schönlieb, C. B. (2021). Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence, 3, 199–217. https://doi.org/10.1038/s42256-021-00307-0

Sheldon, T. (2018, July 24). Preprints could promote confusion and distortion. Nature, 559, Article 445. https://doi.org/10.1038/d41586-018-05789-4

Sun, L. H., & Bernstein, L. (2020, January 21). First U.S. case of potentially deadly Chinese coronavirus confirmed in Washington state. The Washington Post. https://www.washingtonpost.com/health/2020/01/21/coronavirus-us-case/

Vogel, G. (2020, April 21). Antibody surveys suggesting vast undercount of coronavirus infections may be unreliable—Critics question accuracy of tests and media promotion before full results are published. Science. https://www.science.org/content/article/antibody-surveys-suggesting-vast-undercount-coronavirus-infections-may-be-unreliable

Wakeling, S., Willett, P., Creaser, C., Fry, J., Pinfield, S., Spezi, V., Bonne, M., Founti, C., & Medina Perea, I. (2020). ‘No comment’? A study of commenting on PLOS articles. Journal of Information Science, 46(1), 82–100. https://doi.org/10.1177/0165551518819965

West, J. D., & Bergstrom, C. T. (2021). Misinformation in and about science. Proceedings of the National Academy of Sciences, 118(15), Article e1912444117. https://doi.org/10.1073/pnas.1912444117

Zhu, N., Zhang, D., Wang, W., Li, X., Yang, B., Song, J., Zhao, X., Huang, B., Shi, W., Lu, R., Niu, P., Zhan, F., Ma, X., Wang, D., Xu, W., Wu, G., Gao, G. F., & Tan, W. (2020). A novel coronavirus from patients with pneumonia in China, 2019. New England Journal of Medicine, 382(8), 727–733. http://doi.org/10.1056/NEJMoa2001017