Skip to content
Center of
Trial & Error

20 articles

All articles
Meta-Research

Persuasion in Scientific Journal Articles: Exploring Persuasiveness and Informativeness among Authors and Readers

Rrita Bajraktari & Marcus Munafò

Scientific communication is often assumed to prioritize objectivity, yet persuasive strategies may subtly distort research interpretation. While the spread of errors in research is often attributed to methodological or statistical biases (Świątkowski & Dompnier, 2017), rhetorical strategies are less…

doi:10.36850/b364-4157Read
Meta-Research

A Conceptual Framework for Computational Reproductions: Formal Definitions and Epistemic Functions

Florian Kohrt, Filip Melinscak, Richard McElreath & Felix Schönbrodt

Reproductions, which we define as “redoing activities” that work with the same collected data (or simulation source code) and aim to keep the computation as similar as possible, enable the verification of a scientific work. We seek to advance the understanding and conduct of reproductions by…

doi:10.36850/9ec0-4fabRead
Meta-Research

How to Get There From Here? Barriers and Enablers on the Road Towards Reproducibility in Research

Serge P. J. M. Horbach, Nicki Lisa Cole, Simone Kopeinik, Barbara Leitner, Tony Ross-Hellauer & Joeri Tijdink

Reproducibility of research is a hotly debated topic, including aspects like causes and consequences of low levels of reproducibility. While some research fields have led the way and introduced various reproducibility practices and procedures, the call for efforts to ‘improve’ reproducibility in…

doi:10.36850/f4c4-4a38Read
Meta-Research

The Open Science Platforms Fighting Clandestine Abuses of Piracy and Phishing: Case of Open Science Framework

Ayumi Ikeda, Fumiya Yonemitsu, Naoto Yoshimura, Kyoshiro Sasaki & Yuki Yamada

The Open Science Framework (OSF) is a free and open-source platform for managing research projects (Elliott et al., 2021). The OSF is a flagship product of the Center for Open Science, Inc. (COS), whose mission is to increase the openness, integrity, and reproducibility of scientific research. It is…

doi:10.36850/57fe-4c10Read
Meta-Research

Why Not to Use Difference Scores in Research on Affective Polarization: A Critique and a Tutorial for Using Polyvariate Regression to Study Congruence as an Outcome

Lukas K. Sotola

Affective polarization, people’s emotional attachment to their own party and dislike of rival parties, has been a growing area of interest in recent decades. However, studies on affective polarization have tended to use difference score measures to capture affective polarization. Specifically, it is…

doi:10.36850/d6d3-4f14Read
Meta-Research

Type I Error Rates are Not Usually Inflated

Mark Rubin

The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p -hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and,…

doi:10.36850/4d35-44bdRead
Meta-Research

Three Persistent Myths about Open Science

Moin Syed

Knowledge and implementation of open science principles and behaviors remains uneven across the sciences, despite over 10 years of intensive education and advocacy. One reason for the slow and uneven progress of the open science movement is a set of closely held myths about the implications of open…

doi:10.36850/mr11Read
Meta-Research

Issues in Clinical Studies Leading to Medical Research Ethics Committee (MREC) Negative Decisions

Sigrid E. M. Heinsbroek, Vincent Bontrop, Rutger P. Chorus & C. Michel Zwaan

The rationale behind a Medical Research Ethics Committee (MREC) negative decision is always shared directly with the applicants. However, insight into the review process and common reasons for a negative decision may also be valuable for other researchers, clinical research organizations and people…

doi:10.36850/21gx-cy60Read
Meta-Research

Reputation Without Practice? A Dynamic Computational Model of the Unintended Consequences of Open Scientist Reputations

Maximilian Linde, Merle-Marie Pittelkow, Nina R. Schwarzbach & Don van Ravenzwaaij

Practicing open science can have benefits for the career prospects of individual researchers or labs through higher quality work and increased chances of publication. However, being an outspoken advocate of open science might also indirectly benefit individual scientific careers, in the form of…

doi:10.36850/mr10Read
Meta-Research

Tension Between Theory and Practice of Replication

Erkan O. Buzbas & Berna Devezer

A core problem that has been addressed in the scientific reform movement so far is the low rates of reproducibility of research results. Mainstream reform literature has aimed at increasing reproducibility rates by implementing procedural changes in research practice and scientific policy. At the…

doi:10.36850/mr9Read
Meta-Research

A Manifesto for Rewarding and Recognizing Team Infrastructure Roles

Arielle Bennett, Daniel Garside, Cassandra Gould van Praag, Thomas J. Hostler, Ismael Kherroubi Garcia, Esther Plomp, Antonio Schettino, Samantha Teplitzky & Hao Ye

The Scientific Reform Movement has highlighted the need for large research teams with diverse skills. This has necessitated the growth of professional team infrastructure roles (TIRs) who support research through specialised skills, but do not have primary responsibility for conceiving or leading…

doi:10.36850/mr8Read
Meta-Research

Rethinking Transparency and Rigor from a Qualitative Open Science Perspective

Crystal N. Steltenpohl, Hilary Lustick, Melanie S. Meyer, Lindsay Ellis Lee, Sondra M. Stegenga, Laurel Standiford Reyes & Rachel L. Renbarger

Discussions around transparency in open science focus primarily on sharing data, materials, and coding schemes, especially as these practices relate to reproducibility. This fairly quantitative perspective of transparency does not align with all scientific methodologies. Indeed, qualitative…

doi:10.36850/mr7Read
Meta-Research

Reflections on Preregistration: Core Criteria, Badges, Complementary Workflows

Robert T. Thibault, Charlotte R. Pennington & Marcus R. Munafò

Clinical trials are routinely preregistered. In psychology and the social sciences, however, only a small percentage of studies are preregistered, and those preregistrations often contain ambiguities. As advocates strive for broader uptake and effective use of preregistration, they can benefit from…

doi:10.36850/mr6Read
Meta-Research

The Invisible Workload of Open Research

Thomas J. Hostler

It is acknowledged that conducting open research requires additional time and effort compared to conducting ‘closed’ research. However, this additional work is often discussed only in abstract terms, a discourse which ignores the practicalities of how researchers are expected to find the time to…

doi:10.36850/mr5Read
Meta-Research

Questionable Metascience Practices

Mark Rubin

Metascientists have studied questionable research practices in science. The present article considers the parallel concept of questionable metascience practices (QMPs). A QMP is a research practice, assumption, or perspective that has been questioned by several commentators as being potentially…

doi:10.36850/mr4Read
Meta-Research

An Introduction to Complementary Explanation

Joeri van Hugten

This paper introduces the practice of complementary explanation; the practice of taking a published result and writing a focused paper that rigorously and systematically describes the implications for a theory that would be rejected by those results. Such spotlighting of a rejected theory…

doi:10.36850/mr3Read
Meta-Research

Towards an Expanded Conception of Publication Bias

Nicole Nelson

This short essay argues for an expanded conception of publication bias. In addition to considering the selective publication of results, I argue that we need to also consider the selective publication of epistemic by-products—observations and knowledge that scientists accumulate incidentally in the…

doi:10.36850/mr2Read