Epidemiological characteristics and prevalence rates of research reproducibility across disciplines: A scoping review

Background: Reproducibility is a central tenant of research. Explicit reproducibility checks are made across different disciplines trying to assess the replication of previously published studies. We aimed to synthesize the literature on reproducibility and describe its epidemiological characteristics, including how reproducibility is defined and assessed. We also aimed to determine and compare estimates for reproducibility across different fields.Methods and Findings: We conducted a scoping review to identify English language replication studies published between 2018-2019 in economics, education, psychology, health sciences and biomedicine. We searched Medline, Embase, PsycINFO, Cumulative Index of Nursing and Allied Health Literature – CINAHL, Education Source via EBSCOHost, ERIC, EconPapers, International Bibliography of the Social Sciences (IBSS), and EconLit. Documents retrieved were screened in duplicate against our inclusion criteria. We extracted year of publication, number of authors, country of affiliation of the corresponding author, and whether the study was funded. For the individual replication studies, we recorded whether a registered protocol was used, whether there was contact between the replicating team and the original authors, what study design was used, and what the primary outcome was. Finally, we recorded how replication was defined by the authors, and whether the assessed study(ies) successfully replicated based on this definition. Extraction was done by a single reviewer and quality controlled by a second reviewer. Our search identified 11,224 unique documents, of which 47 were included in this review. Most studies were related to either psychology (48.6%) or health sciences (23.7%). Among these 47 documents, 36 described a single replication study while the remaining 11 reported at least two replications in the same paper. Less than the half of the studies referred to a registered protocol. There was variability in the definitions of replication success. In total, across the 47 documents 177 studies were reported. Based on the definition used by the author of each study, 95 of 177 (53.7%) studies replicated. Conclusion: This study gives an overview of research across five disciplines that explicitly set out to replicate previous research. Such replication studies are extremely scarce, the definition of a success in replication is ambiguous, and the replication rate is overall modest.