Figures
Fig. 1. Reports rising. Number of publications recorded in Scopus that have, in the title or abstract, at least one of the following expressions: research reproducibility, reproducibility of research, reproducibility of results, results reproducibility, reproducibility of study, study reproducibility, reproducible research, reproducible finding, or reproducible result. Papers are classified by discipline on the basis of the journal, following an adaptation and expansion of Thomson Reuters’ Essential Science Indicators classification system. Journals not included in the latter database were hand-classified on the basis of their name. The subplot reports the percentage over the total number of records for each discipline, in the last 2 years of the series. Disciplines legend: MA, mathematics; CS, computer sciences; EN, engineering; SP, space science; PH, physics; CH, chemistry; BB, biology and biochemistry; MB, molecular biology; MI, microbiology; PT, pharmacology and toxicology; CM, clinical medicine; NB, neurobiology and behavior; PA, plant and animal sciences; EE, environment and ecology; AG, agricultural sciences; EB, economics and business; PP, psychology and psychiatry; SO, social sciences, general; AH, arts and humanities; MU, multidisciplinary. The time series was truncated at 2014.
Tables
- Table 1. Examples of differences that affect the approach to reproducibility in distinct scientific domains.
Degree of determinism Signal to measurement-error ratio Complexity of designs and measurement tools Closeness of fit between hypothesis and exper
imental design or dataStatistical or analytic methods to test hypotheses Typical heterogeneity of experimental results Culture of replication, transparency, and cumu
lating knowledgeStatistical criteria for truth claims Purposes to which findings will be put and
consequences of false conclusions - Table 2. Terminology to describe practices that introduce or hide multiplicity.
Multiple comparisons (many statisticians) File-drawer problem (29) Pseudoreplication (32) Significance questing (33) Data mining, dredging, torturing (34) Hypothesizing after the results are known
(HARKing) (30)Data snooping (35) Selective outcome reporting (36) Silent multiplicity (37) Specification searching (38) P-hacking (31)
