PerspectiveSCIENTIFIC INTEGRITY

What does research reproducibility mean?

+ See all authors and affiliations

Science Translational Medicine  01 Jun 2016:
Vol. 8, Issue 341, pp. 341ps12
DOI: 10.1126/scitranslmed.aaf5027

Figures

  • Fig. 1. Reports rising.

    Number of publications recorded in Scopus that have, in the title or abstract, at least one of the following expressions: research reproducibility, reproducibility of research, reproducibility of results, results reproducibility, reproducibility of study, study reproducibility, reproducible research, reproducible finding, or reproducible result. Papers are classified by discipline on the basis of the journal, following an adaptation and expansion of Thomson Reuters’ Essential Science Indicators classification system. Journals not included in the latter database were hand-classified on the basis of their name. The subplot reports the percentage over the total number of records for each discipline, in the last 2 years of the series. Disciplines legend: MA, mathematics; CS, computer sciences; EN, engineering; SP, space science; PH, physics; CH, chemistry; BB, biology and biochemistry; MB, molecular biology; MI, microbiology; PT, pharmacology and toxicology; CM, clinical medicine; NB, neurobiology and behavior; PA, plant and animal sciences; EE, environment and ecology; AG, agricultural sciences; EB, economics and business; PP, psychology and psychiatry; SO, social sciences, general; AH, arts and humanities; MU, multidisciplinary. The time series was truncated at 2014.

Tables

  • Table 1. Examples of differences that affect the approach to reproducibility in distinct scientific domains.
    Degree of determinism
    Signal to measurement-error ratio
    Complexity of designs and measurement tools
    Closeness of fit between hypothesis and exper
    imental design or data
    Statistical or analytic methods to test hypotheses
    Typical heterogeneity of experimental results
    Culture of replication, transparency, and cumu
    lating knowledge
    Statistical criteria for truth claims
    Purposes to which findings will be put and
    consequences of false conclusions
  • Table 2. Terminology to describe practices that introduce or hide multiplicity.
    Multiple comparisons (many statisticians)
    File-drawer problem (29)
    Pseudoreplication (32)
    Significance questing (33)
    Data mining, dredging, torturing (34)
    Hypothesizing after the results are known
    (HARKing) (30)
    Data snooping (35)
    Selective outcome reporting (36)
    Silent multiplicity (37)
    Specification searching (38)
    P-hacking (31)