Publication bias

Sciences humaines.svg Educational level: this is a tertiary (university) resource.
Progress-0250.svg Completion status: this resource is ~25% complete.

Publication bias arises from the tendency for researchers, editors, and companies to handle the reporting of experimental results that are positive (i.e., they show a significant finding) differently from results that are negative (i.e. supported the null hypothesis) or inconclusive.

This page provides an undergraduate-level introduction to the problem of publication bias.

Two counter-acting biasesEdit

  • Low Power: under-estimation of real effects
  • Publication Bias or File-drawer effect: over-estimation of real effects

What is publication bias?Edit

  • Publication of results depends on their nature and direction.
  • Studies that show significant effects are more likely to be published.
  • Type I publication errors are underestimated to the extent that they are: “frightening, even calling into question the scientific basis for much published literature.” (Greenwald, 1975, p. 15)

Funnel plotsEdit

  • A scatterplot of treatment effect against study size.
  • Precision in estimating the true treatment effect increases as N increases.
  • Small studies scatter more widely at the bottom of the graph.
  • As studies become less precise, results should be more variable, scattered to both sides of the more precise larger studies … unless there is publication bias.
  • In the absence of bias the plot should resemble a symmetrical inverted funnel.
  • If there is publication bias this will cause meta-analysis to overestimate effects.
  • The more pronounced the funnel plot asymmetry, the more likely it is that the amount of bias will be substantial.

File drawer effectsEdit

  1. Tendency for non-sig. results to be ‘filed away’ (hidden) and not published.
  2. of null studies which would have to ‘filed away’ in order for a body of significant published effects to be considered doubtful.

Countering the biasEdit

  1. Journal Articles in Support of the Null Hypothesis

See alsoEdit

External linksEdit