Research Challenges to Consider

Comparison group

  • Both comparison population (external control) and same population before and after implementation of intervention are used

  • Use of external control population depends on availability of comparable population

Biologic plausibility

  • Effect being tested is biologically plausible

  • Identifying research designs that can address biologic plausibility

  • In hypothesis-generating studies, biologic plausibility is not always known before study is designed

Experimental design

  • Experimental designs are typically best able to demonstrate cause–effect relationship

  • It would not be possible ethically to test effect of secondhand smoke on acute MIs experimentally

Hypothesis clarification

  • Hypothesis being tested is clearly stated

  • Tested hypothesis matches question being asked in interpreting results

  • Studies are designed to test specific hypothesis; users of study results should consider study hypothesis when determining what questions study can answer

Statistical Design

  • Appropriate statistical analysis, determined a priori, controls for appropriate confounders

  • Statistical models can be used to control for potential confounders and trends

  • Statistical modeling includes description of modeling assumptions and sensitivity analysis of impact of model choice and assumptions on modeling results

  • Statistical analysis is generally under control of researchers designing study, but options could be limited by characteristics of available data

  • If appropriate data are available, choice of model and assumptions are under control of researchers

Publication bias

  • Negative results are less apt to be published than positive findings

  • Both researchers and journal editors should overcome their preference for publishing positive findings

a A study typically cannot attain the ideal for all characteristics, so researchers must weigh the importance of each characteristic and the availability of data when determining study design.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement