Table 1

Assessing the quality of the evidence

Quality elementDescription
Risk of biasLimitations in the study design and implementation might bias the estimates of the treatment effect. Major limitations in studies decrease the confidence in the estimate of the effect. Examples of such limitations are selection bias (often due to poor allocation concealment), performance and detection bias (often due to a lack of blinding of the patient, healthcare professional or assessor) and attrition bias (due to missing data causing systematic bias in the analysis).
IndirectnessIndirectness refers to differences in study population, intervention, comparator and outcomes between the available evidence and the review question.
InconsistencyInconsistency refers to an unexplained heterogeneity of effect estimates between studies in the same meta-analysis.
ImprecisionResults are imprecise when studies include relatively few patients and few events (or highly variable measures) and thus have wide confidence intervals around the estimate of the effect relative to clinically important thresholds. 95% confidence intervals denote the possible range of locations of the true population effect at a 95% probability, and so wide confidence intervals may denote a result that is consistent with conflicting interpretations (for example, a result may be consistent with both clinical benefit AND clinical harm) and thus be imprecise.
Publication biasPublication bias is a systematic underestimate or overestimate of the underlying beneficial or harmful effect due to the selective publication of studies. A closely related phenomenon is where some papers fail to report an outcome that is inconclusive, thus leading to an overestimate of the effectiveness of that outcome.
Other issuesSometimes randomisation might not adequately lead to group equivalence of confounders, and if so, this might lead to bias, which should be taken into account. Potential conflicts of interest, often caused by excessive pharmaceutical company involvement in the publication of a study, should also be noted.