Statistics from Altmetric.com
We read with interest that an abnormal platelet count/spleen diameter ratio predicts the presence of oesophageal varices (Gut 2003;52:1200–5). This otherwise excellent article contained a statistical error that we would like to bring to your attention.
The authors reported 100% sensitivity for the diagnostic test (platelet count/spleen diameter ratio at a cut off value of 909) in ruling out the diagnosis of oesophageal varices: all patients with varices had an abnormal ratio. The reported 95% confidence interval (CI) for the sensitivity was 100–100%. This is simply incorrect, and is was probably calculated using a statistical formula that can be only used when:
the sample size is large and
the proportion in question is not equal to 0% or 100%.
SE(p) = sqrt[p(1−p)/n]
where SE = standard error, p = proportion, and n = sample size, gives a value for the standard error of a proportion. However, when the proportion is 0% or 100%, the standard error becomes 0, and this formula is void.
Imagine a diagnostic test that is performed on 10 patients, two of whom have the disease in question. Let us say that the test is positive in both patients with the disease and in two of the eight patients without the disease. The calculated sensitivity is 100% and specificity is 75%. Would any respectable journal editor be convinced that the true sensitivity of this diagnostic test lies somewhere between 100% and 100%—that if a validation cohort underwent this diagnostic test we could be confident that all patients with the disease should test positive? Of course not. But the above formula would also give a standard error of 0 in this case.
The bottom line is that it is never acceptable to give a confidence interval in medicine that is a point estimate. If a computer program or statistical textbook formula gives a point estimate instead of an interval, then the wrong formula was used. This particular statistical error (expressing a confidence interval as a point estimate) is easily identified without any statistical training or calculations.
In the case of the platelet count/spleen diameter ratio, an appropriate statistical test is the score confidence interval (Agresti-Coull) method. Using this method (in JMP 5.1, SAS Institute, Cary, North Carolina, USA), the sensitivity of the platelet count/spleen diameter ratio in the derivation cohort should have been:
89 of 89 patients with varices correctly identified = 100% (95% confidence interval 95.9–100%)
and for the validation cohort:
71 of 71 patients = 100% (95% confidence interval 94.9–100%).
Combining the validation and derivation cohorts:
160 of 160 patients = 100% (95% confidence interval 97.7–100%).
This is not nit-picky as many clinicians will read this article and believe that a normal platelet count/spleen size ratio rules out oesophageal varices with 100% certainty. But surely someone, somewhere, will find an exception to this useful (and remarkably accurate) diagnostic rule.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.