Statistics from Altmetric.com
Just as the weakest link in a chain determines how much weight the chain will hold, the weakest link in the data used by Fiorica et al will determine how much weight we as readers should give to their findings and conclusions regarding neoadjuvant chemoradiotherapy for oesophageal adenocarcinoma (Gut 2004;53:925–30). Clearly, the weakest link in their data is the material by Walsh et al, and prior to placing any confidence in the conclusions by Fiorica et al, a careful assessment of the reliability of the Walsh data is imperative.1 Well known criticisms of the Walsh trial include the lack of routine staging with computed tomography scanning that led to five patients undergoing surgery alone for stage 4 disease, the exclusion of a number of patients in the neoadjuvant arm for “protocol violations” when in fact several had evidence of progressive disease and should have been considered treatment failures, and the lack of a uniform surgical technique that led to five different types of operations being performed and what are arguably the worst surgical results for oesophageal adenocarcinoma reported in the literature. However, these concerns are overshadowed by an even greater problem in the Walsh trial related to internal inconsistencies in the survival data. Careful review of the Walsh manuscript reveals that the survival data in the text of the report does not match the data in the Kaplan-Meier survival curves, but interestingly the discrepancy is only for the neoadjuvant arm.1 In all cases the survival data for the surgery alone arm matches up precisely. For example, in the text of the manuscript, survival by intention to treat at three years in the neoadjuvant arm is reported as 32%, yet on the Kaplan-Meier graph survival by intention to treat in the neoadjuvant arm is approximately 48%.1 Similar discrepancies occur at essentially every data point for both the intention to treat and the treatment actually received graphs, but only for the neoadjuvant arm, with survival on the Kaplan-Meier graphs much improved over the data in the text. Importantly, the statistics for survival are calculated from the Kaplan-Meier curves, raising concern that the difference in survival between groups is in fact not significant. This alarming discrepancy has never been adequately addressed despite a letter to the New England Journal of Medicine and a subsequent reply by Dr Walsh.2 The response by Walsh was that the graphs were mislabelled, but even with a different label the data points continue to be incongruent.
In light of this, I would like to know how Fiorica et al handled the data from the Walsh trial. Did they use data from the Kaplan-Meier survival curves or from the text and tables in the manuscript? Were they aware of the discrepancy and if so why did they not comment on it in their manuscript and specify how they dealt with it in their meta-analysis? In light of these concerns, as well as other issues regarding this trial, is it appropriate to even include it in a meta-analysis unless the raw data are independently reviewed and the statistics validated? This is an especially important issue as the Walsh study is the only trial that included just patients with adenocarcinoma, and as stated in the manuscript by Fiorica et al, robust analysis showed that exclusion of the Walsh trial would lead to loss of statistical significance for overall mortality (Gut 2004;53:925–30). This would leave us where we started, lacking any significant evidence that neoadjuvant therapy improves survival for patients with oesophageal adenocarcinoma.
Conflict of interest: None declared.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.