Article Text
PDF
Abstract
Objective Endoscopic mucosal resection (EMR) is established for the management of benign and early malignant upper GI disease. The aim of this observational study was to establish the effect of endoscopist procedural volume on mortality.
Design Patients undergoing upper GI EMR between 1997 and 2012 were identified from the Hospital Episode Statistics database. The primary outcome was 30-day mortality and secondary outcomes were 90-day mortality, requirement for emergency intervention and elective cancer re-intervention. Risk-adjusted cumulative sum (RA-CUSUM) analysis was used to assess patient mortality risk during initial stage of endoscopist proficiency gain and the effect of endoscopist and hospital volume. Mortality was compared before and after the change point or threshold in the RA-CUSUM curve.
Results 11 051 patients underwent upper GI EMR. Endoscopist procedure volume was an independent predictor of 30-day mortality. Fifty-eight per cent of EMR procedures were performed by endoscopists with annual volume of 2 cases or less, and had a higher 30-day and 90-day mortality rate for patients with cancer, 6.1% vs 0.4% (p<0.001) and 12% vs 2.1% (p<0.001), respectively. The requirement for emergency intervention after EMR for cancer was also greater with low volume endoscopists (1.8% vs 0.1%, p=0.002). In patients with cancer, the RA-CUSUM curve change points for 30-day mortality and elective re-intervention were 4 cases and 43 cases, respectively.
Conclusions EMR performed by high volume endoscopists is associated with reduced adverse outcomes. In order to reach proficiency, appropriate training and procedural volume accreditation training programmes are needed nationally.
- OESOPHAGEAL CANCER
- GASTRIC CANCER
- THERAPEUTIC ENDOSCOPY
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Significance of this study
What is already known on this subject?
Advances in therapeutic endoscopy has led to the endoscopic management of benign and malignant upper GI disease.
Increasing hospital and surgeon volume is associated with a reduction in postoperative mortality from oesophagectomy and gastrectomy and has led to the centralisation of surgical services.
What are the new findings?
High volume endoscopists significantly reduce 30-day mortality of endoscopic mucosal resection, with endoscopist volume of greater importance than hospital volume.
The initial stage of endoscopist competency gain is the most critical and is associated with significant mortality in the first five cases.
A longer endoscopist proficiency gain curve is seen for analysis of oncological outcomes such as the requirement for re-intervention in cancer cases.
How might it impact on clinical practice in the foreseeable future?
Appropriate training and volume accreditation are needed to minimise mortality and requirement for re-intervention during the initial stage of proficiency gain.
Introduction
The UK National audit showed that 25% of early oesophagogastric cancers and 30% of early colorectal cancers were managed by endoscopic resection in 2013.1 ,2 This increase in the uptake of endoscopic therapy for cancer is coupled with a learning effect. The proficiency gain curve for endoscopic submucosal dissection in colorectal and gastric cancers ranges from 25 to 40 cases, when considering complications, operative time and R0 resection margin.3–5 Nevertheless, the vast majority of studies on the outcomes of endoscopic mucosal resection (EMR) originates from single institutions, which does not reflect the uptake of new techniques at a national level.
In recent years, the centralisation of surgical services in the UK led to significant improvement in short-term outcomes of oesophageal and gastric cancer surgical resection.6 The increase in hospital and surgeon volume has been shown to be associated with a reduction in postoperative mortality from oesophagectomy and gastrectomy.7–9 The reasons for this improvement in outcomes is that high volume centres have the appropriate infrastructure and surgical expertise to deliver a consistent level of clinical outcomes from high-risk cancer surgery.10 The effect of centre and clinician's procedural volumes has not been similarly examined for EMR previously.
The aim of this study was to establish the effect of the endoscopist and hospital procedural volume and proficiency gain on mortality and requirement for re-intervention from upper GI EMR.
Methods
Data were derived from the Hospital Episode Statistics (HES) database.11 This is an administrative record-based system that collects patient-level data from all National Health Service hospitals in England. It captures all patients treated in public sector hospitals and a minority of patients treated in privately funded institutions. Patients are given a unique HES identifier that allows all of their hospital admissions to be tracked throughout the data set.
All patients over the age of 17 years who underwent elective endoscopic upper GI EMR between the 1st of January 1997 and the 31st of March 2012 were included in the study. EMRs were identified using the Office of Population Censuses and Surveys Classification of Surgical Operations and Procedures 4th revision codes (G43.1, G43.8 and G43.9). Coding for EMR was independently verified at a local institutional level. Patients sent for emergency intervention within 14 days of the initial EMR episode were identified using codes for repair or drainage of oesophagus G07–09, repair of gastric perforation/injury G34–36 and for endoscopic stent insertion G15.4, G15.6, G15.7, G21.5 and G44.1. No patient within the data set received emergency gastrectomy or oesophagectomy within 14 days of the initial EMR. Cancer diagnoses were identified using the relevant International Classification of Disease 10th revision codes. The codes for oesophagogastric malignancy included C15, C16, D00·1 and D00·2. Patients with oesophagogastric cancer with an inadequate initial EMR and likely positive resection margin were identified through re-treatment after 30 days of the initial EMR, with oesophagectomy (G01, G02), gastrectomy (G27, G28) or repeat EMR (G43.1, G43.8 and G43.9).
The primary outcome was 30-day mortality; secondary outcomes included 90-day mortality, requirement for emergency intervention within 14 days, and in patients with oesophagogastric cancer the requirement for elective re-intervention (repeat EMR, oesophagectomy and gastrectomy) after 30 days. Thirty-day and 90-day mortality were identified by linking HES data with data from the Office for National Statistics with 100% population coverage. Outcomes were evaluated for EMR for the treatment of patients with oesophagogastric cancer and for non-cancer patients in separate analyses.
Risk-adjusted cumulative sum curve analysis
Endoscopist proficiency gain curve
To identify the existence and length of a proficiency gain curve for EMR a combination of risk-adjusted cumulative sum (RA-CUSUM) and change point analysis was performed. RA-CUSUM curves were plotted for the cumulative difference between the observed and the expected outcomes (Y axis) against endoscopist case number (X axis); using the CUSUM equation Si=Si−1+(Σi−ΣR); S0=0: Si is the cumulative sum, Σi the sum of events at procedure number i and ΣR the sum of expected events at procedure number i. Therefore at each case number the curve goes upwards if the outcome is worse than expected and downwards if the outcome is better than expected (see online supplementary material 1). According to the unique anonymised endoscopist codes within HES, the first case in each endoscopist case series was assigned case 1 and subsequent case numbers assigned according to ascending date order. The expected outcomes were derived from logistic regression models for each binary outcome; these provided the predicted probability of each outcome in each case. Potential confounding factors included in the models were age, gender, Social Deprivation Index and Charlson Comorbidity Index, all well recognised predictors of mortality provided in HES.
Supplementary material
An inverse relationship was expected between experience and adverse outcomes and the length of the proficiency gain curve was defined as the number of cases for a sustained improvement in outcome. This was represented graphically on the CUSUM curve as the maximal positive deflection, the point at which outcomes changed from worse than expected to better than expected. The clinical significance of this change point was determined by comparing outcomes before and after. These binary outcomes were compared using the χ2 test and a threshold of significance was set at a p value of <0.05.
To identify the impact of the proficiency gain curve on the volume-outcome relationship cumulative moving average (CMA) 30-day and 90-day mortality curves were compared for high volume and low volume endoscopists (as defined by the CUSUM change point analysis above) for the first eight cases. The CMA curves were created using the equation CMAi=(x1+……..+xi)/i, where xi is the mean mortality at procedure number i.
Endoscopist and hospital volume
RA-CUSUM analysis was also used to determine both the existence and the value of any endoscopist or hospital volume threshold. The volume threshold was defined as the minimum annual case load for an alteration in volume-outcome relationships. Similar to the proficiency gain curve analysis above the curve plots the cumulative difference between the observed and the expected mortality, against the endoscopist or hospital volume. On the basis of this definition, the curve goes upwards every time the observed mortality exceeds the expected mortality and vice versa. As an inverse relationship between volume and mortality, emergency intervention and elective re-intervention was expected, the hypothesis was that the volume threshold would coincide where the curve peaked.
Hierarchical multivariate regression analysis with Bonferroni correction was performed to identify the most important patient factors and endoscopist or hospital volume (as defined by CUSUM change point analysis above) associated with 30-day mortality. Patient variables included in this model were age, gender and medical comorbidities (as reflected by the Charlson Comorbidity Index). To evaluate the influence of endoscopist subspecialty, data from a previous oesophagectomy gastrectomy data set were linked to identify cancer surgeons performing endoscopy, and included in the regression models for the cancer cohort only. Hierarchical logistic regression included three models; (1) patient comorbidities, cancer surgeon specialty and only endoscopist volume, (2) patient comorbidities, cancer surgeon specialty and only hospital volume, and (3) patient comorbidities, cancer surgeon specialty hospital and endoscopist volume.
CUSUM curves were computed using Excel (Excel for Mac 2011, V.14.1.4, Microsoft Corporation). For the remaining statistical analysis SPSS V.22.0 was used (Statistical Package for the Social Sciences software, Chicago, Illinois, USA). Change point modelling was performed in R (V.3.2.3).
Results
The demographics of the patients undergoing EMR are described in table 1. The effects of endoscopist proficiency gain and endoscopist/hospital volume upon clinical outcomes are described in tables 2 and 3, respectively.
Preoperative demographics for patients undergoing endoscopic mucosal resection
Changes in clinical outcomes from EMR before and after change point in RA-CUSUM curve analysis for endoscopist proficiency gain (A) non-cancer patients and (B) patients with oesophagogastric cancer
Changes in clinical outcomes from EMR before and after threshold in RA-CUSUM curve analysis for patients with cancer by (A) annual endoscopist volume and (B) annual hospital volume
Within the study period 11 051 patients underwent upper GI EMR, with 1078 EMR procedures performed for oesophageal or gastric cancer and 9973 for non-cancer cases. Other common non-cancer diagnostic indications for EMR included gastric polyps (2942 patients, 29.5%), benign gastric neoplasms (2465 patients, 24.7%), oesophageal or gastric ulcers (1702 patients, 17.1%) and Barrett's oesophagus (914 patients, 9.2%). There was a steady increase in EMR performed over the study period from 1354 (12.3%) in early years (1997–2000) to 4566 (41.3%) in later years (2009–2012). In particular there was an increase in the number of EMRs performed for oesophagogastric cancer over time, from 85 (7.9%) in 1997–2000 to 538 (49.9%) in 2009–2012. The percentage 30-day and 90-day mortalities over the study period were 1.2% and 2.7%, respectively, for all patients. There was a reduction in 30-day mortality (1.8% to 0.8%, p<0.001) and 90-day mortality (3.6% to 1.8%, p<0.001) over the study period.
Endoscopist RA-CUSUM curve analysis
Analysis of the RA-CUSUM curve for the non-cancer cohort of 9973 patients undergoing EMR showed a significant change point at two cases where a significant reduction in 30-day mortality was observed from 1.8% to 0.5% (p<0.001) (figure 1, table 2). There was a similar change point for 90-day mortality at two cases where the mortality rate decreased from 3.6% to 1.3% (p<0.001). There was no identifiable change point in the RA-CUSUM curve for the need for emergency intervention in the non-cancer cohort.
National risk-adjusted cumulative sum (RA-CUSUM) curves of non-cancer endoscopic mucosal resection (EMR) cases and of EMR for oesophagogastric cancer showing significant change points in clinical outcomes for EMR during endoscopist proficiency gain.
Analysis of patients receiving EMR for oesophagogastric cancer showed a change point at four cases, where a significant reduction in 30-day mortality was observed from 4.5% to 0.3% (p<0.001). The change point for 90-day mortality was two cases with a reduction in 90-day mortality from 10.8% to 2.6% (p<0.001). The requirement for emergency intervention within 14 days of initial EMR had a change point at one case with significant reductions of 1.5% to 0.1% (p=0.009). The requirement for elective re-intervention at least 30 days after initial EMR showed a longer proficiency gain with a change point at 43 cases, and a significant reduction from 24.6% to 10.8% (p=0.005); surgical resection fell from 15.3% to 5.3% (p=0.017) and repeat EMR fell from 9.3% to 5.3% (p=0.238).
The CMA curves for the low volume and high volume endoscopists reveal a gradual decrease in both 30-day mortality and 90-day mortality consistent with the CUSUM curves. The curves for low volume endoscopists over the initial eight cases are equivalent to high volume endoscopists, suggesting that over time they would reach a similar level of proficiency (figure 2).
Thirty-day mortality and 90-day mortality cumulative moving average (CMA) curves for low volume (LV) and high volume (HV) endoscopists, demonstrating similar clinical outcomes during the early component of learning endoscopic mucosal resection.
Endoscopist and hospital volume-outcome RA-CUSUM analysis
The median annual endoscopist EMR volume was 1.9 (range 1–28.6) with 58% of EMR procedures being performed by endoscopists that performed two or less EMR procedures annually.
RA-CUSUM analysis of 30-day mortality and 90-day mortality for endoscopist annual volume of only cancer cases showed significant change points at two procedures per year with reduction in observed mortality from 6.1% to 0.4% (p<0.001) and 12% to 2.1%, (p<0.001), respectively (table 3). There was also a significant reduction in the requirement for emergency intervention in endoscopists that performed more one case per year (1.8% to 0.1%, p<0.002). The endoscopist volume threshold for elective re-intervention in cancer cases was also greater at eight cases with a significant reduction from 24.2% to 17.5% (p=0.033). Analysis of non-cancer cases showed a significant endoscopist volume threshold at one case for 30-day mortality and 90-day mortality, with reduction in observed mortality from 1.3% to 0.6% (p<0.001) and 2.8% to 1.6% (p<0.001), respectively.
To account for the impact of the proficiency gain curve 30-day mortality and 90-day mortality analysis was repeated for endoscopist annual volume after the first four cases. For the cancer cases there was a reduction from 2.5% to 0.2% (p=0.007) and from 6.8% to 1.1% (p<0.001) in 30-day mortality and 90-day mortality, respectively. In the non-cancer cohort there was no obvious annual volume threshold after the first four cases.
The median annual hospital EMR volume was 6.5 (range 1–28.9) with 56.5% of EMR procedures being performed in hospitals that performed eight or less EMR procedures annually. RA-CUSUM analysis of 30-day mortality and 90-day mortality for hospital annual volume of only cancer cases showed significant change points at eight cases per year, with reduction in observed mortality from 5.3% to 1.2% (p<0.001) and 10.8% to 3.4% (p<0.001), respectively. There was also a statistically significant requirement for emergency intervention by hospitals that performed more than five cases per year (1.6% to 0.3%, p=0.016). Analysis of non-cancer cases showed a hospital volume threshold at five cases for 30-day mortality and 90-day mortality, with reductions in observed mortality from 1.3% to 0.7%, (p=0.005) and 2.5% to 2.0% (p=0.065), respectively. There were no statistically significant differences in Charlson Comorbidity Index (CCI) and in patient age between the high volume and low volume hospitals.
Multivariate analysis for 30-day mortality
The Spearman's rank correlation coefficient between annual hospital and endoscopist volume was 0.447 (p<0.001) (table 4). Therefore, to identify the relative importance of annual hospital volume versus endoscopist volume, hierarchical multivariate analysis for patients with cancer (see online supplementary material 2) confirmed that endoscopist annual volume was the only significant factor in reducing 30-day mortality from EMR (OR=0.34; 95% CI 0.17 to 0.68, p=0.002) (table 4), assuming greater significance than hospital volume and cancer surgeon endoscopist. A similar result was observed for the non-cancer cohort with endoscopist annual volume assuming greater importance than hospital volume in reducing 30-day mortality (OR=0.55; 95% CI 0.32 to 0.96, p=0.034). Other factors significantly increasing 30-day mortality included age ≥70 years and Charlson Comorbidity Index score ≥3.
Hierarchical multivariate regression analysis for 30-day mortality following upper GI EMR
Supplementary material
Discussion
Over the study period a threefold increase was observed in the number of patients undergoing oesophagogastric EMR with a marked increase in cancer resection. The endoscopist workload was an independent predictor for mortality. Fifty-eight per cent of EMR procedures were performed by endoscopists with annual volume of two cases or less, with a 12-fold and 6-fold increase in 30-day mortality rate and 90-day mortality rate, respectively, for patients with cancer. The requirement for emergency intervention after EMR was greater with low volume endoscopists. However no patient within the data set received emergency gastrectomy or oesophagectomy within 14 days of the initial EMR, suggesting complications were detected late or were extremely severe. Multivariate analysis demonstrated that risk factors for 30-day mortality and 90-day mortality include endoscopist procedural volume, and patient age and comorbidities including cardiac failure, and renal, peripheral vascular and liver disease. In patients with cancer analysis of elective re-intervention with repeat EMR, oesophagectomy or gastrectomy showed a longer proficiency gain curve and a similarly greater volume threshold to cause a significant reduction in incidence.
Over 50% of EMR procedures were performed by endoscopists that perform less two cases annually, which raises concern over the number of endoscopists who are performing a low number of EMR procedures. About 337 endoscopists (83%) performed <17 cases over the entire study period with the initial period of the proficiency gain curve being the most dangerous component (as the RA-CUSUM curve demonstrated an early change point in mortality). When procedures were undertaken for oesophagogastric cancer, the change point was at four cases with a 15-fold decrease in 30-day mortality following the change point. There were also significant changes in 90-day mortality for both the cancer and the non-cancer groups, and in patients with cancer, emergency intervention reduced following the change point during the initial stage of competency gain. The CMA curves suggest that the low volume endoscopists were learning at a similar rate as their high volume counterparts. The difference in outcomes between low volume and high volume endoscopists can be explained by the fact that low volume endoscopists have not progressed as far along the proficiency gain curve to compensate for the earlier poor outcomes. In cancer cases, the proficiency gain curve for elective re-intervention with repeat EMR, oesophagectomy or gastrectomy for presumed positive resection margin and failure of initial EMR was substantially longer at 43 cases. This highlights the technical challenges in assuring oncological quality and adequate resection associated with EMR, and therefore the substantially longer proficiency gain curve with a high re-intervention rate of 23.6%.
The reduction in 30-day mortality seen with high hospital volume in RA-CUSUM analysis was due to the correlation with high endoscopist volume. Hierarchical logistic regression showed a greater relative importance of high endoscopist volume over high hospital volume in reducing 30-day mortality. In oesophagogastric cancer surgery the established volume-outcome relationship has led to the centralisation of services and a demonstrated improvement in patient outcomes with reduction in surgically related mortality.10 The endoscopist volume-outcome relationship in this study puts forward a compelling argument that endoscopic management of these cancers should also be restricted to high volume endoscopists in centres with specialist multidisciplinary teams where more appropriate patient selection and better clinical outcomes are achievable.
The scale of change in early mortality in the initial stage of competency gain, particularly for cancer, raises concern regarding the method of introducing new endoscopic techniques into clinical practice. Gaining experience at the expense of patient safety is unethical and efforts must be made to avoid mortality as an outcome of proficiency gain. Other end points such as execution time are acceptable parameters. Structured training programmes with competency-based assessment and accreditation process are required at a national level. The trainee's performance of EMR is currently assessed using the Direct Observation of Procedural Skills therapeutic upper GI endoscopy tool and requires formal sign-off by a trainer.12 Trusts are required to submit the outcomes of EMR for cancer and high-grade dysplasia to the National Oesophago-Gastric Cancer Audit. The National Training Program for Laparoscopic Colorectal Surgery in England is a good model where clinical outcomes were not compromised while trainees gained competency.13 During the training programme and in subsequent clinical practice, mortality and complication rates of cases performed by trainees were equivalent to those achieved by experts.14 The most common method of monitoring the introduction of a new technique is through audit that records morbidity and mortality, however audits only detect adverse events after patient harm has occurred. A more sensitive approach is the use of error analysis and near-miss event,15 ,16 to analyse technical performance in order to identify those who require more training to reach proficiency, a standard practice in high reliability organisations.
There are important limitations to this study including its design as a retrospective observational study. The administrative data are limited by the lack of cancer-specific information such as stage with its prognostic implications. However given the utilisation of EMR primarily for early stage cancers, progression of cancer is unlikely to have influenced the mortality data presented in this study. Further it is likely that coding errors may have led to some patients in the non-cancer group being miscoded and actually being treated for oesophagogastric cancer. The limitations of coding are inherent to any national administrative database. Identification of individual practitioners within the HES database was made using a unique anonymised provider-specific code for the endoscopist associated with that hospital episode and discharge. In some circumstances a colleague or a trainee endoscopist may have performed the procedure, and therefore those procedures would have been captured within the senior coded practitioner's RA-CUSUM curve analysis, a limitation shared by all national databases. Further this study has identified changes in mortality associated with endoscopist and hospital procedural volume, however the cause of mortality cannot be accurately assessed using the data provided within this administrative database. Data were extracted over a prolonged time frame to include the whole case series of the majority of endoscopists. It should be recognised there may have been significant changes in EMR technique and practice which may have influenced the results, although the majority of endoscopists started their practice later in the study. Further the primary aim of this study was to retrospectively identify how many cases the endoscopists took to gain proficiency in order to guide training, future practice and possible centralisation of services. Chances in performance over time are not as relevant to this study as we are not prospectively monitoring the current treatment failure rate. Also, learning from other endoscopic procedures not captured within the study may have affected the length of the proficiency gain curves observed. However these are likely to be equally distributed between endoscopists included in the study. Despite the retrospective design of the study and possible confounding factors over time, the conclusions of the study are accurate given the majority of endoscopists started their practice later in the study, and a repeated analysis that factored in changes over time demonstrating similar results (see online supplementary material 3).17
Supplementary material
The introduction of oesophagogastric EMR has been associated with an increase in the number of patients receiving endoscopic management of cancer. The volume of procedures undertaken at an institutional level and most importantly at an endoscopist level is directly related to patient mortality. Appropriate training and volume accreditation are needed to minimise mortality and requirement for re-intervention during the initial stage of proficiency gain.
References
Footnotes
SRM and HM are first co-authors and have contributed equally.
Twitter Follow Laurence Lovat at @GENIE_UCL
Funding SRM is supported by the National Institute for Health Research (NIHR) – Grant Number DRF-2014- 07-088. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Primary data from this study will not be available for sharing from Imperial College. However the primary data may be accessed from National Health Service (NHS) digital.