Operative mortality in colorectal cancer: prospective national study

Data sources

The colorectal cancer study of the Association of Coloproctology of Great Britain and Ireland (ACPGBI) was conducted prospectively in 73 hospitals; participating surgeons joined the study on a voluntary basis. Data were collected locally, either by data managers dedicated to colorectal cancer or by participating surgeons, using the standardised ACPGBI dataset2 or equivalent on all patients presenting to the local hospitals with a new diagnosis of colorectal cancer. Data were submitted centrally by using electronic media and recorded on a Microsoft Access 2000 database (Microsoft Corporation, Seattle, USA). The data in the patients' records were edited extensively to check for missing values or values that were out of range and inconsistencies between data fields. The contributing hospitals reported a total of 8077 new cases of colorectal cancer over a 12 month period between 1 April 1999 and 31 March 2001.

Inclusion and exclusion criteria

We included data on patients undergoing emergency or elective surgery for colorectal cancer. We excluded patients with a diagnosis of colorectal cancer who did not undergo surgery and patients of whom only the name and demographic details were recorded on the database, without information on outcomes or risk factors.

Study end point and risk factors

The primary outcome was operative mortality, defined as death occurring within 30 days of an operative procedure, from whatever cause, occurring either in hospital or after discharge from hospital. The risk factors related to patients and procedures in the ACPGBI database included age; sex; American Society of Anesthesiology (ASA) grade3; cancer site in accordance with the World Health Organization's international classification of diseases and related health problems; surgical procedure categorised according to the UK classification of operative procedures (OPCS-4)4; operative urgency classified as emergency, urgent, scheduled, or elective according to the classification of the National Confidential Enquiry into Perioperative Deaths (NCEPOD)5; cancer staging according to the clinical findings and histological Dukes's classification as modified by Turnbull et al6; timing of surgery during a 24 hour period; cancer excision; and annual case volume.

Statistical analysis

We used unifactorial logistic regression to identify risk factors related to the operative mortality within 30 days. We grouped continuous variables into subcategories of increasing operative risk and used unifactorial logistic regression to compare these with a reference category (table 1). To maximise the information extracted from the predictor variables we used a median imputation technique to substitute for incomplete data.7 This method allows each case to be used by the multifactorial model, thereby reducing the 95% confidence intervals around the model estimates and odds ratios.

ACPGBI colorectal cancer model —We used multifactorial logistic regression analysis to adjust for multiple risk factors, their interactions, and the clustering of adverse outcomes in hospitals (see appendix 1 on bmj.com). We subsequently used the coefficients derived from the multivariate analysis as weights in the derivation of the ACPGBI colorectal cancer score (table 2).

Model validation— We validated the model internally by dividing the data into two distinct sets. We developed the model on 60% of the study population, selected randomly, and the remaining 40% of patients used for testing the model. We used the Hosmer-Lemeshow statistic8 to evaluate the performance of the ACPGBI colorectal cancer model to assess calibration or goodness of fit (the ability of the model to assign the correct probabilities of outcome to individual patients). To obtain this statistic we computed the estimated probability of death for each patient on the basis of the model, ranked them into 10 equal groups of ascending operative mortality, and then statistically evaluated the expected and observed number of outcomes in each tenth. Smaller values represent better calibration of the model. Model discrimination refers to the ability of the model to assign higher probabilities of death to patients who actually die than to patients who live: we measured this by the area under the receiver operator characteristic curve or c-index. Values ranging from 0.7 to 0.8 represent reasonable discrimination, and values exceeding 0.8 represent good discrimination.9 We analysed subgroups by comparing the observed and expected 30 day operative mortality for individual procedures on the validation set of 3000 patients.

Statistical software— We used the following software packages for our analyses: Intercooled STATA 6.0 for Windows (STATA Corporation, College Station, Texas, USA), Statistical Package for the Social Sciences, version 11 for Windows (SPSS, Chicago, Illinois, USA), and MLwiN Version 1.2 (University of London, United Kingdom).

Validity of the data

An essential prerequisite for the evaluation of healthcare outcomes is the provision of comprehensive, accurate, clinically valid, and reliable information. Although currently the quality of the data may be limited by the observational nature of this study, implementation of such a system should increase awareness of the minimum required data, with subsequent improvement in quality. The study was based on data that we had carefully and voluntarily collected from hospitals throughout the United Kingdom. Since these hospitals were self selected, inferences cannot be made about how representative the outcomes are for the overall population. Although the model was internally validated for the study population, it would require external validation by testing the model on further prospective series in different hospitals. As data on an increasing number of cases are collated centrally the colorectal cancer model is expected to evolve from year to year, with individual “hospital effects” becoming more important in modelling the variability of outcome between hospitals or regions. At present the sample size of the cases submitted by individual hospitals is not sufficient to draw any meaningful inferences about the quality of care between providers. In the future, model reliability (the ability to obtain similar values for the same variable in specific patients at different times or in different locations) can be formally assessed, both centrally and at the local level.

Quality of statistical analysis

The ACPGBI colorectal cancer model used a relatively new technique called hierarchical regression analysis. This is particularly useful in modelling observations that have a hierarchical or clustered structure, as they allow for the possibility that patients from the same hospital have more similar outcomes than patients chosen at random from different hospitals. This is often found in studies of patients within hospitals or pupils within schools.11 Multilevel models have been shown to provide more accurate predictions of outcome, and similar techniques have been adopted to display the comparative performance of organizations in cardiac surgery,12 and clinics for in vitro fertilisation.13

Usefulness of the ACPGBI colorectal cancer model and future developments

In the model adjustment was made for age, ASA grade, Dukes's stage, urgency of the procedure, and cancer excision, which are well established, independent, prognostic factors in colorectal cancer.14 In addition, we introduced an interaction term, which was clinically relevant and clearly improved performance of the model. Although ASA grade is simple to use and has been incorporated by other predictive indices,15 it is open to subjectivity and manipulation. In future years the ACPGBI dataset can be augmented with other measures of patients' comorbidity, such as the risk factors from the POSSUM physiological score16 or from the Veterans Affairs surgical risk model.17 The drawback of using specialised investigations such as albumin, prealbumin, or liver function tests is that, although they may be readily, prospectively, collected in elective situations, in the emergency setting data collection and score generation may be incomplete and lead to incorrect assessment of operative risk.18 Further work is therefore required to evaluate the accuracy, reliability and cost involved in collecting such data in addition to the ASA grade. Although POSSUM and p-POSSUM systems have been widely applied in general and colorectal surgery,19 20 disease specific models have been shown to outperform models designed to predict over a range of different conditions. The weights assigned to clinical variables depend on the specific disease context, such as colorectal cancer.21

We describe an accurate and validated model for providing risk adjusted outcomes in patients undergoing surgery for colorectal cancer. The study represents a valuable tool in the multidisciplinary management of cancer patients. The method used in the development of the ACPGBI colorectal cancer model can be applied to other surgical specialties for the objective quantification of operative risk in patients with and without cancer.

The statistical model used by the Association of Coloproctology of Great Britain and Ireland and a list of hospitals participating in data collection appear on bmj.com