Statistics from Altmetric.com
Costa and colleagues (Gut 2005;54:364–8) recently reported a study describing the ability of faecal calprotectin to predict relapse in the following year in patients with inflammatory bowel disease (IBD). They concluded that a calprotectin level >150 µg/g was predictive of relapse in Crohn’s disease (CD) and in ulcerative colitis (UC), but was more effective in predicting relapse in UC. Unfortunately, we believe that the authors failed to demonstrate these two points.
If faecal calprotectin >150 µg/g was clearly predictive of relapse in UC patients, this was not the case in CD (p = 0.07 and p = 0.31 for the likelihood ratio test in univariate and multivariate analyses, respectively). This may be due to the method used to determine the cut off value for calprotectin. Firstly, the receiver operating curve (ROC) method did not provide any cut off value for CD as the curve was not different from the diagonal and the confidence interval of the area under the curve included 0.5 (0.40–0.77). Secondly, the ROC curve method was not appropriate as it does not take into account the time to relapse, in contrast with the proportional hazards model used to test the predictive value of calprotectin. Classical methods related to time to relapse should have been preferred.1,2
The assertion, both in the title and in the text, that calprotectin was a stronger predictive marker of relapse in UC than in CD was not statistically tested by the authors. This assertion probably came from the high value for the hazard ratio in UC, compared with that in CD, but theses values are misleading because of the exponential transformation of the coefficient in the proportional hazard model. When roughly calculating these coefficients and their standard error, the figures are much less convincing. In the univariate analysis the results are 1.39 (0.76) for CD and 2.55 (0.75) for UC, and the comparison between these two estimates gives a p value of 0.28 (p = 0.15 with estimates from the multivariate analysis). These disappointing results may be the consequence of a lack of power due to the relatively small number of patients.
Another important point is that the analysis was based on the assumption that the biomarker is able to predict relapse with the same strength whether the relapse occurs early after evaluation or later during follow up. If this is true it means that the calprotectin level is a characteristic of the disease, including the whole 12 month follow up period. As discussed by the authors, calprotectin, as well as erythrocyte sedimentation rate (ESR) or C reactive protein (CRP), are probably markers of the degree of infra-clinical disease activity at the time of their measurement, and therefore can change with time in a given subject. To test this hypothesis, it should have been verified that their hazard ratios varied with time during follow up (the power of this analysis will however be limited).
Comparison of calprotectin with other classical predictive markers is also debatable. Indeed, cut off values for calprotectin were assessed using ROC curves, with some success for UC, and were three times higher than the upper limit of the normal range. In contrast, for ESR and CRP, the upper limits of the normal range were chosen as cut off values, following failure of the ROC curve method which was unfortunately not appropriate.
Finally, the authors stated that three variables were significant predictors of relapse—namely, calprotectin level, smoking habit, and UC activity index (UCAI) or CDAI—whereas only calprotectin and CDAI were found to be independently correlated to time to relapse in UC and CD, respectively.
In conclusion, if we agree with Pardi and colleagues3 that identification of biomarkers predictive of relapse could have important implications for the management of IBD patients, we are less convinced by the data presented by Costa et al with regard to the methodological weaknesses of their study.
We thank Lemann and Mary for their comments on our article (Gut 2005;54:364–8). We appreciate their careful reading of the text, and their questioning of the validity of our study gives us a unique opportunity for further articulating our findings.
We agree that other methods could be used instead of the receiver operating curve (ROC) to assess a cut off value for calprotectin. As Lemann and Mary noted however, the chosen cut off value of 150 µg/g proved to be optimal in ulcerative colitis (UC) patients. In our opinion, no unique cut off value, however carefully chosen, could improve on the prediction of relapse in Crohn’s disease (CD) patients. Perhaps assessment based on a continuous, rather than a binary, score might provide a somewhat better alternative. Evaluation of predictive models of time to relapse, if worthwhile, would require a larger sample size and it was beyond the scope of our study.
Also, we agree with Lemann and Mary that only calprotectin and CD activity index (CDAI) were found to be independently correlated with time to relapse in UC and CD, respectively. Nevertheless, the important role of smoking habit and UCAI should have been explicitly referred to as confounding. The proportionality of the hazard over time was evaluated to some extent as part of testing the interaction terms for all of the variables. As acknowledged in the letter, the power of this analysis was however limited.
We disagree with Lemann and Mary if they wish to downplay the remarkable difference between the diagnostic groups. Firstly, we strongly discourage comparing estimates for coefficients from the univarate analysis. The conspicuous confounding effect of smoking and CDAI in CD patients makes the crude estimate for the coefficient associated with calprotectin >150 µg/g useless for making any meaningful inference. Secondly, comparing estimates from the multivariate models yielded a p value of 0.10, not 0.15 as reported in the letter. Given the relatively small sample size and the inherent lack of power, appropriately pointed out, such a p value should not be overlooked. Thirdly, it makes no difference to the p value whether hazard ratios or regression coefficients are compared, and we believe that the former are easier to interpret than the latter. Fourthly, the lack of power can certainly explain the fact that the sizeable hazard ratio of 2.2 in CD patients was not statistically significant. But the p value should not divert attention from the estimated magnitude of the effect and its confidence interval.
In conclusion, although our findings should not be considered definitive, they are highly suggestive that a calprotectin level >150 µg/g is predictive of relapse in CD and in UC, but is more effective in predicting relapse in UC.
Conflict of interest: None declared.
Conflict of interest: None declared.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.