Send letter to journal:
Re: Response to Dr A R Weale et al

Dear Editor

the critical letter of Drs Weale and Rees merits some comments:

Multicenter clinical studies on probiotics are just starting up. As is normal practise, they have been preceded by some pilot studies. It is obvious that before investing a large amount of money in a multicenter study, pharmaceutical companies need to know that the effectiveness of the drug under examination is supported by both theory and experimental data. Positive pilot studies on new treatments get published easily, while the negative ones are accepted by journals only with difficulty. In this case, GUT has the merit of having avoided this publication bias. A pilot study is usually based on small group of patients and is necessarily statistically defective. All this is well known. To criticize a pilot study on these grounds is therefore otiose.

The time of randomisation is not specifically mentioned in the article, nor in the discussion, but it is implicitly clear that it took place immediately before surgery. The patients, moreover, were recruited, followed up, operated on, and endoscopised by the authors of the article. The statistical analysis was performed at the request of one of the referees. I do not think that with this kind of results, based on a small number of patients, a complex statistic was really necessary. As we have written in the discussion, a type II error is obviously possible. The merit of the study, finally, is to dampen slightly the enthusiasm for probiotics; it enlarges, also, the field of research by questioning not only the usefulness of probiotics in IBD, but also which type of IBD and which type of probiotic can successfully be employed

Cosimo Prantera

Send letter to journal:
Re: Repeating the same mistakes as other studies

Dear Editor

We read the study by Prantera and colleagues, with great interest. As the authors assert, there have been few good quality and well designed clinical trials, with sufficient power, to examine the effects of probiotics on inflammatory bowel disease.

This trial unfortunately also has a number of weaknesses both in its reporting and methodology.

The report does not make clear the exact time point at which patients were recruited to the trial and randomized. As such recruitment bias may be introduced. The issue of allocation concealment is also ignored in the paper. Furthermore, at no point do the authors fully address the difficult issue of standardisation, be it regarding the patient's disease, the surgeon, the endoscopist or the assessors of clinical outcome.

The authors do not report any attempt to estimate the necessary sample size for the study. The absence of a power calculation or confidence intervals leaves the authors open to the same criticism that they have levelled at other studies. The statistical analysis used is not an intention to treat analysis. We also question whether a chi-squared test was used to compare differences in proportions rather than a z test as stated in the paper.

The study concludes that probiotics are ineffective in preventing recurrence after curative resection. Indeed endoscopic recurrence was 25% greater in the probiotic group, although this was not statistically significant. However this conclusion is based on the analysis of just 37 patients. Clearly, the possibility of a type II error must be acknowledged. The authors do accept that this must be considered a pilot study.

Given the possible biases and lack of power in this study, it should not be considered to be Class Ib evidence. We feel this study re-inforces the fact that a well designed, multi-centre randomised controlled trial is needed to test the hypothesis of the whether probiotics are of any benefit in Crohn's Disease.