Article Text

Download PDFPDF

Original research
Prediction model of bleeding after endoscopic submucosal dissection for early gastric cancer: BEST-J score
  1. Waku Hatta1,
  2. Yosuke Tsuji2,
  3. Toshiyuki Yoshio3,
  4. Naomi Kakushima4,
  5. Shu Hoteya5,
  6. Hisashi Doyama6,
  7. Yasuaki Nagami7,
  8. Takuto Hikichi8,
  9. Masakuni Kobayashi9,
  10. Yoshinori Morita10,
  11. Tetsuya Sumiyoshi11,
  12. Mikitaka Iguchi12,
  13. Hideomi Tomida13,
  14. Takuya Inoue14,
  15. Tomoyuki Koike1,
  16. Tatsuya Mikami15,
  17. Kenkei Hasatani16,
  18. Jun Nishikawa17,
  19. Tomoaki Matsumura18,
  20. Hiroko Nebiki19,
  21. Dai Nakamatsu20,
  22. Ken Ohnita21,
  23. Haruhisa Suzuki22,
  24. Hiroya Ueyama23,
  25. Yoshito Hayashi24,
  26. Mitsushige Sugimoto25,26,
  27. Shinjiro Yamaguchi27,
  28. Tomoki Michida28,
  29. Tomoyuki Yada29,
  30. Yoshiro Asahina30,
  31. Toshiaki Narasaka31,
  32. Shiko Kuribasyashi32,
  33. Shu Kiyotoki33,
  34. Katsuhiro Mabe34,
  35. Tomohiro Nakamura35,
  36. Naoki Nakaya36,
  37. Mitsuhiro Fujishiro37,
  38. Atsushi Masamune1
  1. 1 Division of Gastroenterology, Tohoku University Graduate School of Medicine, Sendai, Japan
  2. 2 Department of Gastroenterology, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan
  3. 3 Department of Gastroenterology, Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan
  4. 4 Division of Endoscopy, Shizuoka Cancer Centre, Shizuoka, Japan
  5. 5 Department of Gastroenterology, Toranomon Hospital, Tokyo, Japan
  6. 6 Department of Gastroenterology, Ishikawa Prefectural Central Hospital, Kanazawa, Japan
  7. 7 Department of Gastroenterology, Osaka City University Graduate School of Medicine, Osaka, Japan
  8. 8 Department of Endoscopy, Fukushima Medical University Hospital, Fukushima, Japan
  9. 9 Department of Endoscopy, The Jikei University School of Medicine, Tokyo, Japan
  10. 10 Department of Gastroenterology, Kobe University Graduate School of Medicine, Kobe, Japan
  11. 11 Department of Gastroenterology, Tonan Hospital, Sapporo, Japan
  12. 12 Department of Internal Medicine, Wakayama Medical University, Wakayama, Japan
  13. 13 Department of Gastroenterology and Metabology, Ehime University Graduate School of Medicine, Toon, Japan
  14. 14 Division of Gastroenterology and Hepatology, Osaka General Medical Centre, Osaka, Japan
  15. 15 Division of Endoscopy, Hirosaki University Hospital, Hirosaki, Japan
  16. 16 Department of Gastroenterology, Fukui Prefectural Hospital, Fukui, Japan
  17. 17 Faculty of Laboratory Science, Yamaguchi University Graduate School of Medicine, Ube, Japan
  18. 18 Department of Gastroenterology, Chiba University Graduate School of Medicine, Chiba, Japan
  19. 19 Department of Gastroenterology, Osaka City General Hospital, Osaka, Japan
  20. 20 Department of Gastroenterology, Toyonaka Municipal Hospital, Toyonaka, Japan
  21. 21 Department of Gastroenterology and Hepatology, Nagasaki University Hospital, Nagasaki, Japan
  22. 22 Endoscopy Division, National Cancer Centre Hospital, Tokyo, Japan
  23. 23 Department of Gastroenterology, Juntendo University School of Medicine, Tokyo, Japan
  24. 24 Department of Gastroenterology and Hepatology, Osaka University Graduate School of Medicine, Suita, Japan
  25. 25 Division of Digestive Endoscopy, Shiga University of Medical Science Hospital, Otsu, Japan
  26. 26 Department of Gastroenterological Endoscopy, Tokyo Medical University Hospital, Tokyo, Japan
  27. 27 Division of Gastroenterology, Kansai Rosai Hospital, Amagasaki, Japan
  28. 28 Department of Gastroenterology and Hepatology, Saitama Medical Centre, Saitama, Japan
  29. 29 Division of Gastroenterology & Hepatology, Kohnodai Hospital, National Centre for Global Health and Medicine, Ichikawa, Japan
  30. 30 Division of Gastroenterology, Kanazawa University Hospital, Kanazawa, Japan
  31. 31 Department of Gastroenterology, University of Tsukuba, Tsukuba, Japan
  32. 32 Department of Gastroenterology and Hepatology, Gunma University Graduate School of Medicine, Maebashi, Japan
  33. 33 Department of Gastroenterology, Shuto General Hospital, Yanai, Japan
  34. 34 Department of Gastroenterology, National Hospital Organization Hakodate National Hospital, Hakodate, Japan
  35. 35 Department of Health Record Informatics, Tohoku Medical Megabank Organization, Sendai, Japan
  36. 36 Department of Health Sciences, Saitama Prefectural University, Koshigaya, Japan
  37. 37 Department of Gastroenterology and Hepatology, Nagoya University Graduate School of Medicine, Nagoya, Japan
  1. Correspondence to Dr Mitsuhiro Fujishiro, Nagoya University Graduate School of Medicine, Nagoya 113-8655, Japan; mtfujish{at}


Objective Bleeding after endoscopic submucosal dissection (ESD) for early gastric cancer (EGC) is a frequent adverse event after ESD. We aimed to develop and externally validate a clinically useful prediction model (BEST-J score: Bleeding after ESD Trend from Japan) for bleeding after ESD for EGC.

Design This retrospective study enrolled patients who underwent ESD for EGC. Patients in the derivation cohort (n=8291) were recruited from 25 institutions, and patients in the external validation cohort (n=2029) were recruited from eight institutions in other areas. In the derivation cohort, weighted points were assigned to predictors of bleeding determined in the multivariate logistic regression analysis and a prediction model was established. External validation of the model was conducted to analyse discrimination and calibration.

Results A prediction model comprised 10 variables (warfarin, direct oral anticoagulant, chronic kidney disease with haemodialysis, P2Y12 receptor antagonist, aspirin, cilostazol, tumour size >30 mm, lower-third in tumour location, presence of multiple tumours and interruption of each kind of antithrombotic agents). The rates of bleeding after ESD at low-risk (0 to 1 points), intermediate-risk (2 points), high-risk (3 to 4 points) and very high-risk (≥5 points) were 2.8%, 6.1%, 11.4% and 29.7%, respectively. In the external validation cohort, the model showed moderately good discrimination, with a c-statistic of 0.70 (95% CI, 0.64 to 0.76), and good calibration (calibration-in-the-large, 0.05; calibration slope, 1.01).

Conclusions In this nationwide multicentre study, we derived and externally validated a prediction model for bleeding after ESD. This model may be a good clinical decision-making support tool for ESD in patients with EGC.

  • endoscopy
  • bleeding
  • gastric cancer

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Significance of this study

What is already known on this subject?

  • Bleeding after endoscopic submucosal dissection (ESD) occurs in 4.1% to 8.5% of patients who undergo gastric ESD.

  • Although antithrombotic agents have been regarded as an important risk factor for bleeding, few studies have appropriately evaluated the risk of bleeding after ESD for each type of agent.

  • No prediction models have been reported for bleeding after ESD for early gastric cancer (EGC).

What are the new findings?

  • We derived a prediction model (BEST-J score: Bleeding after ESD Trend from Japan) for bleeding after ESD for EGC using the largest cohort of patients and then externally validated the model using an independent data set.

  • This prediction model was based on a combination of nine variables: 4 points for warfarin and direct oral anticoagulants; 3 points for chronic kidney disease with haemodialysis; 2 points each for P2Y12 receptor antagonist and aspirin; 1 point each for cilostazol, a tumour size >30 mm, lower-third in tumour location and presence of multiple tumours and −1 point for interruption of each kind of antithrombotic agents.

  • This model demonstrated moderately good discrimination and good calibration.

Significance of this study

How might it impact on clinical practice in the foreseeable future?

  • This prediction model can be used in routine practice to help in clinical decision-making to optimise the management after ESD for EGC based on the individual bleeding risk.


Gastric cancer is the fifth most common cancer and 50% to 70% of gastric cancers in Eastern Asian countries are detected at an early stage.1–4 Endoscopic submucosal dissection (ESD) is a minimally invasive treatment method for early gastric cancer (EGC) with almost no risk of lymph node metastasis. However, bleeding is one of the most frequent adverse events in ESD.5 Specifically, bleeding after ESD occurs in 4.1% to 8.5% of patients undergoing gastric ESD.6–11

Thus far, many factors, including male sex, comorbidities, tumour characteristics and procedure time, have been identified as high-risk factors for bleeding after gastric ESD, and among them, antithrombotic (AT) agents are regarded as an important risk factor for bleeding.6 7 12 13 AT agents include antiplatelet agents (APAs) (aspirin, P2Y12 receptor antagonist (P2Y12RA), cilostazol) and anticoagulants (ACs) (warfarin, direct oral AC (DOAC)), and because of their different mechanisms of action, these agents may carry different risks for bleeding after ESD. However, few studies have appropriately evaluated the risk of bleeding after ESD for each type of agent mainly because of the small number of cases. Furthermore, according to the recent Japanese guidelines for endoscopy with AT agents,14 aspirin and cilostazol can be continued or interrupted, P2Y12RA can be interrupted or replaced with aspirin or cilostazol and double antiplatelet therapy can be changed to monotherapy with aspirin or cilostazol. However, no studies that have evaluated the risk factors for bleeding have considered these factors.

Recently, clinical prediction models have been used in several fields to provide an estimate of the value of a therapy for an individual patient.15–17 However, no prediction models have been reported for bleeding after ESD for EGC. The haemostatic procedure for preventing bleeding in ESD sometimes differs among institutions with/without familiarity with ESD and among different geographical areas. Thus, it is desirable to develop a prediction model in a large-scale study that includes patients in various areas. Therefore, we aimed to derive and externally validate a clinically useful prediction model for bleeding after ESD for EGC based on a nationwide multicentre study in Japan.


This study was conducted in accordance with the guidelines of the Declaration of Helsinki and was approved by the Ethics Committee of Tohoku University Graduate School of Medicine (2018-1-48) followed by the institutional review board of each institution before the recruitment of patients. Written informed consent for ESD was obtained from all patients before the procedure. The need for informed consent of this study was waived via the opt-out method on each participating hospital website.

Study design and patient population

To establish a reliable prediction model for bleeding after ESD for EGC, we conducted this retrospective study consisting of three steps: (1) identification of clinically significant predictors of bleeding after ESD for EGC using a large cohort, (2) development of a simple prediction model for predicting bleeding and (3) external validation of the model using an independent data set. This study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guidelines.18

The study population consisted of adult patients who underwent ESD for EGC. Patients in the derivation cohort were recruited from 25 institutions throughout Japan, except from the northern (Hokkaido and Tohoku) and southern (Kyushu and Shikoku) areas, between November 2013 and October 2016 (online supplementary figure 1). The patients were excluded if (1) ESD was not completed, (2) the follow-up duration after ESD was <28 days, (3) closure of the ulcer or the shielding method using polyglycolic acid sheets and fibrin glue was performed after ESD, (4) they refused use of their clinical data, (5) they underwent photodynamic therapy after ESD, (6) invasion of the muscularis propria or a deeper layer was pathologically confirmed or (7) ESD was performed on the remnant stomach. An external validation was conducted in an independent cohort of patients from eight institutions, which are located in the north or south of Japan (online supplementary figure 1) using these same inclusion and exclusion criteria.

Supplemental material

ESD procedure and pathological evaluation

ESD was performed according to a standard ESD procedure19 at all institutions. Briefly, the procedure consisted of the following steps: (1) marking around the lesion, (2) submucosal injection of a solution, (3) mucosal incision outside of the marked region, (4) submucosal dissection and (5) retrieval of the specimen. Haemostasis of active bleeding and prophylactic coagulation of visible vessels on the ESD ulcer were performed using haemostatic forceps or clips during the procedure.

Scheduled second-look endoscopy (SLE) depended on the institution. During and after ESD, proton pump inhibitor, potassium-competitive acid blocker or H2 receptor antagonists was administered to all patients according to the selection of the doctors in charge. In those with intake of AT agents, the decision on continuing or interrupting the agents before ESD and, where present, the timing of interruption, and the use of heparin bridging were mainly based on the Japanese guidelines,14 which have been available in Japan since 2012.

Candidates for predictive factors of bleeding after ESD for EGC

On the basis of a comprehensive literature review, including a prior systematic review,20 and clinical relevance, we identified that the following variables were potentially associated with bleeding after ESD: sex, comorbidities (ischaemic heart disease, liver cirrhosis and chronic kidney disease (CKD) with haemodialysis), AT therapy (aspirin, P2Y12RA, cilostazol, warfarin and DOAC), the tumour characteristics (multiple tumours, lower-third in tumour location, tumour size >30 mm, undifferentiated type, submucosal invasion ≥500 µm from the muscularis mucosa (SM2) and ulceration) and procedure (ESD procedure time >120 min and SLE). We also selected characteristics that may influence bleeding, including age (≥75 years), piecemeal resection and mild/no endoscopic gastric atrophy according to the Kimura and Takemoto classification.21 With regard to the AT agents, the variables for the interruption of AT agents, heparin bridging and replacement of APAs were included in the model because of their possible influence on bleeding.

Data collection and definition

In each participating institution, demographic and clinical data were collected from the medical records and endoscopy and pathology reports.

According to a previous report,22 bleeding after ESD was defined as haemorrhage with clinical symptoms and confirmed by emergency endoscopy from the time of the completion of ESD to 28 days after ESD. We defined clinical symptoms as haematemesis, melena or a decrease in haemoglobin of >2 g/dL since the patient’s most recent laboratory test. Preventive haemostasis of visible vessels without evidence of bleeding during SLE was not regarded as bleeding after ESD.

Interruption of AT agents was defined as stopping AT agents including temporary replacement by heparin and that by aspirin or cilostazol for the other APAs, the latter of which is referred to as replacement of APAs in this study. This variable was evaluated as interruption of each kind of AT agents.

Outcomes and sample size considerations

The primary outcome of this study was to develop and validate a prediction model for bleeding after ESD for EGC. To fit a prediction model using logistic regression, a minimum of 10 events (ie, bleeding after ESD) per variable (EPV) are recommended.23–25 We evaluated 23 variables in the logistic regression model; thus, the sample size in the derivation stage was at least 230 events. The sample size for the external validation of the model was set as a minimum of 100 events, according to the recommendation of the sample size requirement for an external validation model.26 On the basis of the result of the meta-analysis, the assumed rate of bleeding after ESD would be 5.1%.20 Thus, we planned to collect at least 4510 and 1961 cases for the derivation and external validation cohorts, respectively.

In the subgroup analysis, the derived prediction model was evaluated for each centre-volume, which was divided into very high-volume (≥150 eligible patients per year), high-volume (≥100 and <150), intermediate-volume (≥50 and <100), and low-volume (<50) centres.

Statistical analysis

Categorical variables were summarised as n (%) and were compared using χ2 tests. Continuous variables were summarised as medians and 25th to 75th percentiles, which were reported as P25 to P75, and were compared using the Wilcoxon rank-sum test.

In the derivation cohort, univariate and multivariate logistic regression analyses were used to test the associations of 23 candidate variables with bleeding after ESD. Multicollinearity among the variables was tested using the variance inflation factor (VIF). A cut-off value for inclusion of the assessed factors in the final prediction model was predefined as p<0.05. We assigned weighted points proportional to β regression coefficient values multiplied by 2 to the nearest integer for the factors determined in the multivariate analysis. Using these scores, we derived four risk categories, based on the risk of bleeding in previous reports:6–11 27 low (<4.1% risk), intermediate (≥4.1% and ≤8.5% risk), high (>8.5% and ≤15.9% risk) and very high (>15.9% risk). The trend in the risk for bleeding among the four risk groups was evaluated using the Cochran-Armitage trend test. The model’s overall performance was tested using Brier scores and Nagelkerke R2 , and its discrimination was examined using the c-statistic. The derived prediction model was internally validated by bootstrap resampling, which used 1000 random samples drawn with replacement from the original derivation data set.28 29 Decision curve analysis (DCA), as proposed by Vickers and Elkin,30 was performed to assess the net clinical benefit of the models for predicting bleeding after ESD.

External validation was conducted in the validation cohort to analyse the same performance metrics, including overall performance and discrimination. According to the proposal for better clinical prediction model,31 model calibration was tested using the calibration-in-the-large and calibration slope.

In the subgroup analysis, discrimination and calibration were tested separately in very high-volume, high-volume, intermediate-volume and low-volume centres, using the derivation cohort in order to take advantage of its larger sample size.

Patients with missing data for variables included in the final model were excluded from the final derivation and validation cohorts. Data were analysed using SPSS V.23.0 for Windows software (IBM Corp, Armonk, New York, USA) and R software V.3.5.1 (The R Foundation for Statistical Computing, Vienna, Austria). A p value of <0.05 was considered statistically significant. All statistical analyses were performed by an independent statistician (TN). All authors had access to the study data and had reviewed and approved the final manuscript.


Study flow diagram and patient demographics

The flowchart of patient enrolment is shown in figure 1. A total of 8291 patients were analysed as the derivation cohort. For the external validation cohort, independent 2029 patients met the eligibility criteria for this study. The patient demographics are shown in table 1, and the number of enrolled patients at each institution is described in online supplementary table 1. Several baseline characteristics, such as age and cilostazol intake, were different between the derivation and validation cohorts. The bleeding rates after ESD in the two cohorts were 4.7% (387 events) and 5.0% (102 events), respectively.

Figure 1

The flowchart of patient enrolment for the derivation and validation cohorts. (A) Derivation cohort. (B) Validation cohort. ESD, endoscopic submucosal dissection; EGC, early gastric cancer.

Table 1

Clinicopathological characteristics of patients in the derivation and validation cohorts

Model derivation

We included 23 candidate predictors in the univariate and multivariate models. Online supplementary table 2 shows the results of the univariate analyses for each candidate. There was no interaction between any two AT agents (online supplementary table 3). In the multivariate model, 10 variables were identified as predictors of bleeding (table 2). The VIF (1.00 to 4.58) was less than 5 to 10, indicating that the multicollinearity was not present.32 On the basis of the adjusted β regression coefficient, the scores of selected predictors were assigned an integer score: 4 points each for warfarin and DOAC; 3 points for CKD with haemodialysis; 2 points each for P2Y12RA and aspirin; 1 point each for cilostazol, a tumour size >30 mm, lower-third in tumour location and presence of multiple tumours and −1 point for interruption of each kind of AT agents (table 2). Then, we established a prediction model (BEST-J score: Bleeding after ESD Trend from Japan; table 3). The prediction model had good overall performance, with a Nagelkerke R2 of 0.11 and a Brier score of 0.04, and showed moderately good discrimination, with a c-statistic of 0.71 (95% CI, 0.68 to 0.73; table 4). A cut-off of 1 point had high sensitivity for identifying patients with bleeding and a cut-off of 3 to 8 points had high specificity (online supplementary table 4). DCA revealed that this prediction model is useful for threshold probabilities between 3% and 50% (online supplementary figure 2). The model was then internally validated by bootstrap resampling procedure, which gave a mean c-statistic of 0.71 (95% CI, 0.68 to 0.73).

Table 2

Multivariate logistic regression analysis of predictive factors for bleeding after ESD for EGC in the development cohort and the scoring system

Table 3

Distribution of risk scores and risk classification for bleeding after ESD for EGC in the derivation cohort

Table 4

Performance of the prediction model

According to the definition, the total score was categorised as low-risk (0 to 1 points), intermediate-risk (2 points), high-risk (3 to 4 points) or very high-risk (≥5 points) for bleeding after ESD. As a result, the rates of bleeding for each risk category were 2.8%, 6.1%, 11.4% and 29.7% (table 3), and a significantly increasing trend of risk from low-risk to very high-risk groups was observed (p<0.001, Cochran-Armitage trend test). The c-statistic of the risk classification was 0.68 (95% CI, 0.65 to 0.71).

Because ACs and CKD with haemodialysis were the strongest risk factors for bleeding after ESD, we evaluated the prediction ability of a simple model by combining these two factors. However, this model had only modest discrimination ability (c-statistic, 0.61; 95% CI, 0.57 to 0.64; online supplementary table 5). In DCA, the net benefit of the simple model in predicting bleeding after ESD was lower than that of the risk score and risk classification (online supplementary figure 2).

Interruption of AT agents

The time of interrupting AT agents before ESD, the time of resuming them after ESD and the duration of interrupting them differed according to AT agent (online supplementary table 6). Thus, we evaluated the effect of interruption of AT agents on bleeding after ESD, based on the time of interrupting/resuming AT agents and the kind of them. Statistical significance was observed only for ≥6 days interruption of aspirin before ESD (OR, 0.39; p=0.020), although adjusted ORs of interruption for bleeding tended to differ across the AT agents (online supplementary table 7).


A 65-year-old male, who takes warfarin and aspirin, has an EGC with a maximal diameter of 22 mm, which is located in the middle-third of the stomach.

Total score=4 (for warfarin)+2 (for aspirin)=6.

Therefore, this patient falls in the very high-risk group with a total score of 6 (41.5% risk of bleeding after ESD). When warfarin and aspirin are interrupted (=−2 points), the total score decreases to 4 points (14.5% risk of bleeding).

Model validation

In the external validation cohort, the distribution of the prediction model and the bleeding rates are shown in table 5. The discriminative ability of the prediction model as measured by the c-statistic in the external validation cohort was 0.70 (95% CI, 0.64 to 0.76). This model was well calibrated with a calibration-in-the-large of 0.05 and a calibration slope of 1.01, and the overall performance was also good (table 4).

Table 5

Distribution of the total risk score and risk classification for bleeding after ESD for EGC in the validation cohort

Subgroup analysis

Model discrimination of the BEST-J score in very high-volume, high-volume and intermediate-volume centres was moderately good with c-statistics that ranged from 0.70 to 0.72; however, in low-volume centre, it was modest (c-statistic, 0.69) (table 6). The results pertaining to the model calibration are presented in table 6.

Table 6

Discrimination and calibration of the prediction model based on the centre-volume in the derivation cohort

Management after ESD

In the whole cohort, the periods of hospital stay after ESD for each risk category in those with and without bleeding after ESD are shown in online supplementary figure 3. The median periods of hospital stay in patients with bleeding after ESD in the low-risk, intermediate-risk, high-risk and very high-risk categories were 8, 8, 10 and 12 days, respectively, and those without bleeding were 6, 6, 7 and 7 days, respectively. The differences were significant in all risk categories (p<0.001).

We also evaluated the effect of SLE for bleeding after ESD in each risk category in the whole cohort. As a result, no significant difference in the bleeding rate after ESD was observed between patients with and without SLE in all the risk categories (online supplementary table 8).


Given the relatively low incidence of bleeding after ESD for EGC, the availability of a prediction score would be of extreme importance. In the present study, we evaluated 23 candidate predictors and found 10 independent factors associated with bleeding. Our study highlights the strong impact of ACs, that is, warfarin and DOAC, on bleeding among AT agents. In addition, we first found the risk of P2Y12RA and cilostazol for bleeding after ESD. Furthermore, the impact of CKD with haemodialysis on bleeding was lower than that of ACs but higher than the impact of any kind of APAs. This insight may be informative in the management of patients after ESD.

We also revealed that the interruption of AT agents decreased the bleeding risk to an extent depending on the number of kinds of AT agents. This may also be useful information for reducing the bleeding risk. However, since the recommended time of interruption before ESD differs among AT agents, we further evaluated the association between time of interruption/resumption for each AT agent and the effect of preventing bleeding. Although most variables did not reach statistical significance mainly due to the small number of cases by subdivision, the OR of interruption for bleeding tended to differ among the kinds of AT agents. Especially, heparin bridging was not a risk factor for bleeding in the present study, which is contrary to the previous studies.27 33 Indeed, heparin was not initiated after ESD in 18.4% (36/196) of the patients with heparin bridging, and the patients who did receive heparin after ESD stopped heparin at a median of 2 days after the procedure, whereas dalteparin was initiated at 1 to 3 days after ESD with a mean dosing period of 2.5 days in a previous study.33 Therefore, caution is required when interpreting the risk of heparin bridging in the present study.

To date, two studies created the predictive models for bleeding after ESD. Toya et al reported the usefulness of combining the factors of ACs and resection size ≥35 mm for predicting bleeding after ESD for EGC.13 However, this study has the limitations of not evaluating the role of APA in detail and a small sample size from a single institution. A recent study established a risk-scoring model for predicting bleeding after colorectal ESD.34 In this model, points were assigned to use of AT agents except for aspirin alone and tumour size ≥30 mm, in addition to rectosigmoid area. However, because of low event rates and small sample size from a single institution, the variables of ACs and CKD were not adequately evaluated and the discrimination ability of this model (c-statistic, 0.634) was limited; a c-statistic of ≥0.70 is typically considered sufficient to make clinically useful individual predictions.35 In terms of existing bleeding scores, several scores for patients with atrial fibrillation are also available.36–38 However, these scores also have only modest discrimination (c-statistic ≤0.63) in the external validation cohort.38–40 Our prediction model (c-statistic, 0.70) satisfied the clinically useful discrimination ability in the external validation cohort.35 However, the c-statistic just reached its satisfactory value; thus, it should be noted that our model may not be very reliable. Moreover, since the lower limit of 95% CI was <0.70, further external validation using a larger cohort is required for elucidating whether this model has an acceptable discrimination ability or not. Nevertheless, since this model was established based on the analysis of almost all known key candidate predictors for many patients throughout Japan and achieved good results in the overall performance, calibration ability and net benefit, we believe that this model could be applied as a simple aid to clinical decision-making in routine practice.

We here provide a new proposal for patient management after ESD, which is based on individual bleeding probabilities in the prediction model. Despite that patients at a low risk for bleeding can be safely managed, we found that the period of hospital stay of patients who underwent ESD without bleeding, which reflects the standard schedule of hospital stay for each risk category, did not differ among the risk categories (6 to 7 days was the median for the four categories). Thus, we propose a shortened hospital stay (eg, 3 to 4 days after ESD) in patients with low risk, which may contribute to lowering the cost.

By contrast, some clinical approaches are recommended for patients at high risk. First, when such patients take AT agents, interruption of these can reduce the risk of bleeding and the effect of interruption depends on the number of kinds of AT agents interrupted, although the risk of thromboembolism by the interruption should also be considered. Second, caution would be given to endoscopists to look for possible bleeding sources, which would lead to more careful prophylactic coagulation of visible vessels on mucosal defect after resection. Furthermore, intensive monitoring of high-risk patients may prevent the onset of severe conditions even if bleeding occurs. On the other hand, SLE did not help prevent bleeding after gastric ESD, which is in accordance with a previous report in patients without AT agents.22 According to a recent study, the polyglycolic acid sheet with fibrin glue could not also decrease the risk of bleeding after ESD.41 However, several studies have reported the effect of mucosal defect closure for preventing bleeding after endoscopic resection.42–44 Thus, patients with a high risk for bleeding are candidates for this technique after resection. Although endoclips are sometimes insufficient to maintain closure of a large mucosal defect after ESD,45 46 recent advanced techniques such as endoloop,47 hand-suturing46 and OverStitch endoscopic suturing technique48 may overcome this issue.

Several strengths of the current study warrant mention. First, the designs of the derivation and the external validation are robust, according to the TRIPOD guidelines.18 The method also conformed with recent developments for improved prediction model assessment, including DCA.31 Furthermore, the sample size in this study was calculated based on the recommendations for deriving and externally validating a prognostic model,23–26 thereby increasing the reliability of the prediction model. Although we enrolled twice as many patients as the preplanned sample size, which was based on 10 EPV, in the derivation stage, this increase in sample size may have led to the better prediction model, because model performance was reported to be better as EPV increases from 10 to 50.23 Second, the data for deriving a prediction model were obtained from the largest cohort. Third, missing values were minimal, and actually, whole data are available for over 99% of the enrolled patients. Finally, the management of the AT agents during the perioperative period in the enrolled patients was mainly based on the recent Japanese guidelines,14 which are similar to the European and American guidelines.49 50

This study has several potential limitations. First, this study is retrospective in nature. Second, the prediction model was developed and externally validated in Japan, which leads to two potential issues. One of them is that it is unclear whether this model is applicable to other geographical areas, such as Europe. The other issue is that points assigned to each variable may be different when the model is derived from other areas with less ESD expertise. Hence, fully independent validation in these areas is required and, if this prediction model has the poor performance, a new prediction model using the international cohort should be considered. Third, the timing of bleeding after ESD was not considered in this prediction model. Moreover, some reports showed that risk factors for early and late bleeding were different.6 Fourth, we should be careful about interpreting the risk of heparin bridging, as described previously. Lastly, our subgroup analysis in the low-volume centre did not reach a satisfactory value of discrimination; however, this analysis did not have enough power because of the small number of events.

In summary, we derived and externally validated a prediction model (BEST-J score) for bleeding after ESD for EGC, with good performance metrics. We, therefore, believe that this model can be used in routine practice to optimise the management of patients after ESD based on individual bleeding risk. However, further external validation in other geographical areas is required.


We thank Yohei Ogata (Tohoku University Graduate School of Medicine) for his insightful comments and advice for statistical analyses in this study. We also thank all collaborators in the Fight-Japan study group for the enrollment of patients and data collection: Sho Shiroma (Cancer Institute Hospital, Japanese Foundation for Cancer Research), Hiroyuki Ono (Shizuoka Cancer Centre), Hiroyuki Odagiri (Toranomon Hospital), Kazuhiro Matsunaga and Shigenori Wakita (Ishikawa prefectural central hospital), Shusei Fukunaga, Masaki Ominami, and Taishi Sakai (Osaka City University Graduate School of Medicine), Yuko Miura (The University of Tokyo), Minami Hashimoto, Jun Nakamura, and Ko Watanabe (Fukushima Medical University Hospital), Ryusuke Ariyoshi (Kobe University Graduate School of Medicine), Yutaka Okagawa, Takeyoshi Minagawa, and Ryoji Fujii (Tonan Hospital), Takao Maekita and Kazuhiro Fukatsu (Wakayama Medical University), Yoichi Hiasa (Ehime University Graduate School of Medicine), Daisuke Chinda, Hidezumi Kikuchi, and Tetsuya Tatsuta (Hirosaki University Hospital), Atsushi Goto (Yamaguchi University Graduate School of Medicine), Daisuke Maruoka, Kenichiro Okimoto, and Naoki Akizue (Chiba University Graduate School of Medicine), Tomoaki Yamasaki, Takehisa Suekane, and Yu Yasui (Osaka City General Hospital), Tsutomu Nishida and Masashi Yamamoto (Toyonaka Municipal Hospital), Keiichi Hashiguchi and Naoyuki Yamaguchi (Nagasaki University Hospital), Yoichi Akazawa and Hiroyuki Komori (Juntendo University School of Medicine), Yoshiki Tsujii, Hideki Iijima, and Tetsuo Takehara (Osaka University Graduate School of Medicine), Masaki Murata (Shiga University of Medical Science Hospital), Takashi Ohta (Kansai Rosai Hospital), Hidehiko Takabayashi (Saitama Medical Centre), Yoshiyuki Itakura (Kohnodai Hospital, National Centre for Global Health and Medicine), Kazuya Kitamura (Kanazawa University Hospital), Daisuke Akutsu (University of Tsukuba), and Toshio Uraoka (Gunma University Graduate School of Medicine).



  • Contributors All authors listed have contributed substantially to the design, data collection and analysis and editing of the manuscript. Conception and design: WH; acquisition of data: WH, YT, TY, NK, SH, HD, YN, TH, MK, YM, TS, MI, HT, TI, TM, KH, JN, TM, HN, DN, KO, HU, YH, MS, SY, TM, TY, YA, TN, SK, SK, KM; analysis and interpretation of data: WH; drafting of the manuscript: WH; critical revision of the manuscript: YT, MF, TK, HS, NN, AM; statistical analysis: TN; study supervision: MF, AM.

  • Funding The current study was partially supported by the Japanese Foundation for Research and Promotion of Endoscopy Grant.

  • Competing interests MF declared that he received lecture honoraria from Takeda Pharmaceutical, EA Pharma, Nihon Pharmaceutical and his department received research grants from HOYA Pentax, EA Pharma, Eisai, Taiho Phamaceutical, AbbVie GK, Nippon Kayaku, Chugai Pharmaceutical, Gilead Sciences, Kyorin Pharmaceutical, Mitsubishi Tanabe Pharma outside the submitted work.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

  • Patient consent for publication Not required.

  • Ethics approval The study protocol was approved by the ethics committees of the respective institutions.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement The data that support the findings of this study are available from the first author, WH (ORCID, 0000-0001-9717-0281), upon reasonable request.