Navigation through the colon with a flexible endoscope is technically demanding, like many other image-based procedures [1]. It requires a high level of both psychomotor and visual–spatial skills. Consequently, trainees need a great amount of hands-on experience to master colonoscopy skills. Traditional assessment and accreditation methods are based mainly on a minimal number of supervised procedures, after which average trainees are expected to have achieved a sufficient level of proficiency. Although there are recommendations regarding these minimal numbers [2–6], the suggested minimums differ considerably [7, 8]. There is a growing need for more objective methods of proficiency assessment, and a desire for training until a predetermined level of actual proficiency is achieved [9–11].

In addition, training of basic endoscopy skills in a clinical setting is losing acceptance because of ethical and economic considerations [9, 11–13]. This makes it necessary for novice endoscopists to train in the fundamentals of colonoscopy in a skills lab setting.

Virtual reality (VR) simulators could provide an effective alternative for clinical training and for objective data supplied to educators about the proficiency of their trainees. Currently, VR simulators are obtaining an increasingly prominent position in medical education, and they have enhanced training programs for endoscopic skills [1, 14]. They still are being thoroughly evaluated because their application must be proven valid before their widespread integration into education and training programs [12, 15, 16].

Most VR simulators record multiple performance parameters assumed to provide objective insight into the proficiency level of the trainee. Some of the parameters provided by the simulators are calculated using multiple variables recorded by the simulator (see Appendix A). Currently, one of the major issues concerning the application of VR simulators in training program is to determine which types of exercises are most appropriate and which combination of performance parameters best represents performance [9, 10].

Several VR simulator systems are currently available for lower gastrointestinal flexible endoscopy [12, 15, 16]. The validity of the different systems for lower gastrointestinal endoscopy skills has been studied previously [7, 8, 12, 15, 17–25]. However, most of these studies did not focus on the basic tasks first, but included multiple (complex) tasks or used a relative small numbers of participants. On the basis of our experience with validation of simulators for assessment and training in laparoscopic skills [16, 26], we evaluated the GI Mentor II (Simbionix Ltd., Lod, Israel) in terms of basic navigation skills for colonoscopy and proved its construct validity and didactic value [20].

However, little is known about the improvement of colonoscopy skills by repetitive training on the GI Mentor II. This study aimed to gain insight into the first part of the proficiency curve for basic endoscope navigation skills during training on the GI Mentor II.

Materials and methods

This study investigated several aspects of the learning trajectory with the GI Mentor II simulator. First, the performance of 30 novice endoscopists (with no prior flexible endoscopy experience) on the simulator was assessed over a series of training sessions. Second, the performance of 5 expert endoscopists was investigated for the same series of training sessions. Third, the scores of the novice participants were compared with those of the 20 experienced (200–1,000 colonoscopy procedures performed) and the 40 expert (>1,000 colonoscopy procedures performed) endoscopists to assess their performance within a wider context.

Participants

The participants in this study were 30 novices in flexible endoscopy (10 men and 20 women with a mean age of 25.5 years), all of them medical interns (n = 23) or residents in training (n = 7). Five expert endoscopists (all men with a mean age of 46.2 years) also performed the four training sessions. In addition, the performance data for 20 experienced and 35 expert endoscopists who participated in our validation study was used [20]. The tasks performed on the simulator in the validation study were exactly the same as those performed in the first training session of the study presented in this report [20].

Simulator

The GI Mentor II VR simulator for flexible endoscopy GI Mentor II (Simbionix, Ltd.; Israel, software version 2.7.3.0) (Fig. 1) was used in this study. The GI Mentor II provides hands-on training by various modules for training in basic psychomotor endoscopy skills as well as lower and upper flexible endoscopy procedures on a mannequin with a mouth and a rectal end. The endoscope used is a customized Pentax ECS-3840F endoscope (Pentax Corp., Tokyo, Japan). The steering and torque of the endoscope are controlled as in real endoscopy. Insufflation and suction also are available. The computer simulation program supplies visual and audio feedback while dynamic force feedback devices inside the mannequin provide force feedback sensations, all corresponding to the selected training module and patient scenario. The patient scenarios for VR endoscopy vary in anatomy and pathology. The simulator provides objective measurements and statistics about each performance.

Fig. 1.
figure 1

The GI Mentor II virtual reality simulator.

Full size image

Protocol training sessions

The proficiency of the participants in basic endoscope navigation was assessed during four preset training sessions (1 per day) within 5 consecutive days (Fig. 2). Each participant performed one hand–eye coordination task (EndoBubble level 1) per session. Each of the training sessions also involved multiple different VR colonoscopy cases with varying levels of difficulty to avoid bias resulting from training on only one patient scenario. The participants were not notified about the repetitive nature of the last VR colonoscopy in each session. Because of its discriminatory value, VR colonoscopy I-3 was selected as a repetitive exercise. It is a fairly complicated case with a relatively winding sigmoid and a built-in loop in the ascending colon and hepatic flexure [20]. The performance of the repetitive exercises within each session is defined as a run.

Fig. 2.
figure 2

The study design.

Full size image

Because the study focused on the manipulation and navigation skills of the participants, the assignment for the VR colonoscopy exercises was to visualize the cecum as quickly as possible with as little patient discomfort as possible. When the participant reached the cecum, the exercise was considered accomplished. The participants were instructed not to identify or treat the pathologies presented in the cases. No feedback on performance was given other than that produced by the simulator in full-screen mode.

Before the first session, the participants completed a questionnaire on demographics and their general medical and endoscopy experience. Next, they received an introduction about the simulator and an explanation on how to operate the controls and steer the endoscope tip. The tour and explanation were given by the researcher following a preset objective procedure. Questions concerning the functioning of controls were answered when asked during the training sessions, but no instructions were given on how to optimize performance. The participants were informed about the parameters recorded by the simulator, and their scores were shown to them after each exercise. All the participants performed the tasks on the simulator single-handed without nurse assistance for scope insertion.

Performance parameters

A broad range of variables is recorded by the simulator. However, the time required to accomplish the assignment (time required to reach the cecum in VR colonoscopies or the time required to finish the EndoBubble task) and the (estimated) percentage of time the patient experienced excessive pain were considered key parameters for this study (Table 1).

Table 1. Performance of novice participants on the EndoBubble level 1 task and VR colonoscopy I-3 per session
Full size table

The (estimated) percentage of time the patient experiences excessive pain is a composite parameter calculated by the simulator using several pain-related variables. When the pain level exceeds 0.6, excessive pain is counted. If the level remains for 15 s, excessive pain is recorded again. See Appendix A for a more extensive explanation on the calculation of this parameter.

Statistical analysis

The SPSS 11.0.1 software (SPSS Inc., Chicago, USA) was used for statistical analysis of the data. Because the samples were nonparametric, the median and range of performance parameters are presented. Means also are presented in some cases to provide a complete depiction of the data. Friedman’s analysis of variance (ANOVA) test and Wilcoxon signed ranks test (two-tailed significance) were used to assess potential learning effects and differences in performance within groups, whereas differences between groups were evaluated using the Mann–Whitney U test (one-tailed significance). All differences were considered statistically significant at a p value of 0.05 or less.

Results

Performance of the novices

The novices improved their performance considerably on both VR colonoscopy I-3 and the EndoBubble task (Table 1 and Fig. 3). In the fourth run, the median time required to finish the EndoBubble task was reduced by 58.5%, whereas the range decreased by 77%. The median time required to reach the cecum for VR colonoscopy I-3 was reduced by 72.3%, whereas the range decreased by 85%. The median percentage of time the patient was in excessive pain reached 0% in the fourth run for all the participants. Friedman’s ANOVA test showed that the performance of the novice participants differed significantly between the four runs (p < 0.05 for the number of wall collisions in the EndoBubble task and p < 0.01 for all the other performance parameters). The Wilcoxon signed ranks test showed that the time required to reach the cecum for VR colonoscopy I-3 differed significantly (p ≤ 0.001) for all four consecutive runs on the basis of positive ranks as follows: run 1 with run 2 (Z = −4.08), run 2 with run 3 (Z = −3.31), and run 3 with run 4 (Z = −3.60). The time required to finish the EndoBubble task differed significantly (p ≤ 0.001) between runs 1 and 2 (Z = −4.64) and between runs 2 and 3 (Z = −3.63) on the basis of positive ranks. The performance of the novices in runs 1 and 4 of the VR colonoscopy I-3 differed significantly for all simulator parameters.

Fig. 3.
figure 3

Boxplots for the performance of 30 novices, 20 experienced endoscopists (average of 600 colonoscopy procedures), and 40 expert endoscopists (experience with more than 1,000 colonoscopy procedures) showing time required to accomplish the EndoBubble level 1 task (a) and and the VR colonoscopy I-3 (b), as well as the (estimated) percentage of time the virtual patient was in excessive pain (c). Expertise level is represented by performed exercises or colonoscopy procedures along a logarithmic scale. The reference line represents the median for experienced and expert performances. °Outliers. *Extremes in the data (cases with values of 1.5 to 3 and more than 3 box lengths from the upper edge of the box, respectively).

Full size image

Performance of the experts

At first sight, the performance of the experts appears to differ over the four runs (Fig. 4). However, Friedman’s ANOVA test shows that they did not differ significantly except for the percentage of time the patient was in excessive pain. The Wilcoxon signed ranks test also showed no significant difference between their performances over the four consecutive runs, except for the percentage of time the patient was in excessive pain in VR colonoscopy I-3 between runs 1 and 4.

Fig. 4.
figure 4

Boxplots of the performance parameters for repetitive exercises (EndoBubble level 1 task and VR-colonoscopy I-3) of 5 expert participants showing the time required to accomplish the EndoBubble level 1 task (a) and VR-colonoscopy I-3 (b), as well as the (estimated) percentage of time the virtual patient was in excessive pain (c).

Full size image

Performance of the novices compared with that of the experienced and expert endoscopists

The performance of the novice participants in the first run and both the experienced and expert endoscopists differed significantly on several parameters (Table 2 and Fig. 3). For VR colonoscopy I-3, the performance of the novices in the first run differed significantly from those of experienced and expert endoscopists in the time required to reach the cecum, the occurrence of lost lumen view, and the occurrence of excessive local pressure. Additionally, the novices’ performance in the first run differed from that of the experienced endoscopists in time spent with a clear view. For the EndoBubble level 1 task, the novices required a significantly longer time to finish the task than either the experienced or the expert endoscopists, and caused a higher number of wall collisions than the expert endoscopists.

Table 2. Performances of 20 experienced and 40 expert endoscopists compared with the performances of 30 novice participants (Mann–Whitney U test)
Full size table

Discussion

Virtual reality simulators are becoming a popular tool for endoscopy skills training and could provide medical educators with a tool for objective proficiency assessment and an effective alternative for clinical training [9–11]. First, the use of simulators in skills lab–oriented training programs could reduce patient discomfort and increase patient safety [9–11, 13]. Second, it could reduce workload and costs involved when experts supervise endoscopy trainees [10–12], thus improving the efficacy of the learning process. However, the construct of VR simulators and their role within training programs still are being studied.

Our previous validation study showed that for basic endoscope navigation, the GI Mentor II can differentiate between several levels of expertise. The GI Mentor II also was considered a valuable addition to the training program for novice endoscopists [20]. Other studies also have established both the face and construct validity of the GI Mentor [8, 17–19]. To date, few studies have investigated the learning of lower GI endoscopy skills using the GI Mentor [7, 8, 19]. The designs of these earlier studies varied considerably in terms of the focus, tasks included, and sample size. The studies also varied considerably in training time span, type of training, and amount of training. Although this comparison is difficult, the results encountered in our study appear to be consistent with the findings of those studies [7, 8, 19].

It is important to establish validity for all aspects of the simulator and to assess the training potential of simulators for all available training modules. It is imperative, however, to start by assessing the simulator for basic skills, in this case, the ability to navigate through the colon to the cecum. The first attribute to which a trainee for flexible endoscopy must adapt is counterintuitive navigation. In our study, the participants were given the assignment to reach the cecum as quickly as possible for the VR colonoscopies with minimal patient discomfort. For the EndoBubble tasks, the assignment was to pop 20 balloons as quickly as possible while avoiding wall collisions. Assessment of procedure-related skills and abilities (e.g., identification of pathologies) was intentionally not included. Because the focus was on endoscope navigation, the end point for the VR colonoscopy exercises was reaching the cecum. For this reason, the parameters on the percentage of inspected mucosa and the accuracy and efficiency of screening were excluded.

The consistent and organized nature of the training sessions and the exercises within a set time span created a constructive environment for assessing proficiency improvement per subject, within expertise groups, and between them. It is important to minimize the influence of unfamiliarity with the simulator, or familiarity with specific cases, by using a variety of cases in each training session. After the final training session, when some of the novice participants were informed about the repetitive nature of VR colonoscopy I-3, they stated that they had been unaware of this.

Performance of the novices

All the participants improved their performance significantly over the course of four training sessions, particularly in terms of the key parameters assessed in this study, namely, the time required to accomplish the exercises and the percentage of time the patient was in excessive pain. The novices also improved their performance considerably in relation to other parameters associated with ease of navigation through the colon (e.g., the percentage of time with a clear view and the lost view of the lumen) and the pain level during the procedure (e.g., excessive local pressure and total loop time) (Table 1). This agrees with earlier studies on the learning of tasks related to lower endoscopy on the GI Mentor [7, 8, 19].

The performance improvement of the trainees indicates that the difficulties often experienced by novice endoscopists when navigating through the colon with a flexible endoscope can be reduced considerably by training on the GI Mentor II. The novices appeared capable of learning how to cope with these difficulties. As on occasions when progression of the endoscope image halted because of loop formation, for example, they learned the counterintuitive response of pulling back the endoscope shaft to progress further into the colon, most likely by trial and error. The considerable decrease in the pain-related parameters could mean that they also gained understanding of the factors and actions that cause pain or discomfort for the patient during flexible endoscopy. This contrasts with the results of studies by Mahmood and Darzi [22], and Datta et al. [24]. The greater number of exercises, the combination of different exercise types, or the influence of knowledge of results could contribute to this difference.

Performance of the experts

The possibility that trainees improve their performance by learning tricks that work well on the simulator but do not necessarily improve real-life colonoscopy performance should be taken into account in studies investigating the learning of tasks on VR systems. To verify that the GI Mentor II is not merely an expensive computer game, five expert endoscopists performed the same training sessions as well. The performances of the expert endoscopists show a relatively flat profile, which demonstrates that they are on the plateau of the proficiency curve according to the GI Mentor II performance parameters (Fig. 4). In addition, the performances of the experts over the four runs are not significantly different, except for the parameter on excessive pain. This indicates that the construct of the simulator provides a valid training tool for basic endoscope navigation in colonoscopy and supports validation studies based on the GI Mentor’s capability to distinguish expertise levels [7, 8, 17–20].

Performance of the novices compared with that of the experienced and expert endoscopists

The simulator is able to distinguish between the performance of novices and that of both experienced and expert endoscopists. However, our validation study showed that the differences between the performances of experienced and expert endoscopists on the GI Mentor II simulator are not significant [20].

In the current study, the difference between the performance of novices and that of experienced and expert endoscopists decreased considerably after training on the GI Mentor II, as the performance of the novices improved over the four runs. The values in Table 2 show a reduction in the difference for most of the performance parameters for both experienced and expert endoscopists, except for the percentage of time the patient was in excessive pain and the total time the colon was looped. For most parameters, particularly the time required to accomplish the repetitive exercises, the difference remained significant over all four runs. However, for the percentage of time the patient was in excessive pain and total time the colon was looped, Table 2 shows an increase in the difference. The novices appeared to perform increasingly better in these aspects over the four runs than the experienced and expert endoscopists (Tables 1 and 2).

The mean ranks of the Mann–Whitney U test were lower for the novices than for the experts and experienced endoscopists, except for the novices compared with the experienced endoscopists in the first run. The sum of ranks also was lower for the novices than for the experts. Hesitancy in progression of the endoscope combined with vigilance for causing excessive pain could play a role in the relatively low (estimated) percentage of time the patient was in excessive pain during the performance of VR colonoscopy I-3 by the novice participants.

The total time the colon was looped is a strong factor in the equation used to calculate the composite parameter (estimated) percentage of time the patient is in excessive pain. Although the procedure time decreased significantly by run four, the absolute amount of time the patient experienced excessive pain could still be increased compared with the pain levels during the performances of experienced and expert endoscopists.

The performance curves of the novices appeared not to have reached a plateau within 4 sessions, or 15 exercises. Although this study does provide insight into the first part of the proficiency curve for endoscope navigation by training on the GI Mentor II, it does not provide insight into the value of the GI Mentor II for training in complete colonoscopies, which also includes inspection of the mucosa or performance of therapeutic interventions. This justifies the need for further studies on the potential of the GI Mentor II for assessment of and training in flexible endoscopy, studies more longitudinal by nature and involving more complex tasks. The transfer of skills acquired on the simulator to the performance of real-life clinical colonoscopy should be studied as well.

Conclusions

This study confirms the GI Mentor II as a valid tool to be used in the training of basic flexible endoscopy navigation skills for colonoscopy. The large sample and the strong focus on basic skills set this study apart from earlier studies. The data provided are consistent with those of earlier studies on this topic, although one-to-one comparison is difficult because of differences in the study designs.

In addition, this study proves that combined training in both VR colonoscopies and the EndoBubble task on the simulator has a significant effect on the performance of novice endoscopists. The results provide additional insight and increase the knowledge about the proficiency curve for flexible endoscope navigation with training on the GI Mentor II.