Surgical Skills Transfer After Simulation-Based Training
Surgical Skills Transfer After Simulation-Based Training
Seventeen RCTs and 3 non-RCTs investigating skills transfer after simulation-based training for laparoscopic cholecystectomy and endoscopy were included in this review. Ten studies investigated laparoscopic cholecystectomy procedures (Table 1), and 10 studies investigated endoscopic procedures (Table 2). Participants included general surgery and colonoscopy trainees.
Laparoscopic Cholecystectomy. The review published in 2008 included 4 RCTs and 1 non-RCT comparative study, whereas the 2011 search revealed another 4 RCTs and 1 non-RCT comparative study, all of which compared performance during operative laparoscopic cholecystectomy between participants who had undergone simulation-based training and those who had not received simulation-based training.
The 10 laparoscopic cholecystectomy studies included 7 different modes of simulation (high-fidelity Lap SIM, LAP Mentor, MIST-VR, and Xitact LS500, and low-fidelity FLS/MISTELS and video trainers).
Endoscopy. The review published in 2008 included 6 RCTs for colonoscopy and sigmoidoscopy, whereas the 2011 search revealed another 3 RCTs for colonoscopy and esophagogastroduodenoscopy and 1 non-RCT for colonoscopy. In 7 RCTs and 1 non-RCT study, endoscopy (ie, colonoscopy, sigmoidoscopy, or esophagogastroduodenoscopy) performance was compared between participants who trained using simulation and those who had no simulator training. In 2 RCTs, patient-based performance was compared between participants who trained using simulation only and those who had undergone only patient-based training.
The 10 endoscopy studies included 6 different modes of virtual reality simulation [AccuTouch, GI Mentor, Gastro-Sim, KAIST-Ewha Colonoscopy Simulator II, Olympus (ENDO TS-1) colonoscopy simulator, and sigmoidoscopy simulator] with a variety of training durations.
Ten of the 20 studies reported an overall performance score that was defined as a global summary of all objective performance parameters measured during patient-based assessment procedures or the assessor's evaluation of the participant's overall performance or ability to complete the assessment procedure.
Five of the 10 laparoscopic cholecystectomy studies reported an overall performance score (Table 1). Another RCT used a validated global rating scale or 5-point competency scale for individual tasks but did not report the sum of the overall performance score. Four studies did not report an objective evaluation of overall performance.
Cosman et al reported that a simulator-trained group had significantly higher global scores (median 3.2 points vs 1.8; P = 0.04) than a control group (with only patient-based training). In 2 different studies, Scott et al reported that a simulator-trained group had significantly higher overall performance score than a control group (P = 0.007 and P = 0.05, respectively). Similarly, Sroka et al reported that a simulator-trained group had significantly higher GOALS scores (sum of 5 domains) than a control group (17.4 ± 1.9 points vs 13.8 ± 2.2; P = 0.0003). Conversely, Hogle et al reported no significant difference in the objective assessment of technical skills in the operating room between simulator-trained participants and a control group in any of the 5 GOALS domains (P = 0.99, 0.55, 0.93, 0.56, 0.85, respectively). Beyer et al in a comparison of 2 different simulators reported that both simulator-trained groups scored significantly higher in the GOALS assessment than the control group that had only patient-based training (MISTELS-FLS, P = 0.03; LAP Mentor, P = 0.007, respectively).
Endoscopy (4 Studies). Three RCTS from the 10 endoscopy studies (Table 2) reported an overall performance parameter. Cohen et al found that a simulator-trained group had significantly higher performance scores in sessions 2 (P = 0.002) and 3 (P = 0.005) of the 10-session assessment than a control group. Similarly, Park et al reported that a simulator-trained group had scored significantly higher in an objective global rating score than a control group (17.9 ± 5.2 points vs 14.8 ± 2.5; P = 0.04). Haycock et al compared the overall performance of participants who undertook simulator-based training only with those who undertook only patient-based training finding no significant difference (P = 0.35), suggesting that skills learnt on a simulator can be transferred to real colonoscopy.
Shirai et al did not report an overall score but evaluated 11 performance-related items and found that the scores were significantly higher in a simulator-trained group than in a control group for 5 items, that is, insertion of the endoscope into the esophagus (P < 0.05), passing esophagogastric junction into the antrum (P < 0.01), passing through the pyloric ring (P < 0.05), examination of the duodenal bulb (P < 0.05), and viewing the fornix (P < 0.05), but were not significant difference for the other 6 items.
As listed below under the section "Performance Errors," Yi et al reported a significant difference in overall performance accuracy (P < 0.001) in favor of the simulator-trained participants.
Nine of the 20 studies reported performance time, defined as the time taken to conduct a patient-based assessment procedure.
Laparoscopic Cholecystectomy (4 Studies). Four RCTs found time improvements in laparoscopic cholecystectomy (Table 1) for simulator-trained participants compared with participants who did not have this training, but this difference only reached significance in 1 study (P = 0.021).
Endoscopy (6 Studies). Two RCTs, Ahlberg et al and Tuggy (Table 2), found that a simulator-trained group completed procedures for endoscopy in a significantly shorter time than a control group that did not have this training (P = 0.008;P = 0.01). Conversely, 2 RCTs, Sedlack et al and Sedlack and Kolars reported no significant difference in procedure time for endoscopy after simulator-based training compared with no simulator training. Yi et al using 2 different training scenarios reported that a simulator-trained group completed the endoscopy insertion procedure in significantly shorter time than a control group with no simulator training (P = 0.028).
Haycock et al compared performances of participants who undertook simulator-based training only with those who undertook only patient-based training finding no significant difference (P = 0.11), suggesting that skills learnt on a simulator can be transferred to real colonoscopy.
Success rate was defined as the percentage of participants who were able to complete the patient-based assessment as specified, the number of participants who were able to complete the case independently without assistance from the supervising surgeon, or the number of participants who were given a pass grade. "Autonomy" was included in GOALS scores but the result was reported separately by only 2 studies.
Laparoscopic Cholecystectomy (5 Studies). For laparoscopic cholecystectomy (Table 1), Seymour et al reported that all participants were able to successfully complete the assessment procedure. However, in this study the supervising surgeon had to take over for 6 of 8 trainees who had not received simulation-based training compared with 0 of 8 trainees who had received simulation-based training (P value not reported). Scott et al in 2 different RCTs also found that simulation-based training before patient-based assessment resulted in a significantly greater ability to complete the assessment procedure (recorded as overall performance) (P = 0.007 and P < 0.05). Conversely, Hogle et al and Sroka et al found no significant difference between a simulation-trained group and a control group in "autonomy" as part of GOALS score.
Endoscopy (7 Studies). For endoscopy (Table 2), Ahlberg et al, Sedlack et al, Sedlack and Kolars, and Yi et al found that significantly more simulator-trained participants completed the procedure independently than those with no simulator training (P = 0.0011;P = 0.027 and P = 0.007 for the first 2 blocks of colonoscopies;P = 0.0017; and P = 0.006). Park et al reported that only 1 out of 12 participants from the simulator-trained group reached the cecum whereas none of the 12 control group participants reached the cecum. Gerson and Van Dam and Haycock et al compared a simulation-trained only group with patient-trained only group with different findings. Although Gerson and Van Dam reported that significantly fewer patient-trained only participants required assistance to perform sigmoidoscopy than simulator-trained only participants (P = 0.001), Haycock et al reported no significant difference between the 2 groups for colonoscopy (P = 0.51).
Performance errors were defined as movements or events outside the normal procedure. For laparoscopic cholecystectomy (Table 1), 4 RCTs found that simulator-trained participants made significantly fewer intraoperative errors than those not trained on a simulator (P = 0.037;P = 0.05;P = 0.003; and P < 0.006). Similarly for endoscopy (Table 2), Tuggy reported significant reductions in the number of errors during sigmoidoscopy for simulator-trained participants compared with control participants who did not have this training (P = 0.01 after 5 hours of training and P < 0.01 after 10 hours of training).
Patient discomfort (Table 2) was defined as the pain felt by the patient or standardized patient (volunteer) undergoing the endoscopic assessment procedure, or how well a participant responded to patient discomfort as determined by the assessor. Of the 5 endoscopy studies, 3 studies (Ahlberg et al, Sedlack et al, and Sedlack and Kolars) reported that participants who had received simulator-based training caused patients significantly less discomfort (as determined by the patient) than participants who had not received this training (P = 0.02;P < 0.01; and P = 0.19). Conversely, Tuggy found no differences between the 2 groups. Gerson and Van Dam compared patient-trained only participants with simulator-trained only participants and found that there were no differences in patient discomfort between the 2 groups. In the non-RCT study, Yi et al found that the patient discomfort score for abdominal pain was not significantly different between groups (P = 0.273), but an anal discomfort score was significantly lower in a simulator-trained group (P = 0.002). When determined by the assessor, Sedlack and Kolars found that the simulator-trained participants responded to pain significantly better than participants who had not received this training (P = 0.019 for the first block of colonoscopies). Ahlberg et al and Sedlack et al did not find any differences between the 2 groups.
Manual dexterity was defined as involving 2 hands. For laparoscopic cholecystectomy (Table 1), 2 RCTs, Cosman et al and Sroka et al reported that the simulator-trained group had better bimanual dexterity/coordination than those with no simulator training (P = 0.05;P = 0.04) whereas another RCT, Hogle et al, reported no significant difference between the groups (P = 0.55).
Surgical confidence and/or flow of procedure referred to the ability of a participant to move between steps during a procedure in a continuous and fluent manner, exhibiting confidence with each step. Time and motion assessments measured a participant's ability to conduct the patient-based assessment procedure with efficiency and economy of movement. "Flow of operation" was included in some global rating scores but not reported separately. Schijven et al reported a significant improvement in fluency during laparoscopic cholecystectomy for participants who had undertaken a 4-day training course compared with those who had not (P = 0.0037). Scott et al and Scott et al in 2 different studies found no significant differences between participants who had trained on a simulator and those who had not. In addition, Grantcharov et al and Scott et al reported significantly better time and motion outcomes for simulator-trained participants than controls (P = 0.003;P < 0.15). However, Scott et al did not find any differences between the 2 groups.
Two studies measured the number of endoscopy procedures that experienced staff were able to perform per half-day session when in the presence or absence of trainees in the operative setting. Sedlack and Kolars reported than when trainees were attending 2 half days of simulator-based training, faculty staff members were able to complete an average of 8 colonoscopies (range, 7–9) per half day whereas only an average of 3.5 procedures (range, 2-4) per half day were performed when trainees were present. In another study, Sedlack et al also reported an increase in staff productivity as a result of simulator-based training. During the simulator-based training interval, experienced staff performed an average of 7 procedures per half day compared with an average of less than 3 procedures when in the presence of trainees. Therefore, the 3-hour simulator-based training session allowed an average of 4 additional procedures to be performed by experienced staff.
Results
Included Studies (20 Studies)
Seventeen RCTs and 3 non-RCTs investigating skills transfer after simulation-based training for laparoscopic cholecystectomy and endoscopy were included in this review. Ten studies investigated laparoscopic cholecystectomy procedures (Table 1), and 10 studies investigated endoscopic procedures (Table 2). Participants included general surgery and colonoscopy trainees.
Laparoscopic Cholecystectomy. The review published in 2008 included 4 RCTs and 1 non-RCT comparative study, whereas the 2011 search revealed another 4 RCTs and 1 non-RCT comparative study, all of which compared performance during operative laparoscopic cholecystectomy between participants who had undergone simulation-based training and those who had not received simulation-based training.
The 10 laparoscopic cholecystectomy studies included 7 different modes of simulation (high-fidelity Lap SIM, LAP Mentor, MIST-VR, and Xitact LS500, and low-fidelity FLS/MISTELS and video trainers).
Endoscopy. The review published in 2008 included 6 RCTs for colonoscopy and sigmoidoscopy, whereas the 2011 search revealed another 3 RCTs for colonoscopy and esophagogastroduodenoscopy and 1 non-RCT for colonoscopy. In 7 RCTs and 1 non-RCT study, endoscopy (ie, colonoscopy, sigmoidoscopy, or esophagogastroduodenoscopy) performance was compared between participants who trained using simulation and those who had no simulator training. In 2 RCTs, patient-based performance was compared between participants who trained using simulation only and those who had undergone only patient-based training.
The 10 endoscopy studies included 6 different modes of virtual reality simulation [AccuTouch, GI Mentor, Gastro-Sim, KAIST-Ewha Colonoscopy Simulator II, Olympus (ENDO TS-1) colonoscopy simulator, and sigmoidoscopy simulator] with a variety of training durations.
Overall Performance (10 Studies)
Ten of the 20 studies reported an overall performance score that was defined as a global summary of all objective performance parameters measured during patient-based assessment procedures or the assessor's evaluation of the participant's overall performance or ability to complete the assessment procedure.
Laparoscopic Cholecystectomy (6 Studies)
Five of the 10 laparoscopic cholecystectomy studies reported an overall performance score (Table 1). Another RCT used a validated global rating scale or 5-point competency scale for individual tasks but did not report the sum of the overall performance score. Four studies did not report an objective evaluation of overall performance.
Cosman et al reported that a simulator-trained group had significantly higher global scores (median 3.2 points vs 1.8; P = 0.04) than a control group (with only patient-based training). In 2 different studies, Scott et al reported that a simulator-trained group had significantly higher overall performance score than a control group (P = 0.007 and P = 0.05, respectively). Similarly, Sroka et al reported that a simulator-trained group had significantly higher GOALS scores (sum of 5 domains) than a control group (17.4 ± 1.9 points vs 13.8 ± 2.2; P = 0.0003). Conversely, Hogle et al reported no significant difference in the objective assessment of technical skills in the operating room between simulator-trained participants and a control group in any of the 5 GOALS domains (P = 0.99, 0.55, 0.93, 0.56, 0.85, respectively). Beyer et al in a comparison of 2 different simulators reported that both simulator-trained groups scored significantly higher in the GOALS assessment than the control group that had only patient-based training (MISTELS-FLS, P = 0.03; LAP Mentor, P = 0.007, respectively).
Endoscopy (4 Studies). Three RCTS from the 10 endoscopy studies (Table 2) reported an overall performance parameter. Cohen et al found that a simulator-trained group had significantly higher performance scores in sessions 2 (P = 0.002) and 3 (P = 0.005) of the 10-session assessment than a control group. Similarly, Park et al reported that a simulator-trained group had scored significantly higher in an objective global rating score than a control group (17.9 ± 5.2 points vs 14.8 ± 2.5; P = 0.04). Haycock et al compared the overall performance of participants who undertook simulator-based training only with those who undertook only patient-based training finding no significant difference (P = 0.35), suggesting that skills learnt on a simulator can be transferred to real colonoscopy.
Shirai et al did not report an overall score but evaluated 11 performance-related items and found that the scores were significantly higher in a simulator-trained group than in a control group for 5 items, that is, insertion of the endoscope into the esophagus (P < 0.05), passing esophagogastric junction into the antrum (P < 0.01), passing through the pyloric ring (P < 0.05), examination of the duodenal bulb (P < 0.05), and viewing the fornix (P < 0.05), but were not significant difference for the other 6 items.
As listed below under the section "Performance Errors," Yi et al reported a significant difference in overall performance accuracy (P < 0.001) in favor of the simulator-trained participants.
Performance Time (10 Studies)
Nine of the 20 studies reported performance time, defined as the time taken to conduct a patient-based assessment procedure.
Laparoscopic Cholecystectomy (4 Studies). Four RCTs found time improvements in laparoscopic cholecystectomy (Table 1) for simulator-trained participants compared with participants who did not have this training, but this difference only reached significance in 1 study (P = 0.021).
Endoscopy (6 Studies). Two RCTs, Ahlberg et al and Tuggy (Table 2), found that a simulator-trained group completed procedures for endoscopy in a significantly shorter time than a control group that did not have this training (P = 0.008;P = 0.01). Conversely, 2 RCTs, Sedlack et al and Sedlack and Kolars reported no significant difference in procedure time for endoscopy after simulator-based training compared with no simulator training. Yi et al using 2 different training scenarios reported that a simulator-trained group completed the endoscopy insertion procedure in significantly shorter time than a control group with no simulator training (P = 0.028).
Haycock et al compared performances of participants who undertook simulator-based training only with those who undertook only patient-based training finding no significant difference (P = 0.11), suggesting that skills learnt on a simulator can be transferred to real colonoscopy.
Success Rate (12 Studies)
Success rate was defined as the percentage of participants who were able to complete the patient-based assessment as specified, the number of participants who were able to complete the case independently without assistance from the supervising surgeon, or the number of participants who were given a pass grade. "Autonomy" was included in GOALS scores but the result was reported separately by only 2 studies.
Laparoscopic Cholecystectomy (5 Studies). For laparoscopic cholecystectomy (Table 1), Seymour et al reported that all participants were able to successfully complete the assessment procedure. However, in this study the supervising surgeon had to take over for 6 of 8 trainees who had not received simulation-based training compared with 0 of 8 trainees who had received simulation-based training (P value not reported). Scott et al in 2 different RCTs also found that simulation-based training before patient-based assessment resulted in a significantly greater ability to complete the assessment procedure (recorded as overall performance) (P = 0.007 and P < 0.05). Conversely, Hogle et al and Sroka et al found no significant difference between a simulation-trained group and a control group in "autonomy" as part of GOALS score.
Endoscopy (7 Studies). For endoscopy (Table 2), Ahlberg et al, Sedlack et al, Sedlack and Kolars, and Yi et al found that significantly more simulator-trained participants completed the procedure independently than those with no simulator training (P = 0.0011;P = 0.027 and P = 0.007 for the first 2 blocks of colonoscopies;P = 0.0017; and P = 0.006). Park et al reported that only 1 out of 12 participants from the simulator-trained group reached the cecum whereas none of the 12 control group participants reached the cecum. Gerson and Van Dam and Haycock et al compared a simulation-trained only group with patient-trained only group with different findings. Although Gerson and Van Dam reported that significantly fewer patient-trained only participants required assistance to perform sigmoidoscopy than simulator-trained only participants (P = 0.001), Haycock et al reported no significant difference between the 2 groups for colonoscopy (P = 0.51).
Performance Errors (5 Studies)
Performance errors were defined as movements or events outside the normal procedure. For laparoscopic cholecystectomy (Table 1), 4 RCTs found that simulator-trained participants made significantly fewer intraoperative errors than those not trained on a simulator (P = 0.037;P = 0.05;P = 0.003; and P < 0.006). Similarly for endoscopy (Table 2), Tuggy reported significant reductions in the number of errors during sigmoidoscopy for simulator-trained participants compared with control participants who did not have this training (P = 0.01 after 5 hours of training and P < 0.01 after 10 hours of training).
Patient Discomfort (7 Studies)
Patient discomfort (Table 2) was defined as the pain felt by the patient or standardized patient (volunteer) undergoing the endoscopic assessment procedure, or how well a participant responded to patient discomfort as determined by the assessor. Of the 5 endoscopy studies, 3 studies (Ahlberg et al, Sedlack et al, and Sedlack and Kolars) reported that participants who had received simulator-based training caused patients significantly less discomfort (as determined by the patient) than participants who had not received this training (P = 0.02;P < 0.01; and P = 0.19). Conversely, Tuggy found no differences between the 2 groups. Gerson and Van Dam compared patient-trained only participants with simulator-trained only participants and found that there were no differences in patient discomfort between the 2 groups. In the non-RCT study, Yi et al found that the patient discomfort score for abdominal pain was not significantly different between groups (P = 0.273), but an anal discomfort score was significantly lower in a simulator-trained group (P = 0.002). When determined by the assessor, Sedlack and Kolars found that the simulator-trained participants responded to pain significantly better than participants who had not received this training (P = 0.019 for the first block of colonoscopies). Ahlberg et al and Sedlack et al did not find any differences between the 2 groups.
Bimanual Dexterity (3 Studies)
Manual dexterity was defined as involving 2 hands. For laparoscopic cholecystectomy (Table 1), 2 RCTs, Cosman et al and Sroka et al reported that the simulator-trained group had better bimanual dexterity/coordination than those with no simulator training (P = 0.05;P = 0.04) whereas another RCT, Hogle et al, reported no significant difference between the groups (P = 0.55).
Surgical Confidence (5 Studies)
Surgical confidence and/or flow of procedure referred to the ability of a participant to move between steps during a procedure in a continuous and fluent manner, exhibiting confidence with each step. Time and motion assessments measured a participant's ability to conduct the patient-based assessment procedure with efficiency and economy of movement. "Flow of operation" was included in some global rating scores but not reported separately. Schijven et al reported a significant improvement in fluency during laparoscopic cholecystectomy for participants who had undertaken a 4-day training course compared with those who had not (P = 0.0037). Scott et al and Scott et al in 2 different studies found no significant differences between participants who had trained on a simulator and those who had not. In addition, Grantcharov et al and Scott et al reported significantly better time and motion outcomes for simulator-trained participants than controls (P = 0.003;P < 0.15). However, Scott et al did not find any differences between the 2 groups.
Staff Productivity (2 Studies)
Two studies measured the number of endoscopy procedures that experienced staff were able to perform per half-day session when in the presence or absence of trainees in the operative setting. Sedlack and Kolars reported than when trainees were attending 2 half days of simulator-based training, faculty staff members were able to complete an average of 8 colonoscopies (range, 7–9) per half day whereas only an average of 3.5 procedures (range, 2-4) per half day were performed when trainees were present. In another study, Sedlack et al also reported an increase in staff productivity as a result of simulator-based training. During the simulator-based training interval, experienced staff performed an average of 7 procedures per half day compared with an average of less than 3 procedures when in the presence of trainees. Therefore, the 3-hour simulator-based training session allowed an average of 4 additional procedures to be performed by experienced staff.
