eTable 1: Summary of technical tasks assessed, including description of their setup and performance, as well as the scoring systems used for assessment.

Technical Task / Description1 / Scoring system1
Peg transfer / Six plastics pegs are place on a board. The participant uses their non-dominant hand to pick up the peg with a laparoscopic grasper, then transfers it, in the air, to a grasper in the other hand, and then places it down on the board on the opposite side. This process is repeated until all 6 pegs have been transfer across the board and back again. / Time and accuracy is used to score this task.
Time – the number of seconds it takes to transfer all the pegs across the board and back again, to a max of 300 seconds.
Accuracy – a penalty score is subtracted for the number of pegs dropped outside of the field of view that cannot be retrieved.
Circle Cut / An ink stamp is used to mark a standardized circle onto the surface of a 2-ply, 4 by 4 inch gauze, which is then secured into the box trainer with large clip and two smaller side clips. The participant cuts out the circle using a laparoscopic scissors in one hand and a grasper in the other. / Time and accuracy is used to score this task.
Time- the number of seconds it takes to cut out the circle, to a max of 300 seconds.
Accuracy – penalty is given for the surface area outside the traced line.
Intracorporeal knot tie / An approximately 3cm long piece of Penrose drain is prepared with a 1cm long longitudinal slit, flanked by a black dot on either side. The drain is secured inside the box trainer with hook-and-loop fastener to a board. The participant uses two needle drivers and a 15cm-long suture to tie one intracorporeal knot by driving the needle through the two black dots and securing the suture with three throws. First a double throw, followed by two single throws, then cutting both sutures. / Time and accuracy is used to score this task.
Time - the number of seconds it takes to tie the knot.
Accuracy – penalties given for:
  1. Deviating more than 5mm from the black dots
  2. An air knot or slip knot

One handed tie / The one handed ties were performed using a silk tie suture passed through a Penrose drain fastened with Velcro to an underlying board.
The participants were askedto tie as many square knots as possible and to direct the knot down with their index finger after each throw. / AmOSATS scale was used to score each repetition, excluding the ‘use of assistance’ and ‘knowledge of instruments’ categories found in the original OSATS2 assessment tool, as they were not applicable to this task.
The total mOSATS score for this task was 25 points.
Laparotomy closure / This task was simulated by fastening a felt sheet with a central 15cm long incision to an underlying hollow frame. A 0 monofilament suture was used to close the incision.
This model was modified from the ACS/APDS surgical skills curriculum.3 The students were given 300 seconds to close the mind line incision using a running suture and hand ties where needed. / Each repetition was also scored using a mOSATS, which excluded the ‘use of assistance’ category included in the original OSATS2 assessment tool, because the students completed this task by themselves.
The total mOSATS score for this task was 30 points.

ACS: American College of Surgeons; APDS: Association of Program Directors in Surgery; mOSATS: modified objective structured assessment of technical skills; OSATS: objective structured assessment of technical skills

References

  1. FLS Manual Skills Written Instructions and Performance Guidelines. 2014; September 1, 2015.
  2. 1.Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. British Journal of Surgery 1997; 84:273-278.
  3. Cofer JB, Lee K, Pritts T, et al. ACS/APDS Surgical Skills Curriculum for Residents. Module 12 2015. Available at: Accessed October 12, 2015, 2015

Supplement 1: Methodology used to calculate proficiency scores for open tasks.

Two expert surgeons from the study institution performed each basic and advanced task five consecutive times. Each repetition was evaluated using the associated mOSATS instrument by a single blinded rater. The completion times for all 10 repetitions were used to calculate the mean and standard deviation for each task, with two standard deviations below (slower than) the mean set as the proficiency score for each task (eTable 2). Per the scoring methodology, higher scores represented better performance for all tasks.

eTable 2: Proficiency scores for open tasks
One handed tie
mOSATS / Laparotomy closure
mOSATS
Repetition / Expert 1 / Expert 2 / Expert 1 / Expert 2
1 / 22 / 21 / 30 / 25
2 / 23 / 24 / 26 / 24
3 / 25 / 23 / 25 / 29
4 / 24 / 22 / 29 / 27
5 / 21 / 25 / 26 / 24
Mean / 23 / 23 / 27.2 / 25.8
SD / 1.6 / 1.6 / 2.2 / 2.2
Group mean / 23 / 26.5
Group SD / 1.5 / 2.2
Proficiency score / 20.0 / 22.2
SD: standard deviation; mOSATS:modified objective structure of technical skills; Proficiency score = Group mean - 2(SD)