Graduate education
Development of an objective structured assessment of technical skills for obstetric and gynecology residents

https://doi.org/10.1016/S0029-7844(00)00829-2Get rights and content

Abstract

Objective: To develop an objective structured assessment for evaluating surgical skills of obstetrics and gynecology residents and to evaluate the reliability and validity of the assessment.

Methods: A seven-station, objective, structured assessment of technical skills was administered to 24 residents. The test included laparoscopic procedures (port placement, salpingostomy, suturing, vessel ligation) and open abdominal procedures (hypogastric ligation, repair of enterotomy, salpingo-oophorectomy.) All surgical tasks were done on pigs. Residents were timed and assessed at each station using three methods of scoring, a task-specific checklist, global rating scale, and pass-fail grade.

Results: Assessment of construct validity (the ability of the test to discriminate among residency levels) found significant differences on the checklist and the global rating scale by residency level. Reliability indices calculated with Cronbach’s α were 0.89 for the global rating scale and 0.89–0.95 for the individual skills checklists. Interrater reliability was 0.87 for the global rating scale and 0.78–0.98 for the checklists.

Conclusion: Objective, structured assessment of technical skills can assess residents’ surgical skills with high reliability and validity. These assessments have possible application for identifying residents who need additional training and might provide a mechanism to ensure competence of surgical skills.

Section snippets

Materials and methods

All 24 obstetrics and gynecology residents from the University of Washington participated in the assessment. There were six residents from each postgraduate year. Examinations were scheduled from 4:00 pm to 8:00 pm; six residents were tested in each session, which allowed them to be excused from clinical duties and have adequate coverage for clinical services.

We developed a seven-station examination of technical skills that included laparoscopic (port placement, intracorporeal knot,

Results

Mean global examination scores for the procedures are shown in Table 1. Scores were significantly lower for hypogastric ligation and intracorporeal knot tying. Scores were significantly higher for salpingo-oophorectomy. The mean checklist, global, and pass-fail scores for each residency level are shown in Table 1, Table 2, Table 3.

The one-way analysis of variance with Student-Newman-Keuls post hoc test was used to evaluate construct validity, the ability of the test to distinguish among

Discussion

In a recent survey of obstetrics and gynecology residency directors in the United States,11 we found that most programs rely on subjective faculty assessment of residents’ technical skills. Only 17% used testing or a standardized assessment, and 25% did not assess technical skills of residents. Observation of surgical skills without grading criteria has poor reliability. Without structured criteria, two surgeons rating the same resident typically will have a low level of agreement.1

At the

References (12)

There are more references available in the full text version of this article.

Cited by (118)

  • Training and qualification in gynecological minimal access surgery: A systematic review

    2019, Best Practice and Research: Clinical Obstetrics and Gynaecology
  • Implementation of laparoscopy surgery training via simulation in a low-income country

    2018, Journal of Gynecology Obstetrics and Human Reproduction
View all citing articles on Scopus

Supported in part by a grant from the National Board of Medical Examiners (NBME) Medical Education Research Fund Grant. The project does not necessarily reflect NBME policy and NBME support provides no official endorsement. Supported in part by a grant from United States Surgical Corporation, Norwalk, Connecticut.

View full text