Background Veterinary medical education is increasingly moving towards outcome-based training based on competency frameworks. A source of concern is the translation of competencies into the practice of clinical teaching, for example, surgical skills training. It is suggested that the use of entrustable professional activities (EPAs) might bridge this gap. The purpose of this study, therefore, was to identify EPAs related to surgical skills for companion animal health to enhance competency-based education.
Methods Draft versions of EPAs related to surgical skills were established by an iterative consensus-based approach through 45-min interview sessions. These draft versions were used to explore the opinion of companion animal veterinarians, both veterinarians (specialists, residents and interns) involved in undergraduate teaching and veterinarians working in private practice involved in extramural clinical teaching, on the relevance and level of entrustment of the EPAs through a modified Delphi procedure. Mean (relevance) and median (level of entrustment) scores were calculated and textual comments were analysed to create a final framework of EPAs related to surgical skills.
Results and conclusion The Delphi panel reached consensus in three rounds. Thirty-four per cent of those invited to participate in the study completed the final survey. Finally, a list of 13 EPAs related to companion animal surgical skills a student should be entrusted to perform at time of graduation was established.
- entrustable professional activity
- companion animal health
Statistics from Altmetric.com
Veterinary medical education is increasingly moving towards outcome-based training for reasons of patient safety and high-quality healthcare.1 2 Many programmes advocate identifying and assessing competencies as tools for defining these outcomes. These competencies are defined in integrative competency frameworks to guide educational innovation1–4 and stand for the general qualities that every trainee (student) should acquire at the time of graduation.5 6 Although not new in medical education, a source of concern is the translation of these competency frameworks into the practice of clinical teaching. Competencies might be too abstract and therefore difficult to assess.1 3 4 6–8 It is suggested that the use of entrustable professional activities (EPAs) might bridge this gap.5
An EPA can be defined as a unit of professional practice consisting of multiple integrated competencies that can be entrusted as a task to a trainee (student) with sufficient competence.6 This entrustment in a trainee is primarily evaluated to determine how much supervision the trainee needs for a specific EPA. As soon as the trainee has demonstrated the necessary competence to execute these activities, they will be entrusted unsupervised. EPAs can vary in terms of size and complexity and should be specific, observable, measurable, having a designated time frame and be suitable for an entrustment decision.5 6 Competencies and EPAs should be seen as a two-dimensional matrix that can provide specifications for longitudinal assessment and feedback, for individual development and to ground entrustment decisions.5 6 9 In addition, it can serve to move towards a flexible length of training whereby the outcome of training becomes more important than its time.5 EPAs could increase transparency for supervisors to trust a trainee to fulfil a task and allow students to experience increasing levels of independence and responsibility.5 10
The purpose of this prospective study was to create a list of EPAs for relevant surgical skills in competency-based companion animal health rotations. The reason to focus on EPAs for surgical skills training was due to an increasing demand of society, educators and employers for competent veterinary professionals and methods to assess these skills.2 4 11–14 The aim of this study was to identify EPAs related to surgical skills in companion animal health that veterinary graduates should have acquired when entering clinical practice. For this a modified Delphi procedure was used.
Materials and methods
At the Faculty of Veterinary Medicine, Utrecht University (FVMU), the Netherlands, a curriculum of six years is offered consisting of a three-year preclinical programme followed by a three-year clinical programme. This three-year clinical programme comprises a number of one-week to seven-week clinical rotations in disciplines related to three tracks: equine health, companion animal health and farm animal health. Students select one of these tracks and gain experience in a variety of learning activities. To develop their surgical skills, students within the companion animal health track follow in their first and second year a six-week rotation concerning surgical skills. In their third year they have an eight-week extramural rotation containing surgical procedures.
Draft version EPAs
For this study, the programme outcomes at FVMU15 were used to formulate core EPAs restricted to companion animal surgical skills. The current programme outcomes consist of a list of 182 skills including 78 skills related to surgery (see online supplementary file 1). These 78 skills related to surgery were incorporated into 37 EPA draft versions (box 1). These 37 EPAs were established by an iterative consensus-based approach through in total three 45-min discussion sessions by a group of five veterinarians (three Diplomates of the European College of Veterinary Surgeons being active surgery instructors in the teaching programme at the FVMU and two veterinarians involved in educational research (RF, HB)), all appointed at the Department of Clinical Sciences of Companion Animals of FVMU.
Entrustable professional activities (EPAs) (n=37) at the start of the modified Delphi procedure
Preparing the patient for surgery.
Gloving and gowning (sterile technique).
Basic knot tying and suturing.
Performing various injection techniques.
Taking various punctures and biopsies of approachable masses in the skin.
Collaborating as a member of an interprofessional team.
Adopting the correct posture during surgery (ergonomics).
Recognising a patient requiring urgent or emergent care and initiate evaluation and management.
Managing and treating a traumatic wound.
Surgical drains: care and removal.
Placing a nose, oesophagus, stomach or proventricular feeding tube.
Performing a tracheotomy/air pocket trepanation.
Executing a nostril correction and staphylectomy.
Removing a foreign body in the ear canal/nose.
Treating an othaematoma.
Amputation of an extremity.
Performing conservative and simple surgical fracture treatment: applying bandage/splint.
Treating an acute eye problem.
Reconstructing an entropion/ectropion.
Removing a simple mass in or under the skin or on the eyelid margin.
Removing a lymph node.
Performing a cystotomy.
Dental care/dental cleaning and extraction of teeth.
Treating dislocation of the jaw/symphysis separation.
Correcting a beak.
Castration of a healthy patient and treating possible complications.
Performing a caesarean section.
Treating a vaginal/rectal prolapse.
Removing the anal glands.
Reconstruction of an umbilical hernia.
Assisting with an arthrotomy of the knee joint.
Performing a mastectomy.
Performing a splenectomy.
Performing an enterotomy/enterectomy.
Performing a cardiopulmonary resuscitation.
Performing a euthanasia.
Modified Delphi procedure
To explore opinions on the draft versions of the EPAs related to surgical skills, a modified Delphi procedure16 17 was performed between January 2016 and April 2016. The Delphi procedure, an iterative survey among participants until consensus is reached, has been approved as a useful procedure for establishing a list of EPAs.18 The Delphi procedure was conducted through an online electronic survey tool (SurveyMonkey; nl.surveymonkey.com) to validate the draft framework of the 37 EPAs among 132 companion animal veterinarians (66 veterinarians (specialists, residents and interns) involved in undergraduate teaching working at the FVMU and 66 veterinarians (generalists) working in private practice involved in extramural clinical teaching).
At the start of each survey it was explained to participants what the concept of EPAs is and why the survey was performed. The first round of the procedure was also considered as pilot for the survey questions. After the first round, only minor adjustments were necessary. Participants were invited to judge the relevance of the EPAs at time of graduation19 on a 5-point Likert scale (1=not relevant, 5=very relevant). Participants were also requested to give their opinion about the level of entrustment per EPA at time of graduation, designated by five levels of supervision: (1) has knowledge, but no permission to act; (2) permission to act with direct, proactive supervision present in the room; (3) permission to act with indirect supervision, not present but quickly available if needed; (4) permission to act under distant supervision not directly available; or (5) permission to provide supervision to junior students.5 6 In addition, participants were given the opportunity to add narrative feedback for the formulated EPAs and to indicate which EPAs were missing by the end of a round but were considered important.
After each Delphi round the mean relevance scores and median levels of entrustment were calculated and textual comments were analysed. The EPAs that were rated as relevant or very relevant (4–5) by at least 80 per cent of the participants were included in the framework. This threshold of 80 per cent is based upon experiences described in previously published scientific reports describing the application of the Delphi procedure.1 20–22 If the relevance was rated under 80 per cent, the EPA was being rejected or adjusted based upon the narrative feedback and rated again in the following Delphi round. The EPA was rejected if the textual feedback indicated that the EPA was not relevant to be performed at any kind of level of entrustment by a recently graduated veterinarian or was not considered to be an EPA related to surgical skills. In addition to the relevance of the EPAs, the level of entrustment was also considered. When consensus on the relevance of particular EPAs to be performed at time of graduation was below 80 per cent, but the median level of entrustment was greater than 3 (meaning the student has to perform the activity independently at time of graduation as defined by the five levels of supervision), it was decided to include these EPAs in the final list of EPAs related to surgical skills. In addition, newly formed EPAs, or smaller EPAs within an EPA (‘nested’ EPAs), suggested by the participants were assessed in the next round.
In total, three rounds were necessary to come to a final list of EPAs related to surgical skills a veterinary student should have developed at time of graduation. After two rounds consensus was reached; the third round was used to present to the participants the final established list. Demographic characteristics of participants of all three rounds are summarised in table 1. After round 1, also used as pilot, there was no reason to adapt the procedure.
Delphi round 1
Consensus in this first Delphi round was reached for 12 of the 37 EPAs. Consensus of 100 per cent was reached for one EPA, six exceeded 90 per cent, and five exceeded 80 per cent, respectively. Twenty-five fell below the 80 per cent standard.
Textual comments on all 37 EPAs were examined and processed. Seven of the 12 EPAs for which consensus was reached were removed from the list, along with 12 other EPAs. Based on the textual feedback, all these removed EPAs were considered to be more related to general skills and less specific to surgical skills. For 15 EPAs, both above and below 80 per cent consensus, it was concluded that some components had to be reformulated because narrative feedback indicated that described EPAs were not complete or entirely unnecessary, respectively. For seven EPAs, based on narrative feedback, several smaller units of activities were ‘nested’ within ‘broader’ EPAs (see online supplementary file 2). Based on textual feedback, the EPA ‘Treating an acute eye problem’ was considered too broad, and it was decided to divide this EPA into six separated smaller new EPAs (number 7–12 in online supplementary file 3).
The answers to the question ‘Is there an EPA missing from this list which cannot be excluded as a basic surgical skill at time of graduation?’ were used to create two new EPAs—‘Opening of an abscess’ and ‘Treating a Gastric Dilation Volvulus patient’—and included in the second Delphi round.
Delphi round 2
The second round consisted of 21 EPAs. Two new EPAs were added and 19 initial EPAs were modified based on the textual feedback provided in the first round (see online supplementary file 3). Consensus was reached on the relevance for seven EPAs. As in the first round, additional textual feedback on all EPAs was examined and processed.
Some of the EPAs were divided into animal species and sex (‘nested’ EPAs) but consensus was not reached for all species or sex. These nested EPAs with a level of entrustment of at least 3 were included in the final list.
For the EPAs ‘Placing of a Penrose® drain: Care and removal’, ‘Placing a feeding nose tube (cat)’ and ‘Bladder catheterization’, although the level of entrustment was at least 3, it was judged based on the textual feedback that they were more related to general skills and less specific to surgical skills and were therefore removed from the list. After the second Delphi round, 10 EPAs were additionally included to the list of EPAs related to surgical skills.
Delphi round 3
After the third Delphi round containing 13 EPAs no new comments were received, and therefore this list was considered to be the final list of EPAs related to surgical skills in companion animal health that should be adequately developed at time of graduation (table 2).
This study identified a list of 13 relevant surgical skills defined as EPAs in competency-based education (CBE) of companion animal health. In this study, the participating veterinarians stated that a recently graduated veterinarian must be able to perform the EPAs related to surgical skills at, at least, a level of ‘permission to act with indirect supervision, not present but quickly available if needed’. The established list of EPAs can be used for direct assessment of a student demonstrating a surgical activity. Since experience in clinical practice for students is usually organised in a rotational system with fixed periods of time, the EPA concept may pose logistical challenges, as it requires flexibility in time. Students should be certified for EPAs and competencies that they have been demonstrated to possess,10 whereby the outcome of training becomes more important than its length.5
Hill et al 23 established by a mail-based survey among general practitioners in the USA a ranked list with the frequency of use and proficiency in performance of 26 core surgical skills expected of entry-level veterinarians. Although not directly formulated according to the EPA concept, this list still gives a good opportunity to compare the present study’s list of EPAs for the Dutch curriculum with curricula in the USA. Most of the core surgical skills with a high proficiency, meaning minimal or no supervision required, as indicated by Hill et al 23, resemble the present study’s EPA list directly, for example, ‘Basic knot tying and suturing’ or ‘Preparing the patient for surgery’, or are part of formulated EPAs, for example, ‘Castration of a healthy patient and treating possible complications’. This latter EPA includes, for example, core surgical skills as ‘Atraumatic cosmetic closure of skin’ and ‘Atraumatic manipulation of tissue and viscera’. Also comparable with the study of Hill et al 23 is the fact that orthopaedic procedures are considered to be less relevant at time of graduation or even lacking, and are considered to be procedures that should be developed during postgraduate continuing education.
After the first Delphi round it was proposed for some EPAs to be split up into smaller units of activities, often animal species and/or sex-related, indicating that the relevance and the level of entrustment of particular activities differ between species. These smaller or granular EPAs can be entrusted to a trainee first, and while advancing through training the smaller EPAs can be ‘nested’ in ‘broader’ EPAs for more advanced trainees.6 These ‘nested’ EPAs can be used more specifically for individual learning paths and longitudinal assessment and to develop increasing entrustment in a student.
Based on the programme outcomes, veterinary schools can decide which smaller units are essential to achieve before entrustment can be provided.6 On the other hand, EPAs should comprise a sufficient set of practical work (skills and competencies) and should not become too small. For example, ‘Preparing the patient for surgery’, ‘Gloving and gowning (sterile technique)’ and ‘Basic knot tying and suturing’ were proposed as separate EPAs, whereas they can be seen as specific skills with particular competencies that fit very well in broader EPAs such as ‘Castration of a healthy patient and treating possible complications’. It could be argued that some of the EPAs identified in this study are more granular (eg, basic knot tying and suturing) than others; however, participants identified these as separate, crucial EPAs that were considered that relevant that they were mentioned as separate (smaller) EPAs. This is part of the process of unravelling the use and value of EPAs and EPAs competencies frameworks for curriculum development and assessment. The Association of American Veterinary Medical Colleges recently published eight core EPAs for veterinary education.24 The EPAs described in this study extend this work by defining additional EPAs specifically for surgical skills training.
A limitation of this study might be the modified format of the Delphi procedure. Due to unfamiliarity of the participants with the new concept of EPAs, the authors choose to create a predefined list of EPAs based on the current programme outcomes,15 which might have influenced the composition of the draft versions of the EPAs or resulted in lacking other relevant EPAs related to surgical skills that were not included in the modified Delphi procedure.
The list with the draft version with 37 EPAs was, besides by two veterinarians involved in educational research, established by three surgical specialists, which might have caused bias to the proposed list with units of surgical practice. All three specialists were intensively involved in surgical training for undergraduate students to become a starting veterinarian in private veterinary practice. In addition, in the modified Delphi procedure, both a mixed group of specialists, residents and interns (about 50 per cent of respondents), from all kinds of disciplines (eg, internal medicine, ophthalmology, emergency and critical care, dermatology, exotic animals), and companion animal practitioners in private practice and involved in extramural teaching to students (also about 50 per cent of the respondents), experiencing the best what and how about the surgical skills in first-line private practice, gave their critical opinion.
A point of discussion is the given weight to entrustment scores and qualitative feedback in relation to consensus on relevance. As the majority of participants indicated a high level of entrustment to be important, although the level of relevance being somewhat below the present study’s 80 per cent consensus threshold, it was decided to include that particular EPA in the final list. For the qualitative feedback it is somewhat more complex. When participants came up with a particular suggestion (eg, textual adaptation or new EPA), it was decided to incorporate this suggestion for a new EPA or addition of a ‘nested’ EPA. The use of the Delphi and modified Delphi methods in medical education has been critically reviewed by Humphrey-Murto et al.25 They concluded in their review that the Delphi and modified Delphi methods accounted for over 75 per cent of the consensus methods reported in medical education research articles. They also stated that the most common purposes for using these consensus group methods in medical education research are new curriculum development or reform, assessment tool development, and defining competencies,25 as done in this study. Humphrey-Murto et al 25 also identified points of concern by applying these methods, such as a lack of reported feedback, inability to determine who the participants were, and often consensus was not achieved. Reflecting to these concerns, the authors have described the participants, reported feedback and defined consensus including arguments why and when to deviate.
The strength of this study includes the opinion of a substantial group of veterinarians working within companion animal health distributed in different clinical practices in the Netherlands. The findings are relevant for the development of a more individualised approach to veterinary education with a more robust assessment whether students are ready for safe and high-quality practice without supervision. This list offers the opportunity to implement the concept of EPAs as units of professional practice that can be entrusted as a task to a trainee (student) with sufficient competence, into an individualised outcome-dependent CBE programme. The next step will be defining the specific description of the contents and competencies covered by the particular EPAs.6 Parallel to this, both supervisors and students need to become increasingly familiar with the new concept of EPAs and the ability to trust students to perform activities with increasing independence, based on observations and feedback from multiple observers in combination with self-reflection.
The authors thank the veterinarians who compiled the list of EPAs related to surgical skills as an input for the modified Delphi procedure and all the participants for their contribution to the modified Delphi procedure.
RPF and MG contributed equally.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Ethics approval All participants of the interview sessions and the Delphi procedure were informed that participation was voluntary and full confidentiality and anonymity were assured (informed consent). Participants could only enter the survey if they agreed on participation.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.