P-120 - EVALUATING PERFORMANCE OF PERIPHERAL ENDOVASCULAR INTERVENTIONS: A DELPHI CONSENSUS

TOPIC:
Education & Training
AUTHORS:
Soenens G. (Department of Thoracic and Vascular Surgery, Ghent University Hospital ~ Ghent ~ Belgium) , Gordon L. (Division of Vascular Surgery, University of Toronto ~ Toronto ~ Canada) , Doyen B. (Department of Thoracic and Vascular Surgery, Ghent University Hospital ~ Ghent ~ Belgium) , Sunavsky J. (Department of Medicine, Queen's University ~ Kingston ~ Canada) , Wheatcroft M. (Division of Vascular Surgery, University of Toronto ~ Toronto ~ Canada) , De Mestral C. (Division of Vascular Surgery, University of Toronto ~ Toronto ~ Canada) , Palter V. (International Centre for Surgical Safety, Li Ka Shing Knowledge Institute of St. Michael's Hospital ~ Toronto ~ Canada) , Grantcharov T. (International Centre for Surgical Safety, Li Ka Shing Knowledge Institute of St. Michael's Hospital ~ Toronto ~ Canada) , Van Herzeele I. (Department of Thoracic and Vascular Surgery, Ghent University Hospital ~ Ghent ~ Belgium)
Introduction:
Comprehensive operative capture systems such as the Operating Room Black Box (ORBB) allow for detailed performance assessment by continuously recording and synchronizing multiple intraoperative data sources. (1) Additionally, tools for endovascular performance assessment are necessary, especially because surgical training is shifting towards competency-based education. (2) This study aimed to develop step, error and event frameworks to evaluate technical performance in in peripheral endovascular procedures (PVI).
Methods:
A modified Delphi consensus (3) was used to create evaluation frameworks for steps, errors and events in iliaco-femoro-popliteal endovascular interventions because of symptomatic stenotic/occlusive arterial disease. International experts from Vascular Surgery, Interventional Radiology, Cardiology and Angiology were identified, based on scientific track record. In an initial open-ended survey round, experts volunteered a comprehensive list of steps, errors and events. In subsequent Delphi rounds, items were presented to rate on a 5-point Likert scale and repeated until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) (Figure 1). The items in the final frameworks achieved agreement (response: 4 or 5) from >70% of experts. Framework validation via independent video rating of ten cases by two trained assessors is in progress.
Results:
28 of 98 invited proceduralists formed the expert panel, consisting of 3 Angiologists, 7 Interventional Radiologists, 5 Cardiologists and 13 Vascular Surgeons with 29% from North-America and 71% from Europe. Most (93%) had performed more than 500 PVI procedures, of which > 100 in the last year (86%). Consensus criteria were met following the third Delphi round (Cronbach's alpha; α steps = 0.79; α errors = 0.90; α events = 0.90). In the final framework 15 steps (agreement 73-100%), 26 errors (agreement 73-100%) and 18 events (agreement 73-100%) have been identified and included (Table 1).
Conclusion:
An evaluation tool for the procedural steps, errors and events in iliaco-femoro-popliteal endovascular procedures was developed through a modified Delphi consensus. The framework can be applied to recorded intraoperative data by data capturing platforms such as the ORBB and thereby help to identify hazardous steps, common errors and events. Surgical mastery could be promoted by using the framework to provide proceduralists (both trainees and consultants) with detailed, actionable feedback on technical performance.
References:
1 Jung JJ, Jüni P, Lebovic G, Grantcharov T. First-year Analysis of the Operating Room Black Box Study. Annals of Surgery 2020;271(1):122-7. Doi: 10.1097/SLA.0000000000002863. 2 Reznick RK, MacRae H. Teaching surgical skills--changes in the wind. The New England Journal of Medicine 2006;355(25):2664-9. Doi: 10.1056/NEJMRA054785. 3 de Villiers MR, de Villiers PJT, Kent AP. The Delphi technique in health sciences education research. Medical Teacher 2005;27(7):639-43. Doi: 10.1080/13611260500069947.
ATTACHMENTS: