Winkler-Schwartz A, Yilmaz R, Mirchi N, et al. Machine Learning Identification of Surgical and Operative Factors Associated with Surgical Expertise in Virtual Reality Simulation. JAMA Netw Open. 2019;2(8):e198363. doi:https://doi.org/10.1001/jamanetworkopen.2019.8363 Link to publication
Laboratory website:
http://neurosim.mcgill.ca/
Dr. Winkler-Schwartz and Dr. Yilmaz live in Montreal.
Using machine learning to better assess surgical skills of neurosurgeons
Surgical interventions carry considerable risks to patients and important costs to health care systems. Given this, the demand for an objective demonstration of surgical competence from stakeholders ranging from patient’s rights groups, governmental organization, insurance agencies and hospital administrations is increasing.
Dr. Alexander Winkler-Schwartz and Dr. Recai Yilmaz explored the use of virtual reality surgical simulators as a means of providing objective assessments in surgery at the Neurosurgical Simulation and Artificial Intelligence Learning Centre at McGill University. To do so, they developed a machine learning algorithm to classify participants by level of expertise in a virtual-reality (VR) surgery.
Their study is the first to demonstrate the ability of machine learning algorithms to classify surgical expertise into groups with high accuracy using fewer than 10 performance measures. Fifty individuals (14 neurosurgeons, four fellows, 10 senior residents, 10 junior residents and 12 medical students) from a single university were recruited to participate in 250 simulated tumor resections using NeuroVR, one of the most advanced surgical simulators available. Of the 50 participants, only five were misclassified in terms of their expertise level.
Despite advances in the assessment of technical skills in surgery, a clear understanding of the components of technical expertise was lacking. This study by Dr. Winkler-Schwartz and Dr. Yilmaz helped to identify surgical and operative factors that relate to level of expertise. Surgical simulation allows for the measurement of psychomotor skills during surgery, which is recorded in huge datasets. The machine-learning algorithms developed by Dr. Winkler-Schwartz and Dr. Yilmaz used in the analysis of these datasets were able to identify factors that contribute to surgical skill and efficiency.
This novel methodology has broad applicability in any circumstance in which technical performance is measured. Artificial intelligence and machine learning systems lend themselves well to the analysis of the enormous datasets generated by simulators and can provide objective and novel insights into the technical components of expertise. This study addresses an important knowledge gap in the area of technical skill assessment and provides tools that can be used in many areas of medicine.
Dr. Alexander Winkler-Schwartz and Dr. Recai Yilmaz
Dr. Alexander Winkler-Schwartz is a Neurosurgery resident and PhD Candidate; Dr. Recai Yilmaz is a medical graduate and a PhD Candidate, both at the Neurosurgical Simulation and Artificial Intelligence Learning Centre at McGill University. They are co-first authors on the manuscript.
Funding:
Dr. Winkler-Schwartz reported receiving grants from Fonds de Recherche du Québec–Santé, Robert Maudsley Fellowship for the Royal College of Physicians and Surgeons of Canada, and Di Giovanni Foundation during the conduct of the study. Dr. Yilmaz reported receiving grants from AO Foundation and Di Giovanni Foundation during the conduct of the study.
