Free
JAOA/AACOM Medical Education  |   October 2018
Perceived Value of a Skills Laboratory With Virtual Reality Simulator Training in Arthroscopy: A Survey of Orthopedic Surgery Residents
Author Notes
• From the Lake Erie College of Osteopathic Medicine in Erie, Pennsylvania.
• Disclaimer: The views expressed herein are those of the authors and do not necessarily reflect the official policy or position of the Department of the Army, the Department of Defense, or the US Government.
• Financial Disclosures: None reported.
• Support: None reported.
•  *Address correspondence to Kevin Keith, DO, MS, PSC 482 Box 2727, FPO AP 96362-1600. Email: kkeith@vcom.edu

Article Information
Medical Education / Neuromusculoskeletal Disorders / Graduate Medical Education
JAOA/AACOM Medical Education   |   October 2018
##### Perceived Value of a Skills Laboratory With Virtual Reality Simulator Training in Arthroscopy: A Survey of Orthopedic Surgery Residents
The Journal of the American Osteopathic Association, October 2018, Vol. 118, 667-672.
The Journal of the American Osteopathic Association, October 2018, Vol. 118, 667-672.
Web of Science® Times Cited: 1
Abstract

Background: Arthroscopy is one of the most common procedures performed by orthopedic surgeons. Virtual reality (VR) simulation in general surgery residency training has been increasing over the past decade, but it has seen little use in the field of orthopedic surgery.

Objective: To determine osteopathic orthopedic surgery residents’ perceived value of having access to a VR simulator before performing an arthroscopic procedure on a live patient.

Methods: A survey was developed and sent to all US osteopathic orthopedic surgery residency programs to be disseminated to all of their current residents. The survey consisted of 12 questions, which included Likert-type scale responses and yes or no responses.

Results: Fifty-eight residents out of approximately 507 responded. Forty-one of 57 respondents (72%) were in year 1 of residency when they performed their first arthroscopy, and 53 of 57 (93%) were not very comfortable when they performed their first arthroscopy. With respect to VR simulator exposure, approximately 31of 51 (61%) reported no exposure to a VR simulator, and 40 of 50 (80%) reported that their program did not provide a skills laboratory where they could practice arthroscopy. Of 50 respondents, 37 (74%) believed that a skills laboratory was important, 28 (56%) believed that a resident should perform 1 to 10 arthroscopies in a skills laboratory before performing one in the operating room, 34 (60%) believed that skills acquired in a skills laboratory would transfer to the operating room, and 33 (66%) agreed that every residency program should provide a skills laboratory. However, 29 (58%) believed that a skills laboratory would not improve patient safety.

Conclusion: Osteopathic orthopedic surgery residents indicated that they would benefit from the addition of an arthroscopic skills laboratory with a VR simulator. Furthermore, they believed that the skills learned in the skills laboratory would transfer to the operating room and would increase their comfort level with the procedure.

Orthopedic surgeons are required to obtain the skills necessary to perform arthroscopy. Currently, this skill is developed during residency in an apprentice-master environment, with the apprentice learning on real patients. However, because of the resident work-hour rules, which limit residents’ times with mentors,1 and increased administrative responsibilities of faculty, this model is less than ideal, leading to underprepared orthopedic surgeons entering the workforce. Therefore, an updated process for the training of arthroscopic surgical skills, using competency-based education (CBE) and virtual reality (VR) simulation, is necessary.
Competency-based education is becoming more prevalent and relevant as residency programs reform their graduate medical education (GME) curricula. A complex interaction between cognitive, perceptual, and behavior skills, CBE focuses on outcomes and the skills that students are competent in, allowing residents to progress at their own pace. Residency training around the world has already begun to shift to hybrid frameworks and focused competency-based outcomes to promote the flexibility to meet the needs of individual learners while confronting the necessary outcomes of health care.2 Implementation of a CBE curriculum in GME training will not only provide residency programs a way to meet learning needs, but also demonstrate residents’ proficiency through a summative assessment tool to better gauge their progress.2
The most recent requirements for orthopedic surgery residents by the Accreditation Council for Graduate Medical Education (ACGME) had an effective date of July 1, 2014.3 Along with the 6 core competencies, programs are required to provide basic motor skills training during year 1. This training includes tissue handling, bone management, suturing, fluoroscopy, basic use of orthopedic equipment, and arthroscopy. This requirement alone is reason enough for the development of CBE curricula that include basic arthroscopy skills training. Although the technology itself has continued to advance, simulation in medical education serves as a modality for improving resident skills, and many orthopedic residency education models hinge on this educational paradigm.4 What still needs additional insight, however, is full evaluation of the effectiveness of these paradigm shifts, including the use of VR for training in arthroscopy.
Cannon et al5 used VR simulators to evaluate the skill level between trainees and community orthopedic surgeons. This study demonstrated that as year-1 residents completed more repetitions, their time decreased. This held true for the year-5 residents as well, with their learning curve flattening sooner. Shelty et al6 explored general surgery residents’ perceptions, training, and preferences with regard to laparoscopic simulation training. The investigators found that year-1 residents preferred VR simulators to obtain the skills needed to perform laparoscopic surgery. This preference differed from that of the upper-level residents, who preferred live animal models. The reason behind this difference was reportedly due to the upper-level residents having more operative experience, thereby rendering the virtual simulators as less educational. Grantcharov et al7 found that with experienced surgeons performing VR simulated laparoscopy, their error rate, economy of motion, and time did not differ between repetitions. However, with novice surgeons it was not until the seventh repetition that they were completing the task in nearly the same amount of time, with the same error rate, and fewer unnecessary movements as the intermediate and advanced surgeons.
Studies to determine the perceptions of osteopathic orthopedic surgery residents regarding the value of VR simulator training are lacking. The intent of the current study, therefore, was to survey current osteopathic orthopedic surgery residents with regard to their perception of the value of arthroscopy skills laboratories and to expand the literature.
Methods
This educational institutional review board–approved study used a nonexperimental design to assess orthopedic surgery residents’ value of skills laboratories and VR simulator training. Residents were recruited from all osteopathic orthopedic surgery residencies in the United States. The American Osteopathic Association's opportunities website6 was queried in August 2014 for orthopedic residency programs, and 38 programs were found. The point of contact or program director was recorded, as well as the number of filled positions. Approximately 507 filled positions were found. An email with the link to an online survey was sent to each residency's point of contact in August with a request to forward it to current residents. Each point of contact was sent 2 email reminders, 1 in November 2014 and 1 in February 2015, during the open period.
Data were collected using an anonymous online questionnaire (Survey Monkey) administered between September 1, 2014, and March 20, 2015. The only demographic data obtained were the level of training. The questionnaire comprised 12 questions in Likert-type and yes/no response format. The survey was developed by the lead author during the completion of a master of science degree in medical education. It was reviewed by faculty and other residents for face and construct validity. In accordance with institutional review board exemption, and to ensure complete anonymity, the option to track an IP address was disabled within the website.
Questionnaire responses were coded and analyzed descriptively. Likert-type questions required participants to provide answers on a scale of 1 to 5, corresponding with “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree” or equivalent-type responses. “Strongly agree” and “agree,” or their equivalent, and “strongly disagree” and “disagree” were grouped into 1 “agree” and “disagree” category for the graphical representation of results. For the yes/no questions, “yes” was assigned number 1 and “no” was assigned number 2.
Results
A total of 58 osteopathic orthopedic surgery residents responded to the survey, resulting in an 11.4% response rate. Of the 58 respondents, 50 completed the entire survey, 7 completed the majority of the survey, and 1 responded with year in training only. The resulting percentages are based on the number of respondents who answered each question. The demographic data collected were the year of current training (year 1 [n=13]; year 2 [n=13]; year 3 [n=9]; year 4 [n=10]; and year 5 [n=12]).
The majority of respondents (41 of 57 [72%]) reported that they were in year 1 of residency when they performed their first arthroscopy. Fifty-three of 57 respondents (93%) were not comfortable when they performed their first arthroscopy, regardless of their year of training (Table). Eighteen of 57 respondents reported needing to complete 21 to 30 arthroscopies before feeling comfortable (Figure). Approximately 40% of year-1, year-3, and year-4 respondents reported needing to perform more than 40 arthroscopies in order to feel comfortable. Forty-five percent of year-2 respondents reported needing to perform 11 to 20 arthroscopies, with the remaining 55% needing to perform 21 or more. Of the year-5 respondents, 50% reported needing to perform 21 to 30 arthroscopies.
Figure.
Number of arthroscopies needed to feel comfortable reported by osteopathic orthopedic residents (n=57).
Figure.
Number of arthroscopies needed to feel comfortable reported by osteopathic orthopedic residents (n=57).
Table.
Orthopedic Resident Comfort Level While Performing First Arthroscopy (n=57)
Comfort Level No. (%)
Not comfortable 35 (61)
Little comfortable 18 (32)
Comfortable 4 (7)
Very comfortable 0
Extremely comfortable 0
Table.
Orthopedic Resident Comfort Level While Performing First Arthroscopy (n=57)
Comfort Level No. (%)
Not comfortable 35 (61)
Little comfortable 18 (32)
Comfortable 4 (7)
Very comfortable 0
Extremely comfortable 0
×
With respect to VR simulator exposure, 31 of 51 (61%) reported no exposure to a VR simulator, and 40 of 50 (80%) reported that their program did not have a skills laboratory available to practice arthroscopy. Of the 10 respondents who reported access to an arthroscopic skills laboratory in the program, 5 of 10 (50%) reported that the skills laboratory was not helpful. However, 37 of 50 respondents (74%) believed that a skills laboratory was important. With the exception of the year-4 residents, the level of training was not a factor in the perceived importance of a skills laboratory. Four of 9 year-1 residents (40%) believed that it was very important, and 8 of 19 year-2 and year-3 residents (40%) and 4 of 12 year-5 residents (33%) believed that a skills laboratory was extremely important. The year-4 residents were split, with 5 of 10 indicating that it was not important and 5 of 10 indicating that it was important.
Additionally, 28 of 50 residents (56%) believed that a resident should perform 1 to 10 arthroscopies in a skills laboratory before performing one in the operating room, and 12 of 50 (24%) believed that 11 to 20 should be required. Twenty-nine of 50 respondents (58%) believed that a skills laboratory would not improve patient safety, and 12 of 50 (24%) were neutral. Thirty-four of 50 (68%) believed that skills acquired in an arthroscopic laboratory would transfer to the operating room, and 33 of 50 respondents (66%) agreed that every orthopedic residency program should provide a skills laboratory.
Discussion
This study constituted a cross-sectional survey design to characterize osteopathic orthopedic surgery residents from training programs across the United States with regard to their attitudes toward skills laboratories, their training experiences, and their perceived importance of simulation training in arthroscopy.
The exact curriculum that each program used for arthroscopy training was unknown, but at a minimum, a didactic instruction of skills necessary was assumed. The results demonstrated that the vast majority of the respondents were uncomfortable while performing their first arthroscopy, most of which were performed during year 1 of residency. A study by Cannon et al5 found that year-1 residents took a substantially longer time to perform a set of given tasks compared with upper-level residents. Thus, as a resident performs more procedures, the more comfortable he or she becomes with those procedures. Another contributing factor to a low comfort level in this study could be the lack of arthroscopic VR simulator exposure or lack of a skills laboratory in the residency. In the current study, of the 4 respondents who reported feeling comfortable during their first arthroscopy, 3 had access to an arthroscopic skills laboratory. This finding alone supports the need for further CBE training in arthroscopy. Through VR simulation, residents can become proficient in this skill by practicing the procedure until they are competent.
One surprising finding was the fact that of the 10 respondents who had access to an arthroscopic skills laboratory, 50% reported that this laboratory was only a little helpful. Of those respondents, 1 was in year 4 of training and the other 4 were in year 5. What is not known, however, is at what point in the training program the skills laboratory became available or was part of the curriculum and why some respondents indicated that it was a “little” helpful. Shelty et al,6 for example, found that upper-level residents preferred live animal models because of the more realistic tissue handling characteristics.
The number of arthroscopies needed to perform for the residents to feel comfortable was consistent across all year levels. Cannon et al5 showed that higher levels of surgeon experience resulted in improved efficiency when performing diagnostic knee arthroscopies. Likewise, it has been shown that with repetition of a procedure, the novice surgeon begins to perform it in the same time, with the same error rate, and more efficient motions as the surgeons with intermediate and advanced experience.7
The majority of respondents at each year level believed that a resident should perform between 1 and 10 arthroscopies in a laboratory before performing it in the operating room. Likewise, the majority of all respondents in each year level believed that the skills learned in a skills laboratory would transfer to the operating room. Cannon et al8 demonstrated the transfer validity of VR arthroscopic simulator training before performance in an operating room. As to whether or not the laboratory would improve patient safety, the percentage of respondents in each year was similar to those who believed that the laboratory was important. Although 66% of the respondents agreed that a residency program should provide a skills laboratory, the question remains as to why there was not 100% agreement, especially because all respondents agreed that residents should perform arthroscopy in the laboratory before performing it on a real patient. This is an area for further research.
This study has a few limitations. The most apparent is the low response rate; however, this rate is not uncommon in survey responses among residents.9 It is possible that the low response rate may have skewed the data. This factor may have affected the conclusions, especially regarding the question about the importance of an arthroscopic skills laboratory, considering only 10 of the 58 respondents had an arthroscopic skills laboratory available. The likelihood of recall bias is another limitation and may explain the 50% of the respondents with access to an arthroscopic skills laboratory who reported it as unhelpful. Another limitation is that we did not compare resident performance between those residents with training in an arthroscopic skills laboratory and those without. This is an area for further research to better quantify the importance of simulation and implementation of a CBE curriculum.
While both the ACGME and the American Osteopathic Association navigate through the development of the single accreditation system, it is yet to be determined whether VR simulation as part of a CBE-based curriculum will be required of ACGME residency programs with osteopathic recognition. The cost of a VR system in knee arthroscopy starts at approximately $85,000 for the system and a possible additional cost of$10,000 or more per year for an annual service agreement. Another option would be for the programs to budget for arthroscopic courses that residents could attend, one of which is hosted by the Arthroscopy Association of North America. Currently the registration cost is about \$1500 per resident and does not include travel or hotel. Although the added costs can be substantial, other institutions have reported increased trainer and trainee satisfaction and direct evidence of improved and more efficient learning outcomes.10
Conclusion
This study supports the literature in the indication for CBE training that includes VR arthroscopy simulation. The majority of osteopathic orthopedic surgery residents who responded to the survey believed that an arthroscopic skills laboratory was important, skills learned in the simulation laboratory would transfer to the OR, and an arthroscopic skills laboratory should be provided by residency programs. Given the number of arthroscopy procedures performed by residents (or number of procedures in general), a skills laboratory with a VR simulator and associated curriculum would not only add value to the educational experience of the junior residents, but it would likely improve patient care.
Author Contributions
All authors provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; Drs Keith and Johannessen drafted the article or revised it critically for important intellectual content; Drs Keith and Johannessen gave final approval of the version of the article to be published; and all authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
References
Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-645. doi: 10.3109/0142159x.2010.501190
Turner JL, Dankoski ME. Objective structured clinical exams: a critical review. Fam Med. 2008;40(8):574-578.
AGCME Program Requirements for Graduate Medical Education in Orthopedic Surgery. Chicago, IL: Accreditation Council for Graduate Medical Education; 2014.
Gibbs DB. Surgical simulation in orthopedic surgery residency. Am J Orthop (Belle Mead NJ). 2016;45(7):e426-e428.
Cannon W, Nicandri G, Reinig K, Mevis H, Wittstein J. Evaluation of skill level between trainees and community orthopaedic surgeons using a virtual reality arthroscopic knee simulator. J Bone Joint Surg Am. 2014;96(7):e57. doi: 10.2106/JBJS.M.00779
Shelty S, Zevin B, Grantcharov TP, Roberts KE, Duffy AJ. Perceptions, training experiences, and preferences for surgical residents toward laparoscopic simulation training: a resident survery. J Surg Educ. 2014;71(5):727-733. doi: 10.1016/j.jsurg.2014.01.006
Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J. Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am J Surg. 2003;185(2):146-149.
Cannon WD, Jr Garret WE, Hunter RE, et al. Improving residency training in arthroscopic knee surgery with use of a virtual-reality simulator. J Bone Joint Surg Am. 2014;96(21):1798-1806. doi: 10.2106/JBJS.N.00058
Burns KE, Duffett M, Kho ME, et al. A guide for the design and conduct of self-administered surveys of clinicians. CMAJ. 2008;179(3):245-252. doi: 10.1503/cmaj.080372
Nousiainen MT, McQueen SA, Ferguson P, et al. Simulation for teaching orthopaedic residents in a competency-based curriculum: do the benefits justify the increased costs? Clin Orthop Relat Res. 2016;474(4):935-944. doi: 10.1007/s11999-015-4512-6
Figure.
Number of arthroscopies needed to feel comfortable reported by osteopathic orthopedic residents (n=57).
Figure.
Number of arthroscopies needed to feel comfortable reported by osteopathic orthopedic residents (n=57).
Table.
Orthopedic Resident Comfort Level While Performing First Arthroscopy (n=57)
Comfort Level No. (%)
Not comfortable 35 (61)
Little comfortable 18 (32)
Comfortable 4 (7)
Very comfortable 0
Extremely comfortable 0
Table.
Orthopedic Resident Comfort Level While Performing First Arthroscopy (n=57)
Comfort Level No. (%)
Not comfortable 35 (61)
Little comfortable 18 (32)
Comfortable 4 (7)
Very comfortable 0
Extremely comfortable 0
×