Abstract
Context: Traditionally, an apprenticeship model has been used for the instruction of invasive bedside procedures. Because this approach is subject to nonuniform application, a new model was established to determine the impact of a standardized curriculum on medical students' and residents' medical knowledge and technical skills.
Methods: A procedural instruction curriculum for medical students and residents was developed, and a pilot program with the curriculum was incorporated into an internal medicine residency program. Five common procedures in osteopathic and allopathic internal medicine training programs were included: central venous catheterization, knee arthrocentesis, lumbar puncture, paracentesis, and thoracentesis. An initial assessment of participants' baseline knowledge and skills was obtained. Teaching methods included video instruction; discussion of key concepts; faculty-led, hands-on, simulation-based instruction; and individual deliberate practice. Postinstruction knowledge and skills were evaluated, respectively, through a written test and a quantified assessment (ie, checklist) using direct observation. Participants were asked to provide written feedback at the conclusion of each instructional module.
Results: A total of 60 participants, all in allopathic medicine, underwent the training component. Fifty-two participants were internal medicine residents (including 2 from an outside program); 4 were trainees in a combined internal medicine–pediatrics residency; and 4 were medical students (1 from an outside program). Participants demonstrated a statistically significant improvement (P<.001) in medical knowledge, as evidenced by preinstruction vs postinstruction test scores. Comparison of initial baseline procedural checklist scores with postinstruction checklist scores, during participants' performance on the first live patient, also showed statistically significant improvement (P<.001).
Conclusion: A simulation-based, standardized curriculum in invasive bedside procedural instruction significantly improved the medical knowledge and technical skills of novice physicians.
Since the 1990s, the term
patient safety has been transformed from an esoteric phrase to part of physicians' daily vernacular. The 1999 Institute of Medicine report
To Err is Human: Building a Safer Health System1 indicted medical errors as a cause of harm or death in some 50,000 to 100,000 patients annually. Invasive bedside medical procedures are associated with inherent risks of errors and complications, leading to increases in length of hospital stays and higher associated healthcare costs.
2
Historically, medical residents have been assigned an arbitrary number of procedures to perform under supervision before being allowed to perform them independently.
3 Trainees often completed this predetermined number of procedures by the end of their residencies without developing sufficient proficiency to perform the procedures independently and safely—let alone the mastery necessary to train other physicians. Indeed, many residents report a lack of comfort in the performance of invasive bedside procedures.
4 In addition, a survey of members of the American College of Physicians revealed a decline in the number of such procedures performed by general internists upon completion of their training.
5
Further complicating traditional residency training is the premise of apprenticeship learning. In this paradigm, the learner is completely dependent on the teacher's knowledge base and technical prowess. Therefore, if the teacher is unable to perform a procedure effectively, errors or complications will likely be propagated.
In light of these concerns, a few residency training programs have begun to abandon this less-than-perfect approach.
6,7 Thus, there is need for a consistent resident training and evaluation method to ensure the development of procedural skills and basic competency. The development of such a method should begin with novice physicians.
To our knowledge (ie, to the knowledge of myself and my colleagues at the University of Miami-Jackson Memorial Hospital Center for Patient Safety in Miami, Florida), few allopathic and no osteopathic postgraduate training programs use a standardized curriculum in procedural instruction. Hypothesizing that simulation-based standardized instruction would improve medical knowledge and technical skills of medical students and residents, we developed a procedural instruction curriculum as a pilot program designed to assess knowledge and skills subsequent to regular curricular implementation.
Methods
The present project received initial local institutional review board (IRB) approval from the University of Miami Miller School of Medicine (UMMSM) under an expedited review process in June 2008. The project encompassed the 2007-2008 academic year (July 2007-June 2008) retrospectively, and it has subsequently undergone annual review with continuance. Data reported in the present article are from the pilot phase of the project, from July 2007 through September 2008.
Participating medical students and residents were instructed in the procedures of the present study during monthly sessions at the jointly funded University of Miami-Jackson Memorial Hospital (UM-JMH) Center for Patient Safety in Miami, Florida. Instructional sessions were repeated every 4 weeks throughout the academic year, beginning on July 2, 2007, in concert with the change in resident rotation. As a component of their training (ie, monthly rotational experience), participants then performed the procedures on patients at Jackson Memorial Hospital in Miami. This hospital is an urban, tertiary-care, academic medical center with a primary affiliation with UMMSM.
Test Development
We created procedure-specific written tests to assess participants' medical knowledge. Because bedside checklists have been shown to reduce complications and improve patient-safe practices,
8 we also created procedural checklists to be used in scoring participants on their technical skills in each procedure. For example, the checklist for central venous catheterization appears in
Appendix 1.
Instructor Recruitment and Training
In an effort to strengthen teaching in the training program, we sought to craft a multidisciplinary team of instructors. Support from a variety of departments, at both the University of Miami and Jackson Memorial Hospital, was solicited via e-mail from those faculty members previously identified by students and residents as having excellent teaching skills within their respective areas. We recruited volunteer instructors from the departments of anesthesiology, emergency medicine, family medicine, general internal medicine, neurology, and radiology.
We used instructional videos in clinical medicine that we obtained from those available on the
New England Journal of Medicine Web site (
http://content.nejm.org/misc/videos.dtl?ssource=recentVideos) after receiving explicit permission to use the videos. In an effort to standardize participants' training, faculty instructors were asked to review the videos and to modify their techniques in accordance with the video presentations.
Instructors were oriented to the models and curriculum by the program director (J.D.L.) through a brief instructional session. The director subsequently observed the instructors during their initial teaching sessions. The director provided verbal feedback, as necessary, to help the instructors revise their teaching techniques and to ensure consistent training and evaluation methods.
Participant Recruitment and Training
Participating medical students volunteered for elective rotations. A single slot was available for medical students on a monthly basis. Participating residents also volunteered for the elective rotations during their internal medicine residency training.
Inclusion criteria were enrollment in a medical school; satisfactory completion of the required clerkships in medicine and surgery; or enrollment in a postgraduate training program accredited by the Accreditation Council for Graduate Medical Education or Council on Graduate Medical Education—as well as willingness to participate and to sign the required IRB-approved informed consent document. Exclusion criteria were unwillingness to participate or to sign the informed consent form. Interns were specifically excluded as a result of scheduling logistics.
The total number of participants—including students and residents—was generally limited to 4 per rotation in an effort to provide all participants with ample opportunity to perform procedures on hospitalized patients.
Equipment Procurement
After careful consideration and review of commonly available products, we created a list of required task-trainers (eg, mannequins or models) for the participants to use. Funding was obtained through the internal medicine residency program and the UM-JMH Center for Patient Safety. With this underwriting, we purchased task-trainers for the following five common procedures:
central venous catheterization (Central Venous Access Hands-On Training Model, item BPH600f; Blue Phantom, Redmond, Washington)
knee arthrocentesis (Large Left Knee Injection Model, item 1517-1; Sawbones, Vashon, Washington)
lumbar puncture (Spinal Injection Simulator, item AB-1030; Armstrong Medical Industries, Lincolnshire, Illinois)
paracentesis (Thoracentesis/Paracentesis Task Trainer, item 1513-36; Sawbones, Vashon, Washington)
thoracentesis (Thoracentesis Ultrasound Training Model, item BPTT1000-1; Blue Phantom, Redmond, Washington)
We also purchased vascular access trainers (Branched 2-Vessel Vascular Access Trainer and Branched 4-Vessel Ultrasound Phantom, items BPO100b and BPBV110, respectively; Blue Phantom, Redmond, Washington) and a mobile ultrasound machine (GE LOGIQ e Shared Service Portable Ultrasound System; Priority Medical, Greenbrier, Tennessee).
Instructional Logistics
The instructional component of the program took place during three mornings, for a total of 12 hours. This schedule enabled residents to attend afternoon continuity clinics and manage their other responsibilities. Each of the five procedures was presented as a stand-alone module composed of 12 curricular items (
Figure) that could be taught independently of the other procedures.
To facilitate the participants' ability to perform procedures successfully and efficiently, we provided them with consent cards (Appendix 2), which listed procedure-specific risks that they were encouraged to discuss with patients during the informed decision-making process. We also distributed a set of supply cards (Appendix 3), each of which contained a complete list of necessary supplies for each procedure. These comprehensive cards could be given to the nurse to obtain the equipment.
In addition, we created preprinted, procedure-specific informed consent documents for patients. These documents described the procedures in layman's terms and listed specific potential complications. The use of these forms ensured the standardization of the consent process, so that all patients were given identical information. Finally, we created standardized procedure notes for inclusion in the patients' medical records upon completion of each of the procedures.
Preinstruction and Postinstruction Evaluations
Each participant completed the written preinstruction test to generate data on baseline medical knowledge. Each participant also demonstrated preinstruction procedural technique on the mannequin, and the performance of this technique was evaluated through the use of the procedural checklist (
Appendix 1). The supervising attending physician provided no feedback or coaching. We used an attending physician (rather than a resident) to provide direct supervision, because such supervision has been linked to improved resident procedural competence.
9 We compiled participants' scores and used them to compare baseline knowledge and skills with postinstruction knowledge and skills.
Test questions on medical knowledge were a mixture of true/false and single-answer multiple-choice questions. Responses to all questions were equally weighted. Participants were given 1 point for each correct answer. The total number of questions varied by procedure, and the maximum possible score for each procedure ranged from 10 to 14. The identical test was used before and after the instructional intervention, allowing the trainees to serve as their own controls.
The procedural checklist was scored according to level of completion for each item—0 for a missed item, 1 for an incomplete response, and 2 for successful completion. As with the written tests, the total number of items on the checklist varied by procedure—from 21 to 26—resulting in a maximum possible total score ranging from 42 to 52. All checklist items carried equal weight, because they are all essential to the consistent approach to a procedure.
The checklist tasks for procedural evaluation were divided into three sections (Appendix 1). The first section encompassed universal preprocedural items. The second section was specific for each procedure. The third section contained postinstruction items relevant to all procedures. The inclusion of the first and third sections was intentionally designed to reinforce the necessity of performing those tasks every time for every procedure and to create ingrained procedural habits (akin to a pilot checking the plane before take-off and after landing).
A trainee's initial checklist tally was compared with the checklist score for the trainee after performance of a procedure on a live patient. Checklists were vetted through a multidisciplinary group of faculty, some of whom served as instructors.
Statistical Analysis
We calculated the mean preinstruction test score per procedure and compared it with the mean postinstruction test score. We then used a 2-tailed, paired t test to determine the significance of the difference between preinstruction and postinstruction test scores. Statistical significance was set at P<.001. We used the statistical program SAS 9.2 (SAS Institute Inc, Cary, North Carolina) for all calculations.
Results
A total of 60 participants—all in allopathic medicine—underwent the overall training component of the present study. However, an additional 41 third-year allopathic medical students from UMMSM completed training in paracentesis—extra sessions of which were targeted to those students. Not all participants completed instruction on every procedure. Results of only those participants who had complete data sets for both preinstruction and postinstruction are reported.
Of the 60 participants, 56 were residents and 4 were fourth-year medical students (3 from UMMSM, 1 from Tulane University School of Medicine in New Orleans, Louisiana). Fifty-four residents were in training at Jackson Memorial Hospital, and the other 2 were in their third year of training at Mount Sinai Medical Center in Miami Beach, Florida. Fifty-two residents were in internal medicine programs, and the remaining 4 were in their third year of a combined internal medicine–pediatrics residency program. Thirty-four participants were second-year residents, and 18 were third-year residents.
Table 1 shows comparisons of participants' procedural medical knowledge as scored in preinstruction tests vs postinstruction tests. As previously indicated, the number of participants varied depending on the number completing instruction in each procedure. The results demonstrate immediate, statistically significant improvements (
P<.001) in postinstruction knowledge for all five procedures.
Table 1
Comparisons of Medical Knowledge Written Test Scores, by Procedure, of Participants in University of Miami-Jackson Memorial Hospital Center for Patient Safety Pilot Program (N=60) *
|
|
Score, Mean
| |
|
|---|
Procedure
| Participants, No. (%)†
| Preinstruction
| Postinstruction
| P Value‡
|
|---|
| Central Venous | | | | |
| Catheterization | 47 (78.3) | 6.681 | 8.255 | <.001 |
| Knee Arthrocentesis | 35 (58.3) | 7.486 | 9.314 | <.001 |
| Lumbar Puncture | 47 (78.3) | 5.894 | 9.447 | <.001 |
| Paracentesis* | 101 | 8.366 | 12.515 | <.001 |
Thoracentesis
| 41 (68.3)
| 9.024
| 12.024
| <.001
|
Table 1
Comparisons of Medical Knowledge Written Test Scores, by Procedure, of Participants in University of Miami-Jackson Memorial Hospital Center for Patient Safety Pilot Program (N=60) *
|
|
Score, Mean
| |
|
|---|
Procedure
| Participants, No. (%)†
| Preinstruction
| Postinstruction
| P Value‡
|
|---|
| Central Venous | | | | |
| Catheterization | 47 (78.3) | 6.681 | 8.255 | <.001 |
| Knee Arthrocentesis | 35 (58.3) | 7.486 | 9.314 | <.001 |
| Lumbar Puncture | 47 (78.3) | 5.894 | 9.447 | <.001 |
| Paracentesis* | 101 | 8.366 | 12.515 | <.001 |
Thoracentesis
| 41 (68.3)
| 9.024
| 12.024
| <.001
|
×
Using the same statistical analysis model, we also found statistically significant improvements (
P<.001) in preinstruction vs postinstruction checklist scores for technical skills demonstrated in participants' first performance of procedures on hospitalized patients (
Table 2). However, skill scores for knee arthrocentesis were not recorded or analyzed because this procedure was not performed on hospitalized patients with any significant regularity. In addition, we did not have an opportunity to observe the treatment skills of the medical students who solely underwent paracentesis training. To that end, the skill data of those students were excluded from the final analysis.
Table 2
Comparisons of Technical Skill Checklist Scores, by Procedure, of Participants in University of Miami-Jackson Memorial Hospital Center for Patient Safety Pilot Program (N=60)
|
|
Score, Mean
| |
|
|---|
Procedure*
| Participants, No. (%)†
| Preinstruction
| Postinstruction
| P Value‡
|
|---|
| Central Venous | | | | |
| Catheterization | 40 (66.7) | 43.356 | 51.225 | <.001 |
| Lumbar Puncture | 35 (58.3) | 38.75 | 46.457 | <.001 |
| Paracentesis | 34 (56.7) | 34.912 | 44.975 | <.001 |
Thoracentesis
| 30 (50.0)
| 35.711
| 47.267
| <.001
|
Table 2
Comparisons of Technical Skill Checklist Scores, by Procedure, of Participants in University of Miami-Jackson Memorial Hospital Center for Patient Safety Pilot Program (N=60)
|
|
Score, Mean
| |
|
|---|
Procedure*
| Participants, No. (%)†
| Preinstruction
| Postinstruction
| P Value‡
|
|---|
| Central Venous | | | | |
| Catheterization | 40 (66.7) | 43.356 | 51.225 | <.001 |
| Lumbar Puncture | 35 (58.3) | 38.75 | 46.457 | <.001 |
| Paracentesis | 34 (56.7) | 34.912 | 44.975 | <.001 |
Thoracentesis
| 30 (50.0)
| 35.711
| 47.267
| <.001
|
×
Comment
The educational initiative described in the present report shows promising results. Data support our method of teaching invasive procedures as a means of improving medical knowledge and honing technical skills of novice physicians. The implementation of such a program in an osteopathic resident training facility would be the first of its kind.
A noted limitation of the present study, however, is that the methods were tested in a single facility—a more-than-1500-bed, urban, tertiary-care academic medical center. The feasibility of implementing this same teaching design at smaller institutions is likely but uncertain. Those considering implementing this type of procedural training program need a faculty champion, strong participant desire, and an initial financial outlay for purchasing mannequins, models, and an ultrasound machine. Fortunately, some mannequins and models are relatively inexpensive, and manufacturers of ultrasound machines may be willing to loan or lease equipment.
Some authors have suggested that the definition of competence based on a predetermined number of procedures may have its shortcoming
10,11 and that a training assessment methods such as the one used in the present study could address these deficiencies. Additional studies are needed to replicate our results in both academic and community-based medical centers. Also required is a more direct comparison of our training method with standard training methods. Furthermore, the evaluation of additional procedures and services may be of interest to test the scope of this method of teaching.
Future research will ultimately focus on challenging the conventional paradigm of procedural training and assessing whether training such as that used in the present study reduces iatrogenic complication rates among patients.
Conclusion
Historically, procedural education has been rooted in the apprenticeship model of “see one, do one, teach one.” In the present project, we implemented a new paradigm based on a formal, standardized, simulation-based curriculum, and we tested it in a pilot program at a large, urban, tertiary-care, academic medical center. Expert faculty from a variety of departments served as instructors. Participants demonstrated a statistically significant improvement in both procedural knowledge and skill acquisition after the training.
The scope and impact of our results are noteworthy. Our training methods could be incorporated into residency programs when appropriate resources are available. This approach could potentially be adapted for training individuals in additional procedures beyond those addressed in the present study, including osteopathic manipulative treatment. We believe that the implementation of this method of teaching could transform and improve osteopathic and allopathic procedural medical education and resident training.
Appendix 2
Consent card, listing procedure-specific risks to discuss with patients. *In addition to the risks listed in this table, all procedures have risks of pain and infection.
Procedure
|
Risks*
|
|---|
| Central Venous Catheterization | Arterial puncture |
| Cardiac dysrhythmias |
| Central venous thrombosis |
| Hematoma |
| Hemothorax |
| Pneumothorax |
| Knee Arthrocentesis | Bleeding |
| Damage to cartilage, tendons, or nerves |
| Localized trauma |
| Reaccumulation of fluid |
| Lumbar Puncture | Bleeding |
| Damage to nerve or spinal cord |
| Headache |
| Herniation |
| Persistent leakage of cerebrospinal fluid |
| Paracentesis | Abdominal wall hematoma |
| Circulatory dysfunction, if large volume (>5 L) withdrawn |
| Hemorrhage |
| Injury to intraabdominal organs |
| Persistent leakage of ascitic fluid |
| Thoracentesis | Air embolism |
| Coughing |
| Hemothorax |
| Injury to intraabdominal organs |
| Pneumothorax |
| Postexpansion pulmonary edema, if large volume (>1.5 L) withdrawn
|
Appendix 2
Consent card, listing procedure-specific risks to discuss with patients. *In addition to the risks listed in this table, all procedures have risks of pain and infection.
Procedure
|
Risks*
|
|---|
| Central Venous Catheterization | Arterial puncture |
| Cardiac dysrhythmias |
| Central venous thrombosis |
| Hematoma |
| Hemothorax |
| Pneumothorax |
| Knee Arthrocentesis | Bleeding |
| Damage to cartilage, tendons, or nerves |
| Localized trauma |
| Reaccumulation of fluid |
| Lumbar Puncture | Bleeding |
| Damage to nerve or spinal cord |
| Headache |
| Herniation |
| Persistent leakage of cerebrospinal fluid |
| Paracentesis | Abdominal wall hematoma |
| Circulatory dysfunction, if large volume (>5 L) withdrawn |
| Hemorrhage |
| Injury to intraabdominal organs |
| Persistent leakage of ascitic fluid |
| Thoracentesis | Air embolism |
| Coughing |
| Hemothorax |
| Injury to intraabdominal organs |
| Pneumothorax |
| Postexpansion pulmonary edema, if large volume (>1.5 L) withdrawn
|
×
Appendix 3
Supply card for procedure. Numbers in parentheses indicate how many units of each item are required for the procedure.
Central Venous Catheterization |
| □ Triple lumen central line kit (7 French × 16 cm) |
| or |
| Temporary dialysis catheter kit (16.5 cm; double lumen with or without side port; check with team) Sterile gloves (2) Sterile blue gowns (2) Hairnets (2) Face masks with shield (2) Alcohol prep pads (8) Chux underpads (1) Flushes (≥5) Chlorhexidine swab (1) Sterile, transparent, occlusive dressing (1) Box of sterile 4 × 4 pads (1) Chlorhexidine-impregnated patch (1)
|
Appendix 3
Supply card for procedure. Numbers in parentheses indicate how many units of each item are required for the procedure.
Central Venous Catheterization |
| □ Triple lumen central line kit (7 French × 16 cm) |
| or |
| Temporary dialysis catheter kit (16.5 cm; double lumen with or without side port; check with team) Sterile gloves (2) Sterile blue gowns (2) Hairnets (2) Face masks with shield (2) Alcohol prep pads (8) Chux underpads (1) Flushes (≥5) Chlorhexidine swab (1) Sterile, transparent, occlusive dressing (1) Box of sterile 4 × 4 pads (1) Chlorhexidine-impregnated patch (1)
|
×
Financial Disclosures: None reported.
I am indebted to Lisa Rosen, MA, and S. Barry Issenberg, MD, for their editorial review and advice. I also thank the following people and institutions for their assistance: Stephen Symes, MD; David J. Birnbach, MD, MPH; Jackson Memorial Hospital; the University of Miami; and the staff at the UM-JMH Center for Patient Safety.
Kohn LT, Corrigan JM, Donaldson MS, eds; Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999.
Reynolds MR, Cohen DJ, Kugelmass AD, Brown PP, Becker ER, Culler SD, et al. The frequency and incremental cost of major complications among Medicare beneficiaries receiving implantable cardioverter-defibrillators [published online ahead of print May 30, 2006]. J Am Coll Cardiol.. (2006). ;47(12):2493-2497.
Huang GC, Smith CC, Gordon CE, Feller-Kopman DJ, Davis RB, Phillips RS, et al. Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures. Am J Med. 2006;119(1):e17-e24.
Wigton RS, Alguire P; American College of Physicians. The declining number and variety of procedures done by general internists: a resurvey of members of the American College of Physicians. Ann Intern Med. 2007;146 (5): 355-360.
Lenhard A, Moallem M, Marrie RA, Becker J, Garland A. An intervention to improve procedure education for internal medicine residents [published online ahead of print January 23, 2008]. J Gen Intern Med. 2008;23(3):288-293.
Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, et al. An intervention to decrease catheter-related bloodstream Infections in the ICU [published correction appears in N Engl J Med. 2007;356(25):2660]. N Engl J Med. 2006;355(26):2725-2732.
Sznajder JI, Zveibil FR, Bitterman H, Weiner P, Bursztein S. Central vein catheterization: failure and complication rates by three percutaneous approaches. Arch Intern Med. 1986;146(2):259-261.