Abstract
Background: The Joint Commission requires hospitals to develop systems in which a team of clinicians can rapidly recognize and respond to changes in a patient’s condition. The rapid response team (RRT) concept has been widely adopted as the solution to this mandate. The role of house staff in RRTs and the impact on resident education has been controversial. At Christiana Care Health System, eligible residents in their second through final years lead the RRTs.
Objective: To evaluate the use of a team-based, interdisciplinary RRT training program for educating and training first-year residents in an effort to improve global RRT performance before residents start their second year.
Methods: This pilot study was administered in 3 phases. Phase 1 provided residents with classroom-based didactic sessions using case-based RRT scenarios. Multiple choice examinations were administered, as well as a confidence survey based on a Likert scale before and after phase 1 of the program. Phase 2 involved experiential training in which residents engaged as mentored participants in actual RRT calls. A qualitative survey was used to measure perceived program effectiveness after phase 2. In phase 3, led by senior residents, simulated RRTs using medical mannequins were conducted. Participants were divided into 5 teams, in which each resident would rotate in the roles of leader, nurse, and respiratory therapist. This phase measured resident performance with regard to medical decision making, data gathering, and team behaviors during the simulated RRT scenarios. Performance was scored by an attending and a senior resident.
Results: A total of 18 residents were eligible (N=18) for participation. The average multiple choice test score improved by 20% after didactic training. The average confidence survey score before training was 3.44 out of 5 (69%) and after training was 4.13 (83%), indicating a 14% improvement. High-quality team behaviors correlated with medical decision making (0.92) more closely than did high-quality data gathering (0.11). This difference narrowed during high-pressure scenarios (0.84 and 0.72, respectively).
Conclusion: Our data suggest that resident training using a team-based, interdisciplinary RRT training program may improve resident education, interdisciplinary team-based dynamics, and global RRT performance. In turn, data gathering and medical decision making may be enhanced, which may result in better patient outcomes during RRT scenarios.
The goal of rapid response teams (RRTs) is to treat severely ill patients before they develop adverse events such as cardiac arrest. A statistically significant reduction in non–intensive care unit (ICU) cardiac arrests following the institution of these programs was reported in 2 studies; however, inpatient mortality remained unaffected.
1,2 The Joint Commission requires that hospitals in the United States and Canada develop systems that promptly recognize and respond to changes in a patient’s condition.
1,2 More than 95% of US hospitals have an RRT system in place.
3
Concern that house staff may not be equipped to treat severely ill patients emergently and efficiently has been expressed.
4 Morris et al
4 analyzed RRTs led by residents (n=870) and those led by attending physicians (n=534) in ICUs during a 4-year period.
5 They found that resident-led RRTs had similar outcomes to attending physician–led RRTs with regard to emergency ICU transfers and mortality.
4,5 However, nursing staff and respiratory therapy staff were reported to have more concerns during resident-led RRTs than intensivist-led RRTs.
6 In academic health care systems, appointing senior physicians to lead RRTs is criticized for reducing opportunities for residents to learn how to recognize and treat patients with complications. In an effort to alleviate staff concerns, one institution has implemented interprofessional training with nurses, respiratory therapists, and residents to align efforts among these clinicians to improve medical education and quality of care.
7
First-year residents function under the direct supervision of their senior residents.
8,9 Excellence at data gathering is highly valued in first-year residents, who are more often seen in the role of follower than leader. However, in the second year, residents are expected to assume leadership responsibilities and function under the indirect supervision of attending hospitalists, particularly when participating in RRTs. In addition to data gathering, sound medical decision making (MDM) becomes a key component of resident performance evaluations.
8,9 At Christiana Care Health System (CCHS), leading an RRT is one of the new responsibilities a resident may be called on to assume.
A 3-phase, team-based, interdisciplinary RRT training program was developed for first-year residents at CCHS. The program includes classroom-based didactic sessions conducted by senior residents, experiential training sessions, and resident-run RRT simulation. The aim of the current study was to evaluate the effectiveness of this RRT training program in educating and training first-year residents to improve global RRT performance as residents start their second year.
At CCHS, a community-based 1200-bed tertiary care center, RRTs are the responsibility of the residents in internal medicine and combined internal medicine programs. First-year residents enter the RRT training program in the last 6 months of their first year of residency. All first-year residents eligible to run RRTs as rising senior residents were assessed for inclusion. The 3 categories of RRT-eligible first-year residents are categorical internal medicine, emergency medicine/internal medicine, and internal medicine/pediatrics. Both osteopathic and allopathic residents were eligible. Preliminary and transitional first-year residents were excluded.
Weekly didactic lectures were conducted during a 1-month period by attending physicians and senior residents in internal medicine. The lectures reviewed basic RRT scenarios, triage, and in-hospital processes. Participants were evaluated in the form of a case-based, multiple-choice pretest and posttest using mean score percentage for comparison purposes. Additionally, a survey using a 5-point Likert scale of mean response percentage, where 1 indicated "strongly disagree" and 5 indicated "strongly agree", was used to compare confidence in running RRTs before and after the lectures. The percent difference between the survey scores before and after the lectures was then calculated.
For a minimum of 8 hours, the first-year residents were expected to hold a pager and respond to all RRT calls in the ICU. The residents were allowed to ask questions during the call and use the ICU nurse as a source of education. When the senior residents arrived, the first-year residents had the option of running the RRT with the supervision of the senior resident or observing the RRT. The participants were evaluated in the form of an experiential training confidence survey. A 7-item Likert survey was conducted at the end of phase 2, with responses ranging from "strongly disagree" to "strongly agree." "Strongly agree" and "agree" responses were tallied, and a percentage was calculated to indicate the percentage of positive responses. The seventh question required a qualitative response.
At the end of the year, before the participants were to complete their first year of residency, the third phase of the study was implemented. The participants were divided into 5 teams without regard for the specialty category of the residents. Medical mannequins were used to represent common cases encountered by RRTs. Five simulated RRT events were conducted for each team. Case 1 was supraventricular tachycardia; case 2, change in mental status; case 3, bradycardia; case 4, ventricular tachycardia; and case 5, hypertensive emergency with seizure. Within each team, the participants rotated through 3 different roles: RRT leader, ICU nurse, and floor nurse/respiratory therapist. Teams were evaluated as a group on how they performed in data gathering, team behaviors, and MDM. Two internal medicine attending physicians (P.A.B. and J.D.) and a third-year resident served as evaluators. A debriefing session was conducted at the conclusion of each simulation, with critical feedback provided by the evaluators. Team 1 did not perform case 5 owing to time constraints. Case 4 was identified as high pressure and high risk, with a decreased threshold for error and clinical events that if not managed appropriately would result in cardiac arrest. Therefore, case 4 was analyzed separately for each team with regard to the 3 measures as well as overall performance.
The mean score for each team on each case was calculated for data gathering, team behaviors, and MDM. Using the calculated mean scores for each case, each team’s overall mean score was calculated. Correlation coefficient analyses were then conducted to examine (1) MDM and data gathering and (2) MDM and team behaviors.
Descriptive statistics were used to calculate the mean multiple choice test scores before and after training and the mean survey scores on the surveys before and after phase 1 and after phase 2. Simple linear regression was used to calculate the correlation between MDM and data gathering and between MDM and team behaviors. Statistical analysis was performed with Microsoft Excel 2010 (Microsoft Corporation).
A total of 18 residents were included and assigned to 5 teams (team 1, n=4; team 2, n=3; team 3, n=4; team 4, n=3; and team 5, n=4). In a given simulation, 3 team members at a time participated.
The mean test score of the phase 1 didactic sessions was 50% before training and 70% after training, indicating a 20% improvement in performance. The mean survey score for confidence in running an RRT was 3.44 (69%) before training and 4.13 (83%) after training, demonstrating a 14% increase in confidence (
Figure 1).
The phase 2 experiential training confidence survey had a 100% response rate on the quantitative portion and a 33% response rate on the qualitative portion. The 6 questions used to evaluate program utility demonstrated a greater than 89.9% positive response rate when combining "agree" and "strongly agree" components (
Table 1). The qualitative responses were generally positive and indicated broad acceptance of the training program as a useful measure (
Figure 2). Some comments on potential improvements in logistics and increased exposure to more RRTs with supervision by senior residents were expressed. Overall, both quantitatively and qualitatively, the experiential training was viewed as positive by the participants.
The correlation between MDM and team behaviors in the lower-pressure scenario of case 1 was 0.613. In the higher-pressure scenario of case 4, it was 0.837 (
Table 2). The overall correlation between MDM and team behavior was 0.926, and between MDM and data gathering was 0.112. This finding indicates that there is a high correlation between MDM and team behaviors and less of a correlation between MDM and data gathering.
In a separate analysis of case 4, team 1 performed poorly on both MDM and team behaviors, with a team behavior score of 40% and an MDM score of 36%. Teams 2 and 4, on the other hand, performed well, with team behavior scores of 86% and 100% and MDM scores of 100% and 80%, respectively (
Figure 3). Additionally, data gathering for team 1 was 63%, which was disproportionate with the remainder of team 1’s scores. This finding highlights the high correlation between MDM and team behaviors, particularly in high-pressure RRT medical evaluations such as in case 4.
The current study highlights that MDM is enhanced through successful team behaviors and dynamics both qualitatively and quantitatively in rapid-response clinical scenarios. This contention is supported by the medical literature relevant to cardiac arrest teams. Adams et al
10 found that the use of a cardiac arrest team staffed during a 24-hour period led to faster response times and return of spontaneous circulation compared with other means of staff coverage. Campello et al
11 discovered that team perception may not always be consistent among all types of medical emergency team members. Standardization of practice and reduced individual variability, achieved through use of RRTs, are often felt to improve patient outcomes. Standardization and loss of individuality can have a profound negative effect on graduate medical education if residents feel that their individual contributions to patient care are limited, particularly if resident accountability is minimized.
11 One way to combat this negative effect is through simulation training.
In a study by O’Brein et al,
12 medical interns considered that they had improved their ability to effectively treat a patient in cardiac arrest after undergoing cardiac arrest simulation training. Furthermore, they expressed decreased levels of anxiety and improved confidence in their MDM.
12 A study by Hernandez et al
13 demonstrated that computer-based team behaviors (closed-loop communication) positively affected team behaviors during simulated RRTs.
In the current study, we found that resident-run simulated RRTs demonstrated a higher correlation between MDM and team behavior compared with MDM and data gathering. These findings seem at odds with traditional medical training given that excellence in data gathering has traditionally been seen as necessary for making sound medical decisions. However, a study by Allen et al
14 found that interprofessional relationships strongly affect rapid response systems. In RRT scenarios, particularly high-pressure scenarios, we believe that team dynamics have a direct impact on data gathering and medical knowledge recall. Thus, by effect this system allows for better MDM and may improve patient outcomes.
The nonmedical literature also describes team-based dynamics as a key facet for successful organizational performance. In his 1991 book,
Business Process Improvement,
15 James Harrington highlighted the means IBM President John Akers used to reduce cost and improve performance to enhance IBM’s profitability. The company demonstrated that more than 50% of total billing costs relate to preventing, catching, or fixing errors.
16 In response, IBM developed a system called
business process improvement, which uses team-based initiatives to do things right rather than do the right things. Thus, even if the aim of a given process was inaccurate, employees would learn to perform the process the right way as a team, which resulted in error reduction.
Given the positive feedback by the residents on the RRT training program, we elected to enact systems to enhance team dynamics. For example, interprofessional RRT training using an ICU nurse and respiratory therapist has become a mandatory requirement for residents. In addition, an intensivist now evaluates resident performance during live clinical RRTs. Other similar measures have been undertaken to enhance intern RRT training and education. We have sought to use a variety of disciplines (eg, ICU nurses, respiratory therapists, intensivists) to participate in the RRT training program, attempting to not only enhance resident performance and education, but also to encourage meaningful professional relationships.
There are limitations to the current study. The correlation between MDM and team behaviors is limited by a lack of multivariable analysis. It is unknown how team dynamics correlate with data gathering or how these variables interact with other potential confounders that predict performance. It is difficult to generalize the results from this single-institution study given the low power. Additionally, the low power could lead to bias. The phase 3 results were quantified using a checklist, which is subject to interobserver bias. Furthermore, recall bias can be observed in the participant survey results in phases 1 and 2 as well. Future studies with a larger sample size are needed to determine the effectiveness of interventions to improve team dynamics and patient outcomes.
Our data suggest that resident RRT training may improve global RRT performance and resident education through interdisciplinary team-based dynamics. An RRT training program may enhance data gathering and MDM, which may result in better patient outcomes during RRT scenarios. We suggest a team-based, interdisciplinary RRT simulation training program to enhance the education of residents.
All authors provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; Drs Burke and Vest drafted the article or revised it critically for important intellectual content; Drs Burke and Vest gave final approval of the version of the article to be published; and Drs Burke and Vest agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.