Free
Medical Education  |   April 2013
Ohio Osteopathic Residency Directors' Self-Reported Administrative Knowledge and Skills Before and After Participation in an Administrative Training Program
Author Notes
  • From the departments of academic affairs (Drs Sheehan and Brannan) and family medicine (Dr Sheehan) at the Ohio University Heritage College of Osteopathic Medicine/Centers for Osteopathic Research and Education in Athens 
  • Address correspondence to Olivia Ojano Sheehan, PhD, 245 Grosvenor Hall, Ohio University Heritage College of Osteopathic Medicine, Athens, OH 45701-2979. E-mail: ojano-sh@ohio.edu  
Article Information
Medical Education / Professional Issues / Graduate Medical Education
Medical Education   |   April 2013
Ohio Osteopathic Residency Directors' Self-Reported Administrative Knowledge and Skills Before and After Participation in an Administrative Training Program
The Journal of the American Osteopathic Association, April 2013, Vol. 113, 290-295. doi:10.7556/jaoa.2013.113.4.290
The Journal of the American Osteopathic Association, April 2013, Vol. 113, 290-295. doi:10.7556/jaoa.2013.113.4.290
Abstract

Context: Residency directors require myriad skills to perform their jobs efficiently. However, many residency directors receive no training prior to obtaining their positions.

Objective: To determine the effectiveness of the Residency Directors Residency Administration Program (RD RAP)—a 1-year fellowship training program for Ohio osteopathic residency directors sponsored by the Ohio University Heritage College of Osteopathic Medicine/Centers for Osteopathic Research and Education—by measuring the administrative knowledge and skills of Ohio osteopathic residency directors before and after completion of the program.

Methods: The authors administered a 54-item self-assessment instrument to RD RAP participants before and after the 2009-2010 and 2010-2011 programs. The assessment asked participants to rank their knowledge and skills in administration on a 5-point Likert scale, with lower values indicating higher knowledge and skills. We analyzed data from the pre- and postprogram assessments by using the Wilcoxon signed rank nonparametric test. The 54 assessment items were categorized into 10 content domains.

Results: Ten RD RAP participants completed the assessments. Median scores were statistically significantly lower for each of the 10 content domains after the RD RAP program. The content domain with the greatest change between pre- and postprogram assessment Likert scale scores was Legal Issues in Residency Training, with a median change of 1.7 (P=.007). Role of Program Directors, Personality, and Professional Development had the smallest change in pre- and postprogram assessment Likert scores, with a median change of 0.8 (P=.011).

Conclusion: Statistically significant improvements were found in the osteopathic residency directors' self-reported administrative knowledge and skills after participation in the RD RAP.

Osteopathic residency directors, also referred to as program directors, have faculty status within an Osteopathic Postdoctoral Training Institution, or OPTI, such as the Centers for Osteopathic Research and Education (CORE) in Athens, Ohio. An OPTI is a consortium of osteopathic medical colleges and hospitals that provide clinical training to residents and medical students. In the case of the CORE, an osteopathic residency director is awarded faculty status by Ohio University Heritage College of Osteopathic Medicine, the primary sponsoring medical school. 
As faculty, residency directors belong in the non-tenure track classification. These faculty members have either full- or part-time appointments at university- or community-based programs and work with students and residents. Their primary tasks lie in instruction, patient care, and administration, and their administrative responsibilities may include setting up, monitoring, and evaluating residencies, as well as coordinating clinic operations.1 The role of residency director requires a delicate balancing act that includes being a physician, a teacher, an evaluator, a manager, an administrator, and a coach. To be effective in their roles, residency directors need strong managerial and human resources skills, as well as an understanding of accreditation and legal issues. 
With these required skills and responsibilities, it is no wonder that residency directors stay in their positions only 6 to 7 years, on average, before seeking better oppportunities.2 A study involving all nonmilitary internal medicine residency program directors3 indicated that 49% of the participants had been residency directors for 3 years or less at the start of the study, and 29% were no longer in that role 3 years later. The reasons for high turnover among internal medicine residency directors include overwhelming administrative duties and time demands, difficult colleague relationships, inadequate resources, lack of recognition, and lack of preparedness.4 Much of a program director's expertise is gained from on-the-job training, and many program directors are promoted into the role without the opportunity for focused preparation, training, or gradual development.4 
Many medical schools have developed training programs to prepare faculty for academic careers in medical education,5-7 but these programs have not catered specifically to residency directors. Programs targeting residency directors, such as postgraduate training for program directors in family medicine, faced many challenges when the programs did not include substantial training in finance and administration.8 
In the osteopathic medical profession, the Costin program at the Costin Institute for Osteopathic Medical Educators at Midwestern University/Chicago College of Osteopathic Medicine9 is an example of a certification program designed to train osteopathic physicians to become more effective medical educators. The target audience is academic and clinical faculty, including chief academic officers, directors of medical education, academic administrators, residency program directors, and pre- and postdoctoral faculty. The Costin program offers the following content cluster areas: Navigating the Academic World, The Physician Teacher and Professional Education, Management Issues, Assessment and Evaluation Methods, Innovative and Effective Methods of Teaching, The Educational Practitioner, and Scholarly Activities. The Costin program also offers areas of concentration: academic administration (deans and department chairs), chief learning officer or director of medical education, and residency program director.9 
Within CORE, we conducted an internal needs assessment in 2004 to determine whether Ohio osteopathic residency directors needed a training program focused on administrative knowledge and skills.10 The survey was e-mailed to approximately 47 program directors and informally shared with 12 directors of medical education. Thirty participants completed the survey, for a response rate of approximately 50%. Although the return rate was not as high as anticipated, results indicated that Ohio osteopathic residency directors would benefit from an administrative training program. 
In response to the findings of the 2004 survey, we implemented the Residency Directors Residency Administration Program (RD RAP) in 2009, with the goal of giving Ohio osteopathic residency directors an opportunity to strengthen their knowledge and skills pertaining specifically to graduate medical education administration. The RD RAP curriculum includes the goals for the administrative domain proposed by Bland and colleagues.1 The major responsibilities for non-tenure track faculty in this domain include understanding how environmental pressures affect academic medical centers; understanding the formal structures of and relationships between the organizations they serve; participating in and providing leadership for academic tasks in small and large groups; and managing oneself, others, money, and time for various projects and programs. 
The RD RAP has a blended curriculum that includes a hybrid of live and online sessions. Two live meetings, held in central Ohio, each consisted of 1½ days of lecture, discussion, and small-group activities. The online modules (offered by means of Blackboard, a Web-based learning management system) required the participants to complete learning activities, assignments, and quizzes, as well as actively participate in written discussion forums. All sessions were facilitated by faculty development professionals, clinicians, and medical educators. The curriculum included the following content domains: Role of Program Directors, Personality, and Professional Development; Leading in a Sea of Change; Understanding the Millennial Residents; Selecting Residents That Fit the Program; Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan; Legal Issues in Residency Training; Teaching Role of the Residency Director; Managing Time, Meetings, and Conflict; The Art of Delegation and Negotiation; and Mentoring/Coaching. The objective of our study was to determine the effectiveness of the RD RAP by evaluating whether Ohio osteopathic residency directors' administrative knowledge and skills improved substantially after the program; this was accomplished by comparing results of a self-assessment instrument given to residency directors at the beginning and end of the program. 
Methods
We were granted an exemption by the institutional review board at Ohio University for this study. The study participants were Ohio osteopathic residency directors enrolled in the year-long RD RAP in either 2009-2010 or 2010-2011. Inclusion criteria were graduation from the program and completion of pre- and postprogram assessments. The surveys contained no patient identifiers. 
We developed the assessment instrument by reviewing the literature and consulting with faculty development professionals, clinical faculty, and medical educators about the appropriateness of the instrument's statements in relation to the learning objectives and the content domains. The self-assessment instrument measured osteopathic residency directors' knowledge and skills pertaining to administration. We grouped the 54 items according to the 10 content domains (Table 1). For example, the item “I know of environmental pressures and trends that affect residency training” was included under the domain “Leading in a Sea of Change,” and the item “I know of possible areas of litigation in which an institution associated with residency training can become involved” was included under the domain “Legal Issues in Residency Training.” 
Table 1.
Administrative Training Program for Residency Directors: Self-Assessment Instrument Content Domains and Number of Items
Content Domains No. of Items
Role of Program Directors, Personality, and Professional Development 3
Leading in a Sea of Change 7
Understanding the Millennial Residents 2
Selecting Residents That Fit the Program 5
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 6
Legal Issues in Residency Training 6
Teaching Role of the Residency Director 6
Managing Time, Meetings, and Conflict 5
The Art of Delegation and Negotiation 6
Mentoring/Coaching 8
Table 1.
Administrative Training Program for Residency Directors: Self-Assessment Instrument Content Domains and Number of Items
Content Domains No. of Items
Role of Program Directors, Personality, and Professional Development 3
Leading in a Sea of Change 7
Understanding the Millennial Residents 2
Selecting Residents That Fit the Program 5
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 6
Legal Issues in Residency Training 6
Teaching Role of the Residency Director 6
Managing Time, Meetings, and Conflict 5
The Art of Delegation and Negotiation 6
Mentoring/Coaching 8
×
To score the items, we used a 5-point Likert scale, with responses defined as follows: 1, strongly agree; 2, agree; 3, neutral; 4, disagree; and 5 strongly disagree. By definition for this scale, lower median values indicate higher knowledge and skills in administration, and higher values indicate lower knowledge and skills. 
We collected data by administering the 54-item self-assessment instrument to participants before and after the RD RAP using SurveyMonkey, an online survey tool. To obtain consent, we asked participants whether they agreed to their data being used for research purposes. We made the pre- and postprogram assessment instruments available 1 month before the first session and 1 month after the last session, respectively. 
We developed 2 hypotheses to guide the study. The null hypothesis was that there is no statistically significant difference between pre- and postprogram assessments in the osteopathic program directors' self-assessment of administrative knowledge and skills. The alternative hypothesis was that there is a statistically significant difference between pre- and postprogram assessments. 
To aid and simplify data interpretation, we analyzed each of the 10 content domains instead of individual items. Cronbach α, an internal consistency reliability index, was determined for all items in each of the domains for pre- and postprogram assessments. The items in each content domain were summed and averaged. Because of the small sample size, we analyzed the data by using the Wilcoxon signed rank nonparametric test, because it allows for a more conservative assessment. Differences were considered statistically significant at P≤.05. 
Results
Nineteen Ohio osteopathic residency directors enrolled in the year-long RD RAP in either 2009-2010 (n=13) or 2010-2011 (n=6). Of the 19 directors enrolled, 15 graduated from the program, and 10 of the 15 completed the assessment instruments both before and after the program. Participants included 9 male residency directors and 1 female residency director. Two residency directors were in the specialty of family medicine, 4 in emergency medicine, and 1 each in obstetrics and gynecology, general surgery, orthopedics, and otolaryngology. 
The 54 items scored on the 5-point Likert scale had preprogram assessment and postprogram assessment item internal reliabilities of 0.95 and 0.965, respectively. Table 2 shows the results of the Wilcoxon signed rank nonparametric test for the pre- and postprogram assessments in the 10 content domains. All content domains and the overall content measure showed significantly lower scores (P>.05) for the postprogram assessment than for the preprogram assessment. The content domain with the greatest change in pre- and postprogram assessment Likert scale scores was Legal Issues in Residency Training, with a median change of 1.7 (P=.007). Role of Program Directors, Personality, and Professional Development had the smallest change in pre- and postprogram assessment Likert scores, with a median change of 0.8 (P=.011). 
Table 2.
Administrative Training Program for Residency Directors: Survey Results by Self-Assessment Content Domains and Percentilea
Likert Scale Score by Percentileb
25th 50th (Median) 75th
Content Domains Pre Post Pre Post Pre Post Z Valuec P Value
Role of Program Directors, Personality, and Professional Development 2.0 1.0 2.0 1.2 2.3 2.0 −2.555 .011
Leading in a Sea of Change 2.1 1.3 2.7 1.7 3.3 2.0 −2.812 .005
Understanding the Millennial Residents 2.0 1.0 2.5 1.0 4.0 1.6 −2.829 .005
Selecting Residents That Fit the Program 2.2 1.0 2.5 1.2 3.0 1.6 −2.809 .005
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 2.5 1.0 2.9 1.8 4.0 1.9 −2.807 .005
Legal Issues in Residency Training 2.3 1.0 2.8 1.1 3.4 1.6 −2.677 .007
Teaching Role of the Residency Director 1.6 1.0 2.3 1.3 2.9 2.0 −2.312 .021
Managing Time, Meetings, and Conflict 2.0 1.0 2.4 1.1 2.9 1.5 −2.829 .005
The Art of Delegation and Negotiation 2.3 1.0 2.8 1.7 3.2 2.6 −2.199 .028
Mentoring/Coaching 2.4 1.1 2.8 1.5 3.0 2.0 −2.673 .008
Overall 2.4 1.1 2.5 1.5 3.0 1.8 −2.666 .008
  a Wilcoxon signed rank nonparametric test used for analyses.
  b A 5-point Likert scale was used to score the items, with scores defined by responses as follows: 1, strongly agree; 2, agree; 3, neutral; 4, disagree; and 5, strongly disagree. By definition for this scale, lower median values indicate higher knowledge and skills in administration, and higher values indicate lower knowledge and skills.
  c Based on positive ranks.
  Abbreviations: Pre, preprogram assessment; post, postprogram assessment.
Table 2.
Administrative Training Program for Residency Directors: Survey Results by Self-Assessment Content Domains and Percentilea
Likert Scale Score by Percentileb
25th 50th (Median) 75th
Content Domains Pre Post Pre Post Pre Post Z Valuec P Value
Role of Program Directors, Personality, and Professional Development 2.0 1.0 2.0 1.2 2.3 2.0 −2.555 .011
Leading in a Sea of Change 2.1 1.3 2.7 1.7 3.3 2.0 −2.812 .005
Understanding the Millennial Residents 2.0 1.0 2.5 1.0 4.0 1.6 −2.829 .005
Selecting Residents That Fit the Program 2.2 1.0 2.5 1.2 3.0 1.6 −2.809 .005
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 2.5 1.0 2.9 1.8 4.0 1.9 −2.807 .005
Legal Issues in Residency Training 2.3 1.0 2.8 1.1 3.4 1.6 −2.677 .007
Teaching Role of the Residency Director 1.6 1.0 2.3 1.3 2.9 2.0 −2.312 .021
Managing Time, Meetings, and Conflict 2.0 1.0 2.4 1.1 2.9 1.5 −2.829 .005
The Art of Delegation and Negotiation 2.3 1.0 2.8 1.7 3.2 2.6 −2.199 .028
Mentoring/Coaching 2.4 1.1 2.8 1.5 3.0 2.0 −2.673 .008
Overall 2.4 1.1 2.5 1.5 3.0 1.8 −2.666 .008
  a Wilcoxon signed rank nonparametric test used for analyses.
  b A 5-point Likert scale was used to score the items, with scores defined by responses as follows: 1, strongly agree; 2, agree; 3, neutral; 4, disagree; and 5, strongly disagree. By definition for this scale, lower median values indicate higher knowledge and skills in administration, and higher values indicate lower knowledge and skills.
  c Based on positive ranks.
  Abbreviations: Pre, preprogram assessment; post, postprogram assessment.
×
Comment
To indicate improvement of osteopathic residency directors' administrative knowledge and skills after participation in the RD RAP, we expected the postprogram assessment median to be lower than the preprogram assessment median. Using the Wilcoxon signed rank nonparametric test, we found statistically significant differences in median scores between pre- and postprogram assessments in all content domains. The preprogram scores were closer to 2 (agree), and the postprogram scores were closer to 1 (strongly agree), an indication of increased understanding and confidence in the different areas after the RD RAP. 
The statistically significant differences in median scores between the pre- and postprogram assessments suggest that the osteopathic residency directors' knowledge and skills in administration were improved after they participated in the RD RAP. These findings are consistent with those of the study conducted by Pugno et al11 on the National Institute for Program Director Development for family medicine program directors. Pugno et al concluded that “enhanced preparation for the job of residency program director results in a positive impact on both the director and the program.”11(p209) The Fellowship in Graduate Medical Education, a professional training program for program directors at Stanford Hospital and Clinics that is similar to the RD RAP, was found to offer benefits to both the individual program directors and the Department of Graduate Medical Education in general.12 Information that RD RAP participants obtained from the program faculty, the assigned readings, and each other all helped improve participants' knowledge and skills in administration. 
Although our results are statistically significant, it is important to acknowledge some of the limitations of our study. We had a small sample size, so our findings cannot be generalized to all residency directors. Furthermore, some participants did not complete the program and did not have both pre- and postprogram data, and it was therefore impossible to compare their preprogram data with those of the participants who did complete the program. This limitation is a potential source of selection bias. In addition, we based data on self-reported information (self-assessment), which provided only a single perspective. To gain a more general and robust understanding of the training program's effects and to obtain other perspectives, we recommend that future studies include additional data from other constituencies, such as directors of medical education, residency director colleagues, and residents. 
Conclusion
We found statistically significant gains in RD RAP participants' self-reported pre- and postprogram administrative knowledge and skills, indicating that their participation in the RD RAP training program was beneficial. 
Acknowledgment
We extend our sincerest gratitude to Godwin Dogbey, PhD, Ohio University Heritage College of Osteopathic Medicine/CORE statistician, for his assistance in data analysis. 
   Financial Disclosures: None reported.
 
References
Bland CJ, Schmitz CC, Stritter FT, Henry RC, Aluise JJ. Successful Faculty in Academic Medicine. New York, NY: Springer; 1997.
Neher JO. The unofficial guide to residency director selection. Fam Med. 2006;38(8):577-578. [PubMed]
Beasley BW, Kern DE, Kolodner K. Job turnover and its correlates among residency program directors in internal medicine: a three-year cohort study. Acad Med. 2001;76(11):1127-1135. [CrossRef] [PubMed]
Kertesz J, Pendergast WJ, Holder-Cooper JC. Staying in the program director role. In: Andolsek K, Cefalo R, eds. LIFE Curriculum. Duram, NC: Duke University School of Medicine; 2006:66.
Lown BA, Newman LR, Hatem CJ. The personal and professional impact of a fellowship in medical education. Acad Med. 2009;84(4):1089-1097. [CrossRef] [PubMed]
Steinert Y, Nasmith L, McLeod PJ, Conochie L. A teaching scholars program to develop leaders in medical education. Acad Med. 2003;78(2):142-149. [CrossRef] [PubMed]
Gruppen LD, Frohna AZ, Anderson RM, Lowe KD. Faculty development for educational leadership and scholarship. Acad Med. 2003;78(2):137-141. [CrossRef] [PubMed]
Palmer EJ, Tippy PK, Bope ETet al. National Institute for Program Director Development (NIPDD): a collaborative pursuit of excellence. Ann Fam Med. 2008;6(4):374-375. [CrossRef] [PubMed]
Costin program. Costin Institute for Osteopathic Medical Educators website. http://www.costininstitute.org/costin-program. Accessed October 16, 2012.
Ohio University Heritage College of Osteopathic Medicine Centers for Osteopathic Research and Education. Residency Directors Needs Assessment Survey. Athens: Ohio University; 2004.
Pugno PA, Dornfest FD, Kahn NB, Avant R. The National Institute for Program Director Development: a school for program directors. J Am Board Fam Pract. 2002;15(3):209-213. [PubMed]
Edler A, Dohn A, Davidson H, Grewal D, Behravesh B, Piro N. Building faculty community: fellowship in graduate medical education administration. J Grad Med Educ. 2009;1(1):146-149. [CrossRef] [PubMed]
Table 1.
Administrative Training Program for Residency Directors: Self-Assessment Instrument Content Domains and Number of Items
Content Domains No. of Items
Role of Program Directors, Personality, and Professional Development 3
Leading in a Sea of Change 7
Understanding the Millennial Residents 2
Selecting Residents That Fit the Program 5
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 6
Legal Issues in Residency Training 6
Teaching Role of the Residency Director 6
Managing Time, Meetings, and Conflict 5
The Art of Delegation and Negotiation 6
Mentoring/Coaching 8
Table 1.
Administrative Training Program for Residency Directors: Self-Assessment Instrument Content Domains and Number of Items
Content Domains No. of Items
Role of Program Directors, Personality, and Professional Development 3
Leading in a Sea of Change 7
Understanding the Millennial Residents 2
Selecting Residents That Fit the Program 5
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 6
Legal Issues in Residency Training 6
Teaching Role of the Residency Director 6
Managing Time, Meetings, and Conflict 5
The Art of Delegation and Negotiation 6
Mentoring/Coaching 8
×
Table 2.
Administrative Training Program for Residency Directors: Survey Results by Self-Assessment Content Domains and Percentilea
Likert Scale Score by Percentileb
25th 50th (Median) 75th
Content Domains Pre Post Pre Post Pre Post Z Valuec P Value
Role of Program Directors, Personality, and Professional Development 2.0 1.0 2.0 1.2 2.3 2.0 −2.555 .011
Leading in a Sea of Change 2.1 1.3 2.7 1.7 3.3 2.0 −2.812 .005
Understanding the Millennial Residents 2.0 1.0 2.5 1.0 4.0 1.6 −2.829 .005
Selecting Residents That Fit the Program 2.2 1.0 2.5 1.2 3.0 1.6 −2.809 .005
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 2.5 1.0 2.9 1.8 4.0 1.9 −2.807 .005
Legal Issues in Residency Training 2.3 1.0 2.8 1.1 3.4 1.6 −2.677 .007
Teaching Role of the Residency Director 1.6 1.0 2.3 1.3 2.9 2.0 −2.312 .021
Managing Time, Meetings, and Conflict 2.0 1.0 2.4 1.1 2.9 1.5 −2.829 .005
The Art of Delegation and Negotiation 2.3 1.0 2.8 1.7 3.2 2.6 −2.199 .028
Mentoring/Coaching 2.4 1.1 2.8 1.5 3.0 2.0 −2.673 .008
Overall 2.4 1.1 2.5 1.5 3.0 1.8 −2.666 .008
  a Wilcoxon signed rank nonparametric test used for analyses.
  b A 5-point Likert scale was used to score the items, with scores defined by responses as follows: 1, strongly agree; 2, agree; 3, neutral; 4, disagree; and 5, strongly disagree. By definition for this scale, lower median values indicate higher knowledge and skills in administration, and higher values indicate lower knowledge and skills.
  c Based on positive ranks.
  Abbreviations: Pre, preprogram assessment; post, postprogram assessment.
Table 2.
Administrative Training Program for Residency Directors: Survey Results by Self-Assessment Content Domains and Percentilea
Likert Scale Score by Percentileb
25th 50th (Median) 75th
Content Domains Pre Post Pre Post Pre Post Z Valuec P Value
Role of Program Directors, Personality, and Professional Development 2.0 1.0 2.0 1.2 2.3 2.0 −2.555 .011
Leading in a Sea of Change 2.1 1.3 2.7 1.7 3.3 2.0 −2.812 .005
Understanding the Millennial Residents 2.0 1.0 2.5 1.0 4.0 1.6 −2.829 .005
Selecting Residents That Fit the Program 2.2 1.0 2.5 1.2 3.0 1.6 −2.809 .005
Preparing for Program Internal Review, Program Inspection, and Writing a Corrective Action Plan 2.5 1.0 2.9 1.8 4.0 1.9 −2.807 .005
Legal Issues in Residency Training 2.3 1.0 2.8 1.1 3.4 1.6 −2.677 .007
Teaching Role of the Residency Director 1.6 1.0 2.3 1.3 2.9 2.0 −2.312 .021
Managing Time, Meetings, and Conflict 2.0 1.0 2.4 1.1 2.9 1.5 −2.829 .005
The Art of Delegation and Negotiation 2.3 1.0 2.8 1.7 3.2 2.6 −2.199 .028
Mentoring/Coaching 2.4 1.1 2.8 1.5 3.0 2.0 −2.673 .008
Overall 2.4 1.1 2.5 1.5 3.0 1.8 −2.666 .008
  a Wilcoxon signed rank nonparametric test used for analyses.
  b A 5-point Likert scale was used to score the items, with scores defined by responses as follows: 1, strongly agree; 2, agree; 3, neutral; 4, disagree; and 5, strongly disagree. By definition for this scale, lower median values indicate higher knowledge and skills in administration, and higher values indicate lower knowledge and skills.
  c Based on positive ranks.
  Abbreviations: Pre, preprogram assessment; post, postprogram assessment.
×