Free
Brief Report  |   July 2019
Conceptual Framework to Evaluate Health Care Professionals’ Satisfaction in Utilizing Telemedicine
Author Notes
  • From the College of Osteopathic Medicine (Dr Law) and the Department of Social and Public Health at the College of Health Sciences and Professions (Drs Cronin, Schuller, Jing, Bolon, and Phillips) at Ohio University in Athens. Drs Law, Cronin, Schuller, and Jing contributed equally to the manuscript and thus share the first authorship. 
  • Financial Disclosures: None reported. 
  • Support: None reported. 
  •  *Address correspondence to Xia Jing, MD, PhD, Department of Social and Public Health, 1 Ohio University, Athens, OH 45701-2942. Email: jingx@ohio.edu
     
Article Information
Professional Issues
Brief Report   |   July 2019
Conceptual Framework to Evaluate Health Care Professionals’ Satisfaction in Utilizing Telemedicine
The Journal of the American Osteopathic Association, July 2019, Vol. 119, 435-445. doi:https://doi.org/10.7556/jaoa.2019.080
The Journal of the American Osteopathic Association, July 2019, Vol. 119, 435-445. doi:https://doi.org/10.7556/jaoa.2019.080
Web of Science® Times Cited: 1
Abstract

Context: Several advantages of telemedicine have been reported, such as improved patient satisfaction, reduced costs, and broader access to specialists. However, most studies have focused on patient satisfaction with telemedicine services. A literature gap exists in evaluating health care professionals’ (both referring and performing professionals) satisfaction in delivering telemedicine services. A clear picture of professionals' telemedicine satisfaction has the potential to improve care quality, care accessibility, and telemedicine adoption and to reduce overall health care costs.

Objectives: To develop a framework to evaluate health care professionals’ satisfaction in delivering telemedicine services and to articulate the factors that can contribute to their satisfaction.

Methods: We developed an initial conceptual framework based on existing literature and the research team's expertise. The initial framework was validated by a convenience group of 12 osteopathic and allopathic physicians, and the framework was further revised and refined according to their input.

Results: The refined framework includes 5 main components (professionals’ demographics, care settings, motivations, experiences, and overall satisfaction) that can contribute to health care professionals' satisfaction or dissatisfaction in delivering telemedicine services. Detailed itemization is provided under each component, including hierarchical organizations of the items, definitions, and potential value sets.

Conclusions: The refined conceptual framework may provide a comprehensive evaluation reference for measuring professionals’ satisfaction in delivering telemedicine. Further customization according to the specific setting of telemedicine services is needed. This refined conceptual framework can also be used to identify areas that can be improved and to ultimately improve telemedicine services and care quality.

Telemedicine is an evolving health care delivery mechanism. The term telemedicine is used to define “the provision of medical services across distance,”1 “healing at a distance,”2 or “the use of technologies to remotely diagnose, monitor, and treat patients,”2 while the term telehealth is broader in scope and incorporates health education,1,2 online discussion boards and support groups, and internet-based health communications.2 In this article, we use telemedicine to define the provision of health care services from a care professional to a patient at a distance (though telehealth is retained when discussing articles that used that term).2 
In 1993, the first survey of telemedicine programs identified 12 active programs across the United States and Canada, mostly consisting of interactive video. By 1998, hundreds of programs were classified as telemedicine,3 and as of 2013, more than half of hospitals utilized telehealth,1 with more hospitals beginning to implement services.4 Health care policy has increasingly taken note of telemedicine, resulting in more than 200 telehealth-related state bills introduced in 2015 alone.5 
Although there is some hesitation in the adoption of telemedicine from health care professionals and patients, telemedicine has numerous potential and perceived benefits. Benefits for the patient include less travel time, less travel burden, less missed work or school,2 reduced costs, improved quality of care, reduced number of missed appointments,2,6 and increased access to professionals,7 especially specialty professionals for patients in rural areas.8-12 For patients with chronic conditions, telemedicine has been found to promote medication adherence, thus reducing readmissions and decreasing morbidity and mortality,2 improving access to follow-up care, and showing high patient satisfaction.11,13-16 For patients in rural areas, telemedicine can provide patients with access to care that otherwise may not be available to them.17-20 Among the elderly population, telemedicine programs offered through home health allow easy access to care in the convenience of their own homes.21,22 
Telemedicine has been increasingly used within osteopathic medicine23 and medical education.24 Professionals who refer patients to telemedicine services and who perform telemedicine services and the patients who receive these services are at the center of telemedicine delivery. Health care professionals' satisfaction is as important as patient satisfaction in telemedicine delivery. Settings such as a maternity hospital,25 a cardiology hospital,26 intensive care units (ICUs),27,28 and primary care practices7,29 have found positive telemedicine satisfaction among health care professionals. However, several studies have reported either no significant changes in both perceived and real job satisfaction by delivering telemedicine services30,31 or mixed results.32 
Further research is required to understand how health care professionals perceive the increasingly prevalent telemedicine programs. For example, Whitten and Love33 noted that many of the studies they assessed were hampered by small sample sizes, limited durations, or a lack of an established survey instrument. Such research will be most effective, as noted by Whitten and Love,4 with a rigorously tested instrument built on an established understanding of satisfaction and constructed to comprehensively incorporate the many factors of telemedicine. The purpose of the present article was to devise a conceptual framework for measuring health care professionals' satisfaction comprehensively and systematically. The long-term goal is to identify the possible gaps in telemedicine services; these gaps can then be addressed to improve quality and efficiency of care delivered via telemedicine. 
Methods
The development of this framework occurred over 3 phases: (1) initial framework development, (2) validation, and (3) revision. Development began at Ohio University in 2016. The interdisciplinary team was made up of individuals with expertise and experience in primary care delivery (including a physician who has provided telemedicine services to patients for more than 20 years), health informatics, health care policy, and health services administration. Because of the conceptual nature of this study, there was no institutional review board involvement. 
Initial Framework Development
To initially explore, define, and understand the problem and to inform the development of the initial framework, we performed a comprehensive literature review. We searched PubMed in October 2016 using “telemedicine” and “personal satisfaction” as MeSH terms with no other inclusion or exclusion criteria; 20 articles were returned, of which 2 were related to health care professionals’ satisfaction. These initial literature search results revealed a gap in the literature. In June 2018, we expanded the search strategy to include “telemedicine” and “job satisfaction” as major MeSH terms. Thirteen additional articles were identified, with 8 focusing on health care professionals' satisfaction. We also included 8 additional related articles that the authors had previously noticed, read, and saved for future reference. 
After identifying research relevant to current professionals’ satisfaction in telemedicine, we developed the initial conceptual framework. The development process included several iterative in-person discussions, revisions, and referrals to literature to achieve consensus among the research team. 
Validation and Revisions
We sought feedback from a convenience group of 12 physicians, both DOs and MDs, to validate our initial framework. We also compared our refined framework with the National Quality Forum's report Creating a Framework to Support Measure Development for Telehealth34 and that report's identified domain of Clinician Experience as an additional validation tool. We then revised and refined the initial framework again to incorporate all of the feedback, which resulted in the proposed conceptual framework. 
Results
Refined Conceptual Framework
Overall, the 12 physicians agreed that the initial framework we proposed was valid and comprehensive in measuring health care professionals’ satisfaction with telemedicine. The physicians did not comment on the organization and main components of the framework. Most of their feedback referenced the evaluation items (such as frequency of telemedicine utilization) and more detailed potential value sets (eg, specify zip code for location). 
The Figure represents the refined framework for measuring health care professionals' satisfaction with telemedicine. We identified 5 components that influenced health care professionals’ perception of satisfaction as it relates to telemedicine, as follows: (1) demographics (eg, professional's education/training), (2) care settings (eg, rural, urban), (3) experience (eg, in terms of technology), (4) motivation (eg, patient cost, time), and (5) overall satisfaction (which is influenced by each of the previous 4 components). 
Figure.
The main components and examples of relevant items that contribute to health care professionals’ satisfaction in telemedicine care delivery in the refined conceptual framework.
Figure.
The main components and examples of relevant items that contribute to health care professionals’ satisfaction in telemedicine care delivery in the refined conceptual framework.
Compared with the National Quality Forum (NQF) report,34 our framework covered all of the Clinician Experience–related concepts to telemedicine except “ability to obtain actionable information.” Our framework covered “Access to Care” and “Cost,” which are both covered in the NQF report,34 and provided more comprehensive coverage on other aspects of health care professionals’ satisfaction with telemedicine. For example, our framework covered detailed information about professionals’ motivations, care settings, and demographic data. 
Demographics
This component of the refined framework included personal factors that may influence a health care professional's utilization of telemedicine. Demographics considered characteristics such as age, gender, education/training, whether they live in an urban or rural area, specialty, years in practice, prior utilization of telemedicine, role as a referring or performing professional, and self-perceived technological abilities (Table 1). 
Table 1.
Health Care Professionals’ Individual Characteristics That May Influence Their Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
Demographics of Professional Potential Value Sets and Annotation
Biological Features
 Age, y ≤35; 36-45; 46-55; 56-65; >65
 Gender35 Female; male; nondisclosed
 Race29 White; African American; American Indian or Alaska Native; Native Hawaiian or Other Pacific Islander; Asian; multiracial
 Ethnicity Hispanic or Latino; not Hispanic or Latino
Acquired Features
 Education/training35 Medical student; resident; licensed physician (DO or MD); nonphysician: associate degree, bachelor's degree, master's degree, PhD
 Additional license or credentials to practice telemedicine Yes (please specify); no; not sure
 Professional title29 Physician; nurse practitioner; physician assistant; other (please specify)
 Location of residence35 Rural; suburban; urban; please specify state; please specify zip code
 Specialty29 Family medicine; urgent care; occupational medicine; internal medicine; pediatrics; obstetric-gynecologists; behavioral health; specialists (please specify)
 Practice experience, y29 ≤5; 6-10; 11-20; 21-30; >30
 Practice experience with telemedicine, y36 ≤5; 6-10; 11-15; 16-20; >20
 Frequency of telemedicine utilization Daily (specify how many times); weekly (specify how many times); monthly (specify how many times)
 Role during telemedicine services Provides telemedicine services; receives telemedicine services; both
 Experience with telemedicine Current telemedicine user; previous telemedicine user; no practice with telemedicine but with patients who utilize telemedicine via other providers; other (please specify)
 Attitude toward telemedicine Anti-telemedicine; naive telemedicine; telemedicine fan; neutral; hesitation; prohibiting; supporting; infringing; other (please specify)
 Technological competency Beginner (eg, seek help in solving IT issues); middle level (eg, able to solve one's own IT issues); advanced (eg, tech savvy and able to help others solve IT issues)
 Technology adoption type Innovator; early adopter; early majority adopter; late majority; adopter; laggard
 General confidence level in ability to provide quality medical care37 High; medium; low

Abbreviation: IT, information technology.

Table 1.
Health Care Professionals’ Individual Characteristics That May Influence Their Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
Demographics of Professional Potential Value Sets and Annotation
Biological Features
 Age, y ≤35; 36-45; 46-55; 56-65; >65
 Gender35 Female; male; nondisclosed
 Race29 White; African American; American Indian or Alaska Native; Native Hawaiian or Other Pacific Islander; Asian; multiracial
 Ethnicity Hispanic or Latino; not Hispanic or Latino
Acquired Features
 Education/training35 Medical student; resident; licensed physician (DO or MD); nonphysician: associate degree, bachelor's degree, master's degree, PhD
 Additional license or credentials to practice telemedicine Yes (please specify); no; not sure
 Professional title29 Physician; nurse practitioner; physician assistant; other (please specify)
 Location of residence35 Rural; suburban; urban; please specify state; please specify zip code
 Specialty29 Family medicine; urgent care; occupational medicine; internal medicine; pediatrics; obstetric-gynecologists; behavioral health; specialists (please specify)
 Practice experience, y29 ≤5; 6-10; 11-20; 21-30; >30
 Practice experience with telemedicine, y36 ≤5; 6-10; 11-15; 16-20; >20
 Frequency of telemedicine utilization Daily (specify how many times); weekly (specify how many times); monthly (specify how many times)
 Role during telemedicine services Provides telemedicine services; receives telemedicine services; both
 Experience with telemedicine Current telemedicine user; previous telemedicine user; no practice with telemedicine but with patients who utilize telemedicine via other providers; other (please specify)
 Attitude toward telemedicine Anti-telemedicine; naive telemedicine; telemedicine fan; neutral; hesitation; prohibiting; supporting; infringing; other (please specify)
 Technological competency Beginner (eg, seek help in solving IT issues); middle level (eg, able to solve one's own IT issues); advanced (eg, tech savvy and able to help others solve IT issues)
 Technology adoption type Innovator; early adopter; early majority adopter; late majority; adopter; laggard
 General confidence level in ability to provide quality medical care37 High; medium; low

Abbreviation: IT, information technology.

×
Care Settings
The next component of the framework examined was the setting in which the care was provided and who received the care. By assessing setting, this conceptual model may not only help predict those with the need and desire to use telemedicine but also identify whom to target for further education on the benefits and ramifications of the technology. This part of the framework included measurements of type of practice setting and population served, rurality (urban, suburban, or rural), type of services provided, and type and amount of technical support available (Table 2). The care settings have been identified as one of the confounding factors in the adoption of electronic medical records and telemedicine in some other studies.36,38 
Table 2.
Care Settings That May Influence Health Care Professionals’ Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
  Image not available

a Response options overlap; more than 1 can be selected.

Table 2.
Care Settings That May Influence Health Care Professionals’ Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
  Image not available

a Response options overlap; more than 1 can be selected.

×
Experience
The next component of the framework pertained to the professional's past experience with technology, which might be one of the possible confounders influencing their utilization (or lack thereof) of telemedicine. Access to software, hardware, and technical infrastructure were some of the basic elements in this component of the framework. The access to technical support staff either in the office or by phone (whether locally or overseas) were delineated, as well as how it related to workflow at the point of care (Table 3). 
Table 3.
Health Care Professionals’ Technological Experience With Telemedicine, as Proposed in the Conceptual Framework
Technical Experience With Telemedicine Potential Value Sets and Annotation
General expectations of telemedicine29 Please specify
Telemedicine training experience Sufficient or not; effective or not; please specify
Software-related experience Please specify; usability test
Hardware-related experience Please specify
Internet connection experience Please specify in regard to speed, smooth or freeze frequency, unexpected disconnection, reliable services35
Technical support staff Please specify in regard to availability, effectiveness, costs
Quality of images and sounds while utilizing telemedicine37 Please specify
Table 3.
Health Care Professionals’ Technological Experience With Telemedicine, as Proposed in the Conceptual Framework
Technical Experience With Telemedicine Potential Value Sets and Annotation
General expectations of telemedicine29 Please specify
Telemedicine training experience Sufficient or not; effective or not; please specify
Software-related experience Please specify; usability test
Hardware-related experience Please specify
Internet connection experience Please specify in regard to speed, smooth or freeze frequency, unexpected disconnection, reliable services35
Technical support staff Please specify in regard to availability, effectiveness, costs
Quality of images and sounds while utilizing telemedicine37 Please specify
×
Motivation
The penultimate component is health care professional motivation. Without the proper motivation to utilize or at least learn more about telemedicine, other parameters may have little to no impact on satisfaction. Measurements in this part of the framework included the obvious potential benefits to both the professional and the patient, such as reduced costs and improved access and efficiency (Table 4). 
Table 4.
Health Care Professionals’ Motivations Pertaining to the Utilization of Telemedicine, as Proposed in the Conceptual Framework
  Image not available
Table 4.
Health Care Professionals’ Motivations Pertaining to the Utilization of Telemedicine, as Proposed in the Conceptual Framework
  Image not available
×
Satisfaction
At the heart of the conceptual model was the professional's overall satisfaction with and attitudes toward telemedicine. This part of the model assessed the professional's expectations, attitudes, and confidence in the use of telemedicine (Table 5). A visual analog scale gauging professional expectations would be utilized in this part of the framework. The peer support available to each professional would be measured through directed questions. We would also look into how satisfaction ties in with expectations of the devices available. 
Table 5.
Health Care Professionals’ Overall Satisfaction With and Attitude Toward Telemedicine,18 as Proposed in the Conceptual Framework
Satisfaction Potential Value Sets and Annotation
Overall Satisfaction and Attitude Measures
 Favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Least favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Perception of care quality provided via telemedicine Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Whether telemedicine can fit into the routine workflow29,37 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill); if not, what should be done to improve?
Expectations of Telemedicine
 Specify expectations of telemedicine Please specify in terms of benefits to patients, to providers, and to care quality
 Meet the expectations Yes; no; not sure; please specify
Peer Support7
 Competitive advantages from peers Yes; no; not sure; please specify
 Insecurity in practice Yes; no; not sure; please specify
Attitudes About Using Telemedicine
 When telemedicine services are provided via providers in supplemental specialties Support; doubtful; against; neutral; not sure, please specify
 When telemedicine services are provided via providers in similar specialties Support; doubtful; against; neutral; not sure, please specify
Table 5.
Health Care Professionals’ Overall Satisfaction With and Attitude Toward Telemedicine,18 as Proposed in the Conceptual Framework
Satisfaction Potential Value Sets and Annotation
Overall Satisfaction and Attitude Measures
 Favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Least favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Perception of care quality provided via telemedicine Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Whether telemedicine can fit into the routine workflow29,37 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill); if not, what should be done to improve?
Expectations of Telemedicine
 Specify expectations of telemedicine Please specify in terms of benefits to patients, to providers, and to care quality
 Meet the expectations Yes; no; not sure; please specify
Peer Support7
 Competitive advantages from peers Yes; no; not sure; please specify
 Insecurity in practice Yes; no; not sure; please specify
Attitudes About Using Telemedicine
 When telemedicine services are provided via providers in supplemental specialties Support; doubtful; against; neutral; not sure, please specify
 When telemedicine services are provided via providers in similar specialties Support; doubtful; against; neutral; not sure, please specify
×
Discussion
Interpretation and Potential Use of the Conceptual Framework
Our refined framework included 5 components: demographics, care settings, experience, motivation, and overall satisfaction. The initial validation showed these components and subcomponents were essential to evaluate health care professionals’ satisfaction in delivering telemedicine. We believe that developing a conceptual framework is a necessary first step to systematically studying professionals’ satisfaction with telemedicine. The framework will enable a future comprehensive and core reference for telemedicine utilization. 
The potential use of the framework includes the following: (1) identifying existing gaps at the organizational level, technological level, and policy level during telemedicine care delivery; (2) developing strategies or seeking solutions to fill the gaps; (3) understanding the utilization of telemedicine and improving health care professionals’ experiences; and (4) improving care quality and patients’ experiences. Furthermore, evaluation results may help health care management teams, telemedicine products’ suppliers/vendors, and policymakers to improve telemedicine care delivery. The intended audiences of this conceptual framework include health care organizations who would like to evaluate existing telemedicine services, telemedicine services vendors (eg, technical providers) who would like to get feedback from the telemedicine service users, potential adopters of telemedicine services, and researchers who engage in telemedicine evaluation. 
Comparison With Existing Literature
In comparison with the summary from Langbecker et al41 on the “constructs commonly reported in telehealth research,” our conceptual framework addresses all of their constructs, such as satisfaction, experience, technical quality, effectiveness, usefulness, and the effect on interaction. We noticed that most of the references used in that article41 are about patient satisfaction. Although the authors recognize that “the same constructs measured at the patient level apply equally to clinicians,”41 there was no comprehensive measurement that focused on health care professionals. In our framework, we include “benefits to care quality” as a category to identify the clinical outcomes related to telemedicine, which is recognized as a defect in current existing satisfaction measures for telemedicine by the authors.41 
Hoonakker et al32 focused on the motivation and job satisfaction of tele-ICU nurses. Our conceptual framework differs in that it separates health care professionals’ experience, motivation, and satisfaction, whereas Hoonakker et al used a more task-oriented model. Our intention was to provide a comprehensive conceptual framework that evaluates professionals’ satisfaction across areas, regardless of the department in which the telemedicine services are utilized (ie, ICU, primary care, or others). In addition, both Romig et al28 and Engström et al31 provide more detailed measurements in satisfaction. 
Limitations and Future Plans
This article reviews many possible factors that could affect a health care professional's satisfaction with telemedicine. As such, one of the next steps will be to conduct focus groups to inform strategic prioritization of the survey elements, identify task-specific details that need to be included, and aid our further validation of the conceptual framework. If we can obtain a sufficient sample size of respondents on the future survey, a screening correlation analysis between health care professionals’ satisfaction and each measure/item identified by the conceptual framework will provide more objective evidence on the validation of each measure. 
Our findings about health care professionals' satisfaction and telemedicine could be similar to other patient care technologies (eg, electronic health record systems). Although this conceptual framework focuses on utilizing telemedicine, the comprehensive coverage may provide a starting point to evaluate professionals' satisfaction in utilizing other types of technologies during care delivery. Our long-term goal is to explore professionals' satisfaction across different types of care delivery technologies in addition to telemedicine and to detect and address possible gaps to improve care quality. 
There were limitations in the validation of this conceptual framework, including use of a small convenience group and lack of representation of nonphysician health care professionals. In future studies, we will aim to recruit a larger and more representative sample. Considering our conceptual framework is intended to be used as a default collection of evaluation items, additional customization and evaluation are needed to assess feasibility and utility of the framework. Additional limitations include lack of task-specific details in the framework and lack of objective measurement in the validation process. 
Conclusion
We believe the refined conceptual framework can provide a comprehensive tool for measuring health care professionals’ satisfaction in delivering telemedicine. Further customization of the framework according to the specific setting for telemedicine services may be needed. This refined conceptual framework can be used as a critical reference to evaluate health care professionals' satisfaction, to identify areas that can be improved, and to improve telemedicine services and care quality. 
Acknowledgments
We would like to thank Kelly Nottingham, MPH, for her helpful editing and valuable suggestions to the manuscript. 
Additional Information
Mr Phillips passed away when this project was conducted. 
References
Fatehi F, Wootton R. Telemedicine, telehealth or e-health? a bibliometric analysis of the trends in the use of these terms. J Telemed Telecare. 2012;18(8):460-464. doi: 10.1258/jtt.2012.gth108 [CrossRef] [PubMed]
Kvedar J, Coye MJ, Everett W. Connected health: a review of technologies and strategies to improve patient care with telemedicine and telehealth. Heal Aff (Millwood). 2014;33(2):194-199. doi: 10.1377/hlthaff.2013.0992 [CrossRef]
Whitten P. Telemedicine in Michigan: A Policy Report Addressing Legal and Regulatory Barriers. East Lansing, MI: Michigan State University; 2001.
American Hospital Association. The promise of telehealth for hospitals, health systems and their communities. TrendWatch. January 2015. http://www.aha.org/research/reports/tw/15jan-tw-telehealth.pdf. Accessed August 11, 2017.
Center for Connected Health Policy. State Telehealth Laws and Medicaid Program Policies: A Comprehensive Scan of the 50 States and District of Columbia. February 2015. Cited by: Wilson F, Trout K, Rampa S, Stimpson J. An Examination of Private Payer Reimbursements to Primary Care Providers for Healthcare Services Using Telehealth, United States 2009-2013. Washington, DC: Health Care Cost Institute; February 25, 2016. http://www.burness.com/wp-content/uploads/2016/02/hcci_nebraska_embargo.pdf. Accessed May 14, 2019.
Committee on Pediatric Workforce; Marcin JP, Rimsza ME, Moskowitz WB. The use of telemedicine to address access and physician workforce shortages. Pediatrics. 2015;136(1):202-209. doi: 10.1542/peds.2015-1253
Liddy C, Afkham A, Drosinis P, Joschko J, Keely E. Impact of and satisfaction with a new eConsult service: a mixed methods study of primary care providers. J Am Board Fam Med. 2015;28(3):394-403. doi: 10.3122/jabfm.2015.03.140255 [CrossRef] [PubMed]
Berman M, Fenaughty A. Technology and managed care: patient benefits of telemedicine in a rural health care network. Health Econ. 2005;14(6):559-573. doi: 10.1002/hec.952 [CrossRef] [PubMed]
Marcin JP, Nesbitt TS, Struve S, Traugott C, Dimand RJ. Financial benefits of a pediatric intensive care unit-based telemedicine program to a rural adult intensive care unit: impact of keeping acutely ill and injured children in their local community. Telemed J E Health. 2004;10(suppl 2):S-1-5. [CrossRef]
Dimmick SL, Mustaleski C, Burgiss SG, Welsh T. A case study of benefits & potential savings in rural home telemedicine. Home Healthc Nurse. 2000;18(2):124-135. [CrossRef] [PubMed]
Wilkinson JR, Spindler M, Wood SM, et al. High patient satisfaction with telehealth in Parkinson disease: a randomized controlled study. Neurol Clin Pract. 2016;6(3):241-251. doi: 10.1212/CPJ.0000000000000252 [CrossRef] [PubMed]
Polinski JM, Barker T, Gagliano N, Sussman A, Brennan TA, Shrank WH. Patients’ satisfaction with and preference for telehealth visits. J Gen Intern Med. 2015;31(3):269-275. doi: 10.1007/s11606-015-3489-x [CrossRef] [PubMed]
Gustke SS, Balch DC, West VL, Rogers LO. Patient satisfaction with telemedicine. Telemed J. 2000;6(1):5-13. doi: 10.1089/107830200311806 [CrossRef]
Doorenbos AZ, Eaton LH, Haozous E, Towle C, Revels L, Buchwald D. Satisfaction with telehealth for cancer support groups in rural American Indian and Alaska Native communities. Clin J Oncol Nurs. 2010;14(6):765-770. doi: 10.1188/10.CJON.765-770 [CrossRef] [PubMed]
Welch G, Balder A, Zagarins S. Telehealth program for type 2 diabetes: usability, satisfaction, and clinical usefulness in an urban community health center. Telemed J E Health. 2015;21(5):395-403. doi: 10.1089/tmj.2014.0069 [CrossRef] [PubMed]
Grant LA, Rockwood T, Stennes L. Client satisfaction with telehealth services in home health care agencies. J Telemed Telecare. 2015;21(2):88-92. doi: 10.1177/1357633X14566589 [CrossRef] [PubMed]
Siminerio L, Ruppert K, Huber K, Toledo FG. Telemedicine for Reach, Education, Access, and Treatment (TREAT): linking telemedicine with diabetes self-management education to improve care in rural communities. Diabetes Educ. 2014;40(6):797-805. doi: 10.1177/0145721714551993 [CrossRef] [PubMed]
Telemedicine helps rural EDs access critical neurology expertise for stroke patients. ED Manag. 2012;24(3):30-33. [PubMed]
Tanguay A, Dallaire R, Hébert D, Bégin F, Fleet R. Rural patient access to primary percutaneous coronary intervention centers is improved by a novel integrated telemedicine prehospital system. J Emerg Med. 2015;49(5):657-664. doi: 10.1016/j.jemermed.2015.05.009 [CrossRef] [PubMed]
Smith NM, Satyshur RD. Pediatric diabetes telemedicine program improves access to care for rural families: role of APRNs. Pediatr Nurs. 2016;42(6):294-299. [PubMed]
Eudes Y, Mangeney K, Diméo M, Pierrel S, Sokolakis S, Carnein S. The benefit of telemedicine for enabling elderly or disabled people to access care [in French]. Soins. 2016;61(810):41-44. doi: 10.1016/j.soin.2016.09.008 [CrossRef] [PubMed]
Ramos-Ríos R, Mateos R, Lojo D, Conn DK, Patterson T. Telepsychogeriatrics: a new horizon in the care of mental health problems in the elderly. Int Psychogeriatr. 2012;24(11):1708-1724. doi: 10.1017/S1041610212000981 [CrossRef] [PubMed]
Subbarao I, Cooper GP. Telehealth 2.0 and osteopathic medicine. J Am Osteopath Assoc. 2017;117(10):613-615. doi: 10.7556/jaoa.2017.117 [CrossRef] [PubMed]
Phillips BO, Duffrin C. Ohio Osteopathic Network of Excellence: establishing a statewide telehealth consortium. J Am Osteopath Assoc. 2001;101(12):720-722. [PubMed]
Hishitani T, Kawachi F, Fujimoto Y, Sugamoto K, Hoshino K, Ogawa K. Increased satisfaction of maternity hospital staff involved in fetal telediagnosis. Pediatr Int. 2014;56(4):644-646. doi: 10.1111/ped.12393 [CrossRef] [PubMed]
Vanagas G, Zaliu¯nas R, Benetis R, Slapikas R, Smith W. Clinical-technical performance and physician satisfaction with a transnational telephonic ECG system. Telemed J E Health. 2008;14(7):695-700. doi: 10.1089/tmj.2007.0113 [CrossRef] [PubMed]
Oliveira GL, Cardoso CS, Ribeiro AL, Caiaffa WT. Physician satisfaction with care to cardiovascular diseases in the municipalities of Minas Gerais: Cardiosatis-TEAM Scale. Rev Bras Epidemiol. 2011;14(2):240-252. [CrossRef] [PubMed]
Romig MC, Latif A, Gill RS, Pronovost PJ, Sapirstein A. Perceived benefit of a telemedicine consultative service in a highly staffed intensive care unit. J Crit Care. 2012;27(4):426.e9-e16. doi: 10.1016/j.jcrc.2011.12.007 [CrossRef]
Rodriguez KL, Burkitt KH, Bayliss NK, et al. Veteran, primary care provider, and specialist satisfaction with electronic consultation. JMIR Med Inform. 2015;3(1):e5. doi: 10.2196/medinform.3725 [CrossRef] [PubMed]
Vallières F, McAuliffe E, van Bavel B, Wall PJ, Trye A. There's no app for that: assessing the impact of mHealth on the supervision, motivation, engagement, and satisfaction of community health workers in Sierra Leone. Ann Glob Health. 2016;82(5):936-949. doi: 10.1016/j.aogh.2016.07.002 [CrossRef] [PubMed]
Engström M, Scandurra I, Ljunggren B, Lindqvist R, Koch S, Carlsson M. Evaluation of OLD@HOME virtual health record staff opinions of the system and satisfaction with work. Telemed J E Health. 2009;15(1):53-61. doi: 10.1089/tmj.2008.0049 [CrossRef] [PubMed]
Hoonakker PL, Carayon P, McGuire K, et al.   Motivation and job satisfaction of Tele-ICU nurses. J Crit Care. 2013;28(3):315.e13-e21. doi:10.1016/j.jcrc.2012.10.001 [CrossRef]
Whitten P, Love B. Patient and provider satisfaction with the use of telemedicine: overview and rationale for cautious enthusiasm. J Postgrad Med. 2005;51(4):294-300. [PubMed]
Creating a Framework to Support Measure Development for Telehealth. Washington, DC: National Quality Forum; 2017. https://www.qualityforum.org/Publications/2017/08/Creating_a_Framework_to_Support_Measure_Development_for_Telehealth.aspx. Accessed May 14, 2019.
McFarland LV, Raugi GJ, Reiber GE. Primary care provider and imaging technician satisfaction with a teledermatology project in rural Veterans Health Administration clinics. Telemed J E Health. 2013;19(11):815-825. doi: 10.1089/tmj.2012.0327 [CrossRef] [PubMed]
Soteriades ES, Talias MA, Harmon KT, Schumann SC, Kales SN. Electronic medical record use among US occupational medicine physicians: a national survey. J Occup Environ Med. 2013;55(10):1191-1196. doi: 10.1097/JOM.0b013e31829b3a4c [CrossRef] [PubMed]
Becevic M, Boren S, Mutrux R, Shah Z, Banerjee S. User satisfaction with telehealth: study of patients, providers, and coordinators. Health Care Manag (Frederick). 2015;34(4):337-349. doi: 10.1097/HCM.0000000000000081 [PubMed]
Wood EW, Strauss RA, Janus C, Carrico CK. The use of telemedicine in oral and maxillofacial surgery. J Oral Maxillofac Surg. 2016;74(4):719-728. doi: 10.1016/j.joms.2015.11.026 [CrossRef] [PubMed]
Wakefield BJ, Buresh KA, Flanagan JR, Kienzle MG. Interactive video specialty consultations in long-term care. J Am Geriatr Soc. 2004;52(5):789-793. doi: 10.1111/j.1532-5415.2004.52220.x [CrossRef] [PubMed]
Abdel-Qader DH, Cantrill JA, Tully MP. Satisfaction predictors and attitudes towards electronic prescribing systems in three UK hospitals. Pharm World Sci. 2010;32(5):581-593. doi: 10.1007/s11096-010-9411-z [CrossRef] [PubMed]
Langbecker D, Caffery LJ, Gillespie N, Smith AC. Using survey methods in telehealth research: a practical guide. J Telemed Telecare. 2017;23(9):770-779. doi: 10.1177/1357633X17721814 [CrossRef] [PubMed]
Figure.
The main components and examples of relevant items that contribute to health care professionals’ satisfaction in telemedicine care delivery in the refined conceptual framework.
Figure.
The main components and examples of relevant items that contribute to health care professionals’ satisfaction in telemedicine care delivery in the refined conceptual framework.
Table 1.
Health Care Professionals’ Individual Characteristics That May Influence Their Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
Demographics of Professional Potential Value Sets and Annotation
Biological Features
 Age, y ≤35; 36-45; 46-55; 56-65; >65
 Gender35 Female; male; nondisclosed
 Race29 White; African American; American Indian or Alaska Native; Native Hawaiian or Other Pacific Islander; Asian; multiracial
 Ethnicity Hispanic or Latino; not Hispanic or Latino
Acquired Features
 Education/training35 Medical student; resident; licensed physician (DO or MD); nonphysician: associate degree, bachelor's degree, master's degree, PhD
 Additional license or credentials to practice telemedicine Yes (please specify); no; not sure
 Professional title29 Physician; nurse practitioner; physician assistant; other (please specify)
 Location of residence35 Rural; suburban; urban; please specify state; please specify zip code
 Specialty29 Family medicine; urgent care; occupational medicine; internal medicine; pediatrics; obstetric-gynecologists; behavioral health; specialists (please specify)
 Practice experience, y29 ≤5; 6-10; 11-20; 21-30; >30
 Practice experience with telemedicine, y36 ≤5; 6-10; 11-15; 16-20; >20
 Frequency of telemedicine utilization Daily (specify how many times); weekly (specify how many times); monthly (specify how many times)
 Role during telemedicine services Provides telemedicine services; receives telemedicine services; both
 Experience with telemedicine Current telemedicine user; previous telemedicine user; no practice with telemedicine but with patients who utilize telemedicine via other providers; other (please specify)
 Attitude toward telemedicine Anti-telemedicine; naive telemedicine; telemedicine fan; neutral; hesitation; prohibiting; supporting; infringing; other (please specify)
 Technological competency Beginner (eg, seek help in solving IT issues); middle level (eg, able to solve one's own IT issues); advanced (eg, tech savvy and able to help others solve IT issues)
 Technology adoption type Innovator; early adopter; early majority adopter; late majority; adopter; laggard
 General confidence level in ability to provide quality medical care37 High; medium; low

Abbreviation: IT, information technology.

Table 1.
Health Care Professionals’ Individual Characteristics That May Influence Their Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
Demographics of Professional Potential Value Sets and Annotation
Biological Features
 Age, y ≤35; 36-45; 46-55; 56-65; >65
 Gender35 Female; male; nondisclosed
 Race29 White; African American; American Indian or Alaska Native; Native Hawaiian or Other Pacific Islander; Asian; multiracial
 Ethnicity Hispanic or Latino; not Hispanic or Latino
Acquired Features
 Education/training35 Medical student; resident; licensed physician (DO or MD); nonphysician: associate degree, bachelor's degree, master's degree, PhD
 Additional license or credentials to practice telemedicine Yes (please specify); no; not sure
 Professional title29 Physician; nurse practitioner; physician assistant; other (please specify)
 Location of residence35 Rural; suburban; urban; please specify state; please specify zip code
 Specialty29 Family medicine; urgent care; occupational medicine; internal medicine; pediatrics; obstetric-gynecologists; behavioral health; specialists (please specify)
 Practice experience, y29 ≤5; 6-10; 11-20; 21-30; >30
 Practice experience with telemedicine, y36 ≤5; 6-10; 11-15; 16-20; >20
 Frequency of telemedicine utilization Daily (specify how many times); weekly (specify how many times); monthly (specify how many times)
 Role during telemedicine services Provides telemedicine services; receives telemedicine services; both
 Experience with telemedicine Current telemedicine user; previous telemedicine user; no practice with telemedicine but with patients who utilize telemedicine via other providers; other (please specify)
 Attitude toward telemedicine Anti-telemedicine; naive telemedicine; telemedicine fan; neutral; hesitation; prohibiting; supporting; infringing; other (please specify)
 Technological competency Beginner (eg, seek help in solving IT issues); middle level (eg, able to solve one's own IT issues); advanced (eg, tech savvy and able to help others solve IT issues)
 Technology adoption type Innovator; early adopter; early majority adopter; late majority; adopter; laggard
 General confidence level in ability to provide quality medical care37 High; medium; low

Abbreviation: IT, information technology.

×
Table 2.
Care Settings That May Influence Health Care Professionals’ Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
  Image not available

a Response options overlap; more than 1 can be selected.

Table 2.
Care Settings That May Influence Health Care Professionals’ Satisfaction in Delivery of Telemedicine Services, as Proposed in the Conceptual Framework
  Image not available

a Response options overlap; more than 1 can be selected.

×
Table 3.
Health Care Professionals’ Technological Experience With Telemedicine, as Proposed in the Conceptual Framework
Technical Experience With Telemedicine Potential Value Sets and Annotation
General expectations of telemedicine29 Please specify
Telemedicine training experience Sufficient or not; effective or not; please specify
Software-related experience Please specify; usability test
Hardware-related experience Please specify
Internet connection experience Please specify in regard to speed, smooth or freeze frequency, unexpected disconnection, reliable services35
Technical support staff Please specify in regard to availability, effectiveness, costs
Quality of images and sounds while utilizing telemedicine37 Please specify
Table 3.
Health Care Professionals’ Technological Experience With Telemedicine, as Proposed in the Conceptual Framework
Technical Experience With Telemedicine Potential Value Sets and Annotation
General expectations of telemedicine29 Please specify
Telemedicine training experience Sufficient or not; effective or not; please specify
Software-related experience Please specify; usability test
Hardware-related experience Please specify
Internet connection experience Please specify in regard to speed, smooth or freeze frequency, unexpected disconnection, reliable services35
Technical support staff Please specify in regard to availability, effectiveness, costs
Quality of images and sounds while utilizing telemedicine37 Please specify
×
Table 4.
Health Care Professionals’ Motivations Pertaining to the Utilization of Telemedicine, as Proposed in the Conceptual Framework
  Image not available
Table 4.
Health Care Professionals’ Motivations Pertaining to the Utilization of Telemedicine, as Proposed in the Conceptual Framework
  Image not available
×
Table 5.
Health Care Professionals’ Overall Satisfaction With and Attitude Toward Telemedicine,18 as Proposed in the Conceptual Framework
Satisfaction Potential Value Sets and Annotation
Overall Satisfaction and Attitude Measures
 Favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Least favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Perception of care quality provided via telemedicine Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Whether telemedicine can fit into the routine workflow29,37 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill); if not, what should be done to improve?
Expectations of Telemedicine
 Specify expectations of telemedicine Please specify in terms of benefits to patients, to providers, and to care quality
 Meet the expectations Yes; no; not sure; please specify
Peer Support7
 Competitive advantages from peers Yes; no; not sure; please specify
 Insecurity in practice Yes; no; not sure; please specify
Attitudes About Using Telemedicine
 When telemedicine services are provided via providers in supplemental specialties Support; doubtful; against; neutral; not sure, please specify
 When telemedicine services are provided via providers in similar specialties Support; doubtful; against; neutral; not sure, please specify
Table 5.
Health Care Professionals’ Overall Satisfaction With and Attitude Toward Telemedicine,18 as Proposed in the Conceptual Framework
Satisfaction Potential Value Sets and Annotation
Overall Satisfaction and Attitude Measures
 Favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Least favorite features in delivering telemedicine services40 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Perception of care quality provided via telemedicine Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill)
 Whether telemedicine can fit into the routine workflow29,37 Please specify the care setting and the services, open response (eg, consultation, monitoring, prescription refill); if not, what should be done to improve?
Expectations of Telemedicine
 Specify expectations of telemedicine Please specify in terms of benefits to patients, to providers, and to care quality
 Meet the expectations Yes; no; not sure; please specify
Peer Support7
 Competitive advantages from peers Yes; no; not sure; please specify
 Insecurity in practice Yes; no; not sure; please specify
Attitudes About Using Telemedicine
 When telemedicine services are provided via providers in supplemental specialties Support; doubtful; against; neutral; not sure, please specify
 When telemedicine services are provided via providers in similar specialties Support; doubtful; against; neutral; not sure, please specify
×