Abstract
Objective
Direct-to-consumer telemedicine is rapidly changing the way that patients seek medical care. This study provided the first report on access, cost and quality of text- and image-based tele-dermatology e-consults, in China.
Methods
We adopted the Standardized Patients approach to evaluate the services of tele-dermatology e-consults in two metropolitan cities, that is, Beijing and Hangzhou, in China. We measured quality from four dimensions: service process, diagnosis accuracy, prescription and treatment comprehensiveness, based on China's national clinical guidelines. We performed logistic regressions to investigate factors that were associated with high-quality care.
Results
For 114 physicians eligible for inclusion, we succeeded in 87 (76%) validated visits. The median waiting time was 100 minutes (IQR 19–243 minutes) and the median length of consultation was 636 minutes (about 10 hours, IQR 188–1528 minutes). Per visit costs varied from $0 to $38, with a median of $8 (IQR 4–9). Among all, 15% of visits showed high quality in service process, 84% arrived in the correct diagnosis, 24% provided high-quality prescriptions and 71% provided comprehensive treatment. Providing images was associated with high quality in service process (OR 7.22, 95% CI 1.49–34.88). Visits in metropolitan Beijing and on non-work days had better prescription quality than that in metropolitan Hangzhou (OR 6.05, 95% CI 1.75–20.95) and that on workdays (OR 3.75, 95%CI 1.27–11.04), respectively.
Conclusions
Tele-dermatology e-consults seem to be easy to access and less expensive in China. However, great efforts are warranted to ensure that service processes and prescriptions adhere to clinical guidelines.
Introduction
Direct-to-consumer (DTC) telemedicine, which markets and delivers health care services online, is rapidly changing the way that patients seek medical care.1,2 One promising modality is e-consults, where patients consult professional clinicians via a web platform for opinions on diagnoses and prescriptions. 1 The emerging e-consults may improve access to specialty care and have particular merits when in-person visits are blocked.3,4 For example, the number of people using telemedicine in the United States increased by 120 folds in the first 3 weeks of the COVID-19 pandemic. 5
With physical examinations principally assessed visually, DTC telemedicine, namely tele-dermatology e-consults, has a great potential to change the practice of dermatology.6,7 For example, a systematic review shows that tele-dermatology reduced 3–17 days waiting time for patients with skin cancers in respect of in-person visits. 8 To date, real-world evidence that appraises the quality of tele-dermatology is scarce. 9 Although several studies reported a moderate to a high degree of diagnostic and management concordance between tele-dermatology and conventional in-person visits, no study has been done to test whether services delivered in e-consults settings are abided by evidence-based clinical guidelines, nor did any validate the diagnoses against golden standards.10–15 In addition, data were inconsistent regarding the health care costs that were associated with tele-dermatology.6,12,16,17 Some authors reported that tele-dermatology approximately reduced $10–80 per visit as compared to an in-person visit 18 ; whilst others reported higher costs of tele-dermatology in settings that require offline visits for further tests and examinations. 19
In China, the number of DTC telemedicine users has exceeded 215 million in 2021, 20 with e-consults taking the largest market. In practice, most e-consult services are based on interactive communication platforms, where patients leave messages in the form of text and images, and doctors are required to reply within a certain time interval. There are a few reports that describe the characteristics of patients or physicians using tele-dermatology e-consults. 21 But to our best knowledge, no study has evaluated the real-world performance of tele-dermatology in this country. In this study, we adopted the Standardized Patients (SPs) approach, by training players to present consistent cases in medical consultations, to evaluate the access, cost and quality of tele-dermatology e-consults services in China. We made a unique analysis, reporting internationally comparable data, on the performance of tele-dermatology e-consults in China. We drew lessons on how to better regulate and improve quality in clinical practice.
Methods
Settings
We investigated two metropolitan cities—Beijing and Hangzhou—which played leading roles in developing e-consults in China. We focused on tertiary public hospitals because they have the highest reputation and demand in the Chinese population for specialty care.22,23 And these providers are also most incentivized to provide DTC telemedicine, claiming the same quality of care against their offline outpatient services.
We performed comprehensive internet searches for all tertiary public hospitals that provided DTC telemedicine in the two cities. For inclusion, hospitals should offer e-consults dermatology services. The physicians should be formal employees in the departments of Dermatology, Allergy and Immunology, Traditional Chinese Medicine or other related departments that offer tele-dermatology consultations. Because most e-consult platforms are text- and image-based, we excluded the limited cases where online hospitals require live interactive videos or other types of online platforms (Figure 1).
Physicians from metropolitan Beijing and Hangzhou included in the study.
The e-consults practice processes are as follows (Figure 2). First, a patient registers on the official website, WeChat or other business APPs from the specific hospital he/she is interested in. Second, he/she selects an available doctor, describes the complaints and makes an appointment on the platform and then makes the payment. Third, the system automatically determines whether the doctor's expertise fits and then initiates an interactive e-consult. Last, the patient may text, post pictures or videos to communicate with the doctor via the online system and receive the doctor's diagnoses and advices.
The patients flow of the tele-dermatology e-consults services.
Study design
We adopted the SPs approach and used urticaria as the tracer condition. We used urticaria because this is a very common condition in e-consults and the clinical guidelines are well-defined. 24 We designed a standard script (Appendix 1) and recruited five teachers and students from a medical university to play the SPs. Each SP was trained three times by a professional team comprised of two dermatologists and five experts who were experienced in SPs. The team checked whether the SP candidate could consistently portray the emotional, physical and psychosocial aspects of the script (95% accuracy). The SP was then randomly selected to e-consult with the eligible physicians who were available on the platform from 20 April to 31 May 2021. Each physician was only appointed once to avoid contamination. Each visit was expected for diagnosis and advice from the physician, and then the SP completed a semi-structured questionnaire to describe the consultation process (Appendix 2 for the questionnaire). The players are required to save mobile phone screenshots of each consultation for further validation.
Figure 1 shows one example of such a consultation, where the SP was a 24-year-old female with a complaint of wheals with itching. She presented the appropriate history, including onset time and duration, main symptoms, following symptoms, triggering factors and relapse, medical recordd, past history and family history, and other necessary information. She was assigned to provide skin lesion images. Of note, we considered that the show-up of the picture of the lesion would substantially affect physicians’ judgment and thus randomly assigned only half visits showing the images.
Measures
We appraised the access, cost and quality of the e-consults. For access, we ascertained waiting time, defined as the time it took from the appointment to the physician's first reply. We also reported on lengths of consultation, defined as the time interval between the patient's first question and the physician's last reply. We ascertained cost by measuring the charge for each visit, with all monetary values adjusted to American Dollars (¥100.0 = $15.7). For the quality of care, we assessed four dimensions (Appendix 1): (a) service process. According to the case-specific checklist for urticaria in China's national clinical guideline (2018), 24 six components should be queried in the consultation process: main symptoms, onset time and duration, past history and family history, medical record, following symptoms, and trigger and regulation. We defined high quality should be either three of the components being queried (see Appendix 3); (b) diagnosis accuracy, as against the golden standard (urticaria); (c) prescription quality. China's national guideline (2018) recommends a list of medicines, including first-line and second-line, for the pharmacological management of urticaria, which requires doctors to adjust to second-line medicines or to consider drug combinations should first-line medicine be less effective, as in our case. It also made obligations that prescriptions should be accurate in terms of the name, form, dosage and course durations. We defined full agreement with these recommendations as high quality; and (d) treatment comprehensiveness, defined as whether proper lifestyle modification advice was provided in addition to prescriptions of any medicine.
For explanatory variables, we included city, type of the institution, sex and title of the doctor, whether images of the lesion were shown and whether the visits happened on workdays.
Statistical analysis
We described the characteristics of the visits and appraised access, costs and quality of care in the four dimensions. We stratified the analysis by the two cities and performed χ2 test or rank-sum test to assess differences between the two cities. We performed logistic regressions, reporting crude and adjusted odds ratio (OR), respectively, to investigate factors that were associated with quality of care. We chose p < 0.05 as the statistically significant thresholds (two-tailed). We used the Statistical Package for the Social Sciences (SPSS) version 20.0 to do the analysis.
This study was approved by the Ethics Committee of Capital Medical University, number 2021SY074. Participants provided verbal informed consent.
Results
Characteristics
A total of 23 (29%) hospitals out of 79 from the two cities provide e-consults services, where just 13 hospitals offer tele-dermatology. Among the 13 hospitals, there were 114 physicians eligible for inclusion, 20 (18%) of which made no response to the SP's appointment requests and 7 visits (6%) had invalid recordings because of not following the script or having incomplete information. No physicians voiced any suspicions during the consultation (Figure 1).
Finally, we received 87 validated records/physicians. A total of 52 physicians (60%) located in metropolitan Hangzhou and the other 35 (40%) practiced in metropolitan Beijing. Fifty-seven (66%) were from general hospitals and the others worked in traditional Chinese medicine hospitals. Fifty-five physicians (66%) were female and 50 (59%) were high-profiled physicians (associate chief or chief). About 51% of SPs did not provide images of the lesion. About 77% of SPs visited the physician on workdays and this is the only characteristic that finds a difference between the two cities (p = 0.040) (Table 1).
Characteristics of the providers and patients in metropolitan Beijing and Hangzhou.
Note. Three doctors have missing values in sex and title.
Access, cost and quality
The median waiting time for the first reply was 100 minutes (IQR 19–243), which was longer in metropolitan Beijing (137, IQR 11–458) than in metropolitan Hangzhou (52, IQR 20–217), but presenting no statistical significance (p = 0.255). The median length of consultation was 636 minutes (about 10 hours, IQR 188–1528), reporting no statistical significance across the two cities (p = 0.446). Per visit costs varied from $0 to $38, with a median of $8 (IQR 4–9), lower in metropolitan Hangzhou ($5, IQR 3–8) than in metropolitan Beijing ($8, IQR 8–9), p = 0.025 testing the difference.
Among all, 13 visits (15%) showed high quality in the service process, 73 (84%) arrived at the correct diagnosis, 21 (24%) provided high-quality prescriptions and 62 (71%) provided comprehensive treatment, which offered lifestyle modification advices. Of note, physicians in metropolitan Beijing performed much better in prescription quality (43%) than in metropolitan Hangzhou (12%) (P = 0.001); while performances did not vary across the two cities in the other quality dimensions (Table 2).
Access, cost and quality of tele-dermatology e-consults in metropolitan Beijing and Hangzhou.
In the survey of prescription quality (see Appendix 4), 16 (18%) visits did not result in any prescriptions. Nine (10%) had prescriptions but drug names were unclear. Eleven (13%) did not receive adjusted prescriptions as recommended by the clinical guideline. The other 30 (35%) visits, although coming out with correct medicine, either had indefinite dosages or missed information for course durations.
Factors associated with quality of care
Table 3 shows the results from logistic regressions that investigate the factors that were associated with high quality in service process and prescriptions, respectively. Data for the other two quality dimensions were shown in Appendix 5, because no variations were identified. Compared to visits that did not provide images, visits that provided images were seven times more likely to receive high-quality service processes (crude OR 7.22; 95% CI 1.49–34.88), which were independent of other factors (adjusted OR: 5.77; 95% CI 1.11–30.11). The results also show substantial effects of city and workday on prescription quality, where visits in metropolitan Beijing had an adjusted OR of 6.05 (95% CI 1.75–20.95) against metropolitan Hangzhou; and non-work day visits had a higher crude OR of 3.75 (95% CI 1.27–11.04) against workday visits, although the effects were slightly attenuated in the adjusted analysis (adjusted OR 3.36; 95%CI 0.94–12.03).
Factors associated with quality of tele-dermatology e-consults in metropolitan Beijing and Hangzhou, in services process and prescription.
Note. Three doctors have missing values in sex and title; adjusted for city, types of the institution, doctors’ sex and title, whether images were provided and timing of the visit. Reference category is Hangzhou, Traditional Chinese medical hospital, Female, Resident physician, No and Workday (Reference category of title is attending physician in the analysis of high prescription quality).
Discussion
Main findings
Using the method of standardized patients, we assessed the access, cost and quality of tele-dermatology practices in two metropolitan cities that lead to the development of e-consults in China. We found that only less than one-third of high-profiled hospitals in the two cities provided tele-dermatology services. With a median waiting time of less than 2 hours and per-visit cost of $8, tele-dermatology seems accessible and affordable from the best public hospitals in China. The quality of e-consults services is fine in terms of diagnosis accuracy and treatment comprehensiveness. However, service process and prescription quality are less commendable, where we found only less than a quarter of the physicians provided high-quality care, as against standard clinical guidelines. Notably, physicians were more likely to process high-quality queries only when images of the lesion were provided, and visits in metropolitan Beijing and on non-work days are much more likely to benefit from quality prescriptions than metropolitan Hangzhou and workday visits.
Interpretation
Most of the prior studies used predefined subjective scales to appraise the quality of telemedicine, such as the aspects of tangibility, reliability, responsiveness, assurance and empathy. 25 However, empirical evaluation in real-world practice is rare. 26 Using a very standard procedure performed by SPs, we provided the first assessment on the access, cost and quality of tele-dermatology e-consults practices in China. Globally, our systematic search of the literature identified just one prior study that adopted the SPs approach to assess the access, cost and quality of the services in real tele-dermatology practice. 27 But unfortunately, the study only reported specific data on service process quality. We are unique in assessing the quality of care more specifically and comprehensively from four aspects, namely: service process, diagnosis, prescriptions and comprehensiveness, and in addition, the factors that were associated with high quality.
Our findings that nearly 90% of the visits had an accurate diagnosis yielded equivalent or higher estimates than prior data that assessed the accuracy of diagnosis of in-person dermatology practice, ranging between 68% and 87%.28–33 This finding suggests that dermatologists are capable of providing comparable and compelling decisions in tele-dermatology practices. However, we found that physicians contributed considerably low efforts in medical history taking in telemedicine settings, consistent with studies from other disciplines.27,34 Medical histories, including the following symptoms and triggering factors, are important considerations to keep patients away from delays of severe consequences. 35 Causes could be attributed to the lack of regulation efforts and/or physicians’ low awareness of the online practice norms, physicians’ low financial incentives and heavy workloads.36,37 Interestingly, we found that the provision of images was associated with better quality in service process, which seemed to be counter-intuitive since dermatologists are expected to ask more details should they have no chance to see the skin lesions. But one has to bear in mind that dermatology is a visual specialty and only text descriptions of the lesion may decrease the physician's confidence, as one prior study has shown. 38 Thus, it might be a prerequisite condition to incorporate clear images of the skin lesions in tele-dermatology practice.
Importantly, we found only less than a quarter of the patients received appropriate prescriptions, as per clinical guidelines. The standard patient we played in this study is not sensitive to Cetirizine. As recommended by China's national guideline (2018), for such a case, second-line treatment—including changing Cetirizine to other antihistamines or drug combinations—should be prescribed. 24 However, one-third of our visits arrived with no prescriptions or inappropriate medicines, which may delay the patients to find the right medicine or even worsen the conditions. In addition, one-third of visits did not offer definite dosages or course durations, which may cause recurrent attacks. For example, one physician prescribed Bacilli Calmette Gurin Polysaccharide and Nucleic Acid Injection without suggesting course durations, but this medicine will not work unless used consistently for more than 15 days. 39 Our data show that physicians perform better on non-work days in respective to workdays, in line with data from the Netherlands. 40 In China, e-consults are an extra job for physicians, and thus more spare time on non-work days may explain the better quality of prescriptions. However, it should be recognized that increasing workload on non-work days is infeasible, and a systematic approach is needed to adjust the schedule of online services. Additionally, we found there was significantly better prescription quality in metropolitan Beijing. More engagement may help explain why physicians in metropolitan Beijing may be more familiar with and abide by the guidelines. There were more dermatologists from metropolitan Beijing (six physicians, as compared to two physicians from metropolitan Hangzhou) who participated in the development of the Guideline for Diagnosis and Treatment of Urticaria in China (2018). 24 More importantly, the government of metropolitan Beijing has put forward more regulation measures on e-consults services, including real-time monitoring of online prescriptions and timely feedback to physicians. 41 Indeed, physicians’ poor adherence to clinical guidelines could be improved using managerial levers. 36 For patient safety, national standards on inquiries and prescriptions are urgently needed for e-consults practices in China. Particular efforts are needed to monitor practitioners’ efforts to better understand patients’ medical history, and to order prescriptions in international nonproprietary names and the form of texts rather than handwritten, scanned, voiced or any other ambiguous forms. Our unique finding of low comprehensiveness of care also suggests that patient education should always be added to tele-dermatology e-consults in writing, to make sure that the patients transform these instructions into behavioral changes.
In respective of the body of literature, our findings on the waiting time and price per consultation were much lower than offline services internationally. For example, patients normally wait 14–90 days for an offline appointment in China's tertiary hospitals and about 32 days in the United States.42,43 We are the first to report the waiting time and costs of tele-dermatology in China. Our estimation of just 100 minutes is much lower than that from San Francisco, 0–8 days. 44 The elimination of appointment time is the key benefit of e-consults for a certain population. We found that the cost of tele-dermatology ($5–8) was cheaper than offline services, which typically range from $7 to $14 in China, though our estimates of the costs are higher than prior investigation on general e-consults services in China. 45 Indeed, $8 per visit is much cheaper than costs data from other settings, for example, $35–$95 per visit in San Francisco. 44 e-Consults offer opportunities for patients to directly access specialty care. Our findings suggest that dermatology e-consults may be a better option than offline during the COVID-19 pandemic and when medical tests are not necessary.
Strength and limitations
This is the first study that adopted the approach of SPs to evaluate the access, cost and quality of tele-dermatology e-consults services in China. The method has several advantages. First, the use of SPs is regarded as a “golden standard” to assess the quality of health care. 46 The illness—either the diagnosis, standard clinical querying process or managing protocol as against evidence-based guidelines—is predetermined into a standard script, which largely avoids recall bias and allows for valid quality comparisons across various types of physicians and hospitals. Second, urticaria is easy to play because the appearance could return to normal spontaneously, and the risk of being penetrated by the doctor is low. As we found, not a single visit was eventually questioned.
However, the study has three important limitations. First, 20 (17.5%) of the eligible physicians made no response to the SP's appointment requests, which is not uncommon in prior studies though. 47 Reasons might be attributed to doctors’ heavy offline workload and/or the failure of the notification system for 18 visits happened on workdays. 47 Doctors without response simply do not meet their commitments. They may have a higher probability to perform worse than their colleagues who actually served the patients who were officially appointed. Therefore we may overestimate the actual quality of care in the two cities. Second, seven visits (6.1%) had invalid recordings for not following the script or having incomplete records. These missing data are likely to be random because e-consults is an emerging service in China, and some online platforms are unstable to use. Third, we recruited students and teachers from a medical school to play the patient, which is a typical treatment in standardized patient studies. However, clinical practice is knowledge and experience intensive. For further improvement, it is better to be conducted by professional dermatologists in future studies. Finally, the information presented on the platform is limited, and we are not able to include more information on doctors’ characteristics to analyze the factors associated with quality of care, leaving an area for further investigation. 48
Conclusions
Tele-dermatology e-consults is easy to access and less expensive in China. When patients suffer from minor skin ailments, e-consults seem a better option. However, great efforts are warranted to ensure that the service process and prescriptions adhere to evidence-based clinical guidelines. It is important for health care systems to be proactive in regulating this market and better engage physicians to provide high-quality care.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076221140763 - Supplemental material for Access, charge and quality of tele-dermatology e-consults in China: A standardized patients study
Supplemental material, sj-docx-1-dhj-10.1177_20552076221140763 for Access, charge and quality of tele-dermatology e-consults in China: A standardized patients study by Rui Guo, Mengchi Hou, Yangyang Han and Xing Lin Feng in Digital Health
Footnotes
Acknowledgements
We would like to thank Yuan Gao, Peking Union Medical College Hospital, for his assistance and guidance in this research.
Contributorship
RG and MH researched literature and conceived the study. RG, MH and XF were involved in protocol development, gaining ethical approval, patient recruitment and data analysis. RG lead the literature review. XF finalized the draft and other authors participated in the critical interpretation of the data. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
The ethics committee of Capital Medical University approved this study (number: 2021SY074).
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China [grant number 72174131].
Guarantor
RG.
Supplemental material
Supplementary material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
