A low-resource digital infrastructure to streamline viewing and interpretation of radiographic images: A case study of Uganda’s hospital-wide environment

Abstract

Picture Archiving and Communication Systems (PACS) are said to improve patient quality of care through timely access to radiological images by clinicians. However, they are costly to be considered for hospital wide environment in low income countries. Ordinary core i3 computer systems (PCs) can provide an affordable and faster alternative solution for PACS workstations. This comparative study assessed the diagnostic accuracy, image quality of ordinary PC systems versus PACS workstations and patient turnaround time (PTAT). Forty images were randomly obtained and viewed by four raters from both PACS and PC. The findings showed modest agreement among raters (kappa 0.644 for PACS and 0.5164 PC) with acceptable diagnostic accuracy for PC (AUC = 0.7990), 97.5% reproduction of images on PC and significant reduction in PTAT after a switch to PC (4.8 min), p < 0.001, suggesting that PC display can improve quality of health care services through timely access to radiographic images.

Keywords

image accuracy image quality PACS workstations PTAT PC-based display systems

Introduction

In recent decades, we have seen the growth of technological innovations in the IT domain aimed at addressing challenges faced by patients, clinicians, and administrators.1 The main drawback in healthcare service delivery has been linked to long patient waiting times. Long waiting times have been said to be indicators of poor health system performance.² For instance, during emergency or critical care where patients require acute medical care, waiting times have been confirmed to have damaging effect on patient survival rates and a greater risk of poor health outcomes.^3,4

Radiology services account for the largest time (61%) spent in the hospital and significantly contributes to long patient turnaround time (TAT) and that affects quality of care.5,6

In radiology, TAT can be defined as “time from the physician’s request” for imaging until the “time the physician views the results”.⁷ Now days, Diagnostic imaging systems are being designed to improve both quality of care and efficiency of clinical work processes.⁸

Digital imaging is key in emergency care such as treatment of acute cases, since it allows quick access to diagnostic information that is essential for making clinical decisions^9,10 and observes the Digital Imaging and Communication in Medicine (DICOM) standard. DICOM is one of the standards that is universally accepted for the management and communication of medical imaging data.¹¹

While digital imaging systems have been primarily implemented in countries with a higher share of resources, the World Health Organization (WHO),¹² observes that low income countries could significantly benefit from implementing digital imaging for radiological examinations.^3,13
-16

However, digital imaging systems also have limitations such as high costs particularly the dedicated Picture archiving and communication systems (PACS), and their cost being a major obstacle for hospital wide implementation^17,18 which has a bearing on several levels of health care delivery. Some studies show that PACS can be easily integrated with other systems and applications on ordinary PC-based viewing systems.^19,20

In addition Digital images require an application (DICOM viewer) for their display on monitors²¹ and may not display directly on ordinary operating systems such as windows.²² Furthermore, DICOM images are in large files and occupy considerable storage space. Using compression, medical image size can be condensed, reducing on storage space,²³ Tingberg observes that an image with good quality is one that fulfills its diagnostic purpose. While, there is a general acknowledgment that compression techniques are associated with some degree of image data loss and the need to assess the diagnostic quality of compressed images, most common literature highlights JPEG compression as an industrial standard that facilitates compression and streaming with less effect to the original images.^24,25

Sharing of digital images has been via the Picture archival and communication systems (PACS).²⁶ PACS have what is referred to as the PACS viewer, a computer program that receives and displays radiology images.¹⁷ Further, PACS workstations support image networking on both the Local Area Networks (LAN) and Wide Area Network (WAN).^8,16

Ordinary PC-based Viewing systems have been recommended for several years in different studies because of their cost effectiveness.^19,27 A study by Wu et al.²⁷ found that ordinary PC-based systems were affordable especially for low-cost hospital wide PACS and they were easy to obtain both in terms of hardware and software support locally.

In our study, we compared PACS work stations with ordinary core i3 PC based viewing systems as affordable digital image display/viewing solution. Ordinary core i3 computers according to Indrajit and Alam,²⁸ comprise of mid-range microprocessors more powerful than entry level Celeron and Pentium processors. The purpose of this study was to explore the use of ordinary core i3 PC based viewing systems as existing low-cost digital infrastructure to streamline clinician’s viewing and interpretation of radiology images to improve delayed access of radiographic images to clinicians.

Materials and methods

The study was conducted at St. Francis Hospital Nsambya, Kampala Uganda. The hospital had recently installed Picture Archiving and Communication Systems (PACS) in radiology to reduce long waiting times, improve patient management, hospital workflows, and stay cost effective. The study population comprised of medical doctors from Emergency and ICU, radiologist and radiographer from Radiology. Using Yamane, (1967)’s²⁹ formula for Sample sizes we obtained a sample size (n = 12). However, given the homogenous nature of population these units, we incorporated the cost-based theorem including only the sample population readily available to the study; giving a sample size of (n = 6). This included two clinicians from ICU, two clinicians from Emergency unit, one radiologist, and one radiographer. To measure Image diagnostic accuracy, a total of 40 images were randomly obtained using systematic random sampling and viewed on PACS Workstation and PC-based display system.

To measure patient turn-around time, a random sample of (n = 20) patients was used in an experimental exercise to define PTAT. The Sample size was calculated by Yamane (1967)’s²⁹ formula for Sample sizes. A 95% confidence level and p = 0.5 are used.

Study design

The study adopted a comparative study to collect quantitative data with in a case study research design.

We chose a comparative study because we wanted to elucidate the similarities and differences associated with viewing radiographic images on ordinary PC display system versus PACS workstation.

Data collection

Primary data were collected from Patient X-ray images, questionnaires and observations, and secondary data were obtained from the online journals and the Hospital internal documents. The “Test-Retest” method was used to test reliability of the instruments.

For image review to determine accuracy of the display systems sensitivity, specificity and ROC analysis were employed. Forty images were arranged in groups of ten and were randomly viewed by four clinicians on core i3 PC based display system; the same images were again viewed by the same clinicians on a PACS workstation after a period of 3 weeks.

During each viewing session, the clinicians indicated their diagnostic assessment of each image as either positive (with pathology) or negative (without pathology). The gold standard was determined from the outcomes of the same images reviewed by radiologists based on the services they provide while utilizing the PACS workstations. The representation of the raters (clinicians) was two male clinicians and two female clinicians. Also, two of the clinicians worked in ICU and two other clinicians worked in emergency unit. The mean number of years of experience was 2.5 years. The study also incorporated the Area under the ROC curve (AUC) to determine the distinctive power of the test as a measure for diagnostic accuracy.

The radiologist ensured all the 40 images were selected from a sample that fulfilled the image quality criteria (IC) guideline as developed by the European commission.³⁰ These same images were converted to JPEG format and viewed by the radiographer on core i3 PC based display system to see if there was deviation in quality. To observe patient turnaround time (PTAT), Pre- and post-intervention evaluations were observed against the target turnaround time of 10 min, that is, from the time of examination performance to the time the image was viewed by clinicians. The PTAT was recorded on a metric time sheet using time of acquisition seen on the image itself and the time the same patient was prescribed treatment or admitted or referred as recorded in the EMRS.

Data analysis

The study employed Kappa statistic to measure the interobserver reliability among the raters. The majority rule was employed to categorize the outcome of each image by the four raters. The majority rule was considered where more than three raters agreed on the image outcome. Where there were two raters in agreement, the image was dropped from this analysis. Data from image quality assessment forms was analyzed based on the European guidelines on quality criteria for diagnostic Radiographic images incorporating all images that fulfilled the image criteria. Using descriptive statistics, the mean PTAT, standard deviation (SD) and percentiles were determined when analyzing PTAT. Statistical software STATA (ver. 13.0) was utilized to conduct statistical analysis of the data.

Quality control

The observers were blinded on clinical history associated with the images to minimize observer bias and maximize validity of the results. A visual wash out period of three (3) weeks was incorporated to minimize observers visual recall bias. This period was reached upon based on data from previous studies involving intraobserver recall bias. Furthermore, guidance from the American FDA to reduce recall bias recommends sufficient time between intra rater reviews of the same images to reduce recall bias. However, it does not specify the period. However some studies recommend a period between 3 and 45 days.³¹

To ensure some level of similarity in gray scale perception, basic appearance was used for each given image between different viewing systems. Ordinary core i3 PC-based display systems were corrected to acceptable brightness values of 80–100 cd/m²,³² using the Windows Display Color Calibration software.

Ethical considerations

The Higher Degrees, Research and Ethics Committee (HDREC) of Makerere University School of Public Health and St. Francis Hospital Nsambya provided the ethical clearance and permission to conduct this study respectively. The purpose of the study was clearly explained to the respondents and consent for participating in the study sought from them. Participation in the study was voluntary with no coercion.

Results

Diagnostic accuracy

During the study, the raters (clinicians) showed modest agreement with kappa statistics of 0.644 and 0.516 for both PACS workstation and ordinary PC, all well above 0.5. With PC based display system; clinicians correctly identified 80.0% of the images, yielding a sensitivity of 83.3%, and a specificity of 76.4% (Table 1). Using PACS workstation, clinicians correctly identified 97.3% of the images, yielding 100% sensitivity, and 94.1% specificity. (Table 2). The accuracy rates between the two display systems were 80% and 97.3% for ordinary PC based display system and PACs workstation respectively.

Table 1.

Detailed report of sensitivity and specificity when using PC display.

Cut point	Sensitivity (%)	Specificity (%)	Correctly classified (%)	LR+	LR−
(⩾0)	100.00	0.00	51.43	1.0000
(⩾1)	83.33	76.47	80.00	3.5417	0.2179
(>1)	0.00	100.00	48.57		1.0000

Table 2.

Detailed report of sensitivity and specificity when using PACS workstation.

Cut point	Sensitivity	Specificity	Correctly classified	LR+	LR−
(⩾0)	100.00	0.00	54.05	1.0000
(⩾1)	100.00	94.12	97.30	17.0000	0.0000
(>1)	0.00	100.00	45.95		1.0000

Accuracy rate = TP + TN ÷ TP + TN + FP + FN × 100. The difference in accuracy rate was not statistically significant (p = 0.66) (Figure 1).

Figure 1.

X-ray image as viewed on PACS workstation and ordinary PC display at p = 0.66.

Diagnostic accuracy of the two systems was further assessed using the receiver operating characteristic (ROC) curve where the area under the curve (AUC) was estimated. AUC = 0.5 when the ROC curve corresponds to random chance and 1.0 for perfect diagnostic accuracy.

The AUC for ordinary PC (0.799) and PACS (0.971) significantly varied (p = 0.01) but were both above the acceptable AUC cut-off (0.7). (Figures 2 and 3).

Figure 2.

Area under the curve for ordinary PC based display.

Figure 3.

Area under the curve for PACS workstation.

Quality of images

The images were rated based on a 5 point scale (Table 3). Using ordinary PC-based display system 35 JPEG images fulfilled 97.5% of the IC compared to 100% for DICOM images using PACS workstation. The difference in image quality was not statistically significant (p = 0.66).

Table 3.

Compliance with European guidelines for 40 JPEG converted chest PA x-ray examinations Guidelines adopted from European commission, (1996).

Image criteria	Number of scores per score category
Visualization and sharp reproduction of characteristic features;	5	4	3	2	1
(a) Vascular pattern of the lungs (peripheral)	38	1	1	0	0
(b) Trachea and proximal bronchi	39	1	0	0	0
(c) Borders of the heart and aorta	40	0	0	0	0
(d) Diaphragm and costo-phrenic angles	40	0	0	0	0
(e) Retro cardiac lung and mediastinum	37	2	1	0	0
(f) Spine through the heart shadow	40	0	0	0	0

Patient turnaround time (PTAT)

The study desired 10 min as the benchmark for PTAT. The minimum PTAT before ordinary PC implementation was 11 min (1% percentile) and a maximum of 35 min (99% percentile). The mean PTAT before was 22.65 min with a standard deviation of 6.49 min. The minimum PTAT after ordinary pc implemention was 3 min (1% percentile) and a maximum of 7 min (99% percentile). The mean PTAT after was 4.80 min with a standard deviation of 1.15. From the t statistic the mean difference before and after was 17.85 demonstrating a significant difference (p < 0.001).

Discussion

The main purpose of this study was to explore the use of existing low-cost digital infrastructure to streamline clinician’s viewing and interpretation of radiology images and improve delayed access to radiographic images. Ordinary PC display system as a low-cost digital infrastructure was compared with PACS workstations and the comparison mainly focused on the diagnostic accuracy and quality of images viewed on either of these systems. The effect of using ordinary computers on patient turnaround time was also assessed.

Diagnostic accuracy

From the findings, it is suggested that clinicians demonstrated better diagnostic outcomes while viewing images using PACS workstations compared to ordinary PC based display systems and analysis of the data showed significantly better discriminative ability for PACS workstations than ordinary PC based display system (AUC 0.97 vs 0.79 p = 0.01).

Using Kappa statistic, the interater agreement of the clinician outcomes for PACS Workstation was more substantial (kappa 0.644) compared to ordinary PC display systems (kappa 0.51). Although moderate, the level of agreement with ordinary PCs was still within the acceptable degree of agreement and is consistent with McHugh,³³ who stated that kappa agreement as low as 0.41 may be accepted for health related research.

Notwithstanding the ordinary PC weaknesses, their outcomes of diagnostic accuracy recorded good specificity (76.4%) and sensitivity (83.3%) with acceptable discriminative ability on the ROC (AUC = 0.79) at the score cut-offs utilized in the analysis, which was similar to what has been reported for PC systems and ROC analysis in other study settings^34,35 and partly confirming findings in earlier research.³⁶

The findings from this study indicate that although PACS workstations are superior (100% sensitivity and 94.1 specificity), PC based display systems can as well be utilized to triage patients based on radiographic findings and promote rational use of these technologies in hospital settings with single or no PACS correlating with another study,¹⁹ indicating that PC based display systems offers good diagnostic image accuracy.

Quality of images

Although the DICOM protocol for digital medical images is associated with superior quality,³⁷ our study indicated good quality consistency as per the image criteria in the recreated images after they were converted from DICOM to JPEG and viewed on ordinary computers. 97.5% of the total images passed the quality criteria. These findings are related to another study stating that while using the image criteria on ordinary PCs, most chest X-rays and lumber spine X-ray passed by 70%³⁸ and is supported by results in another study stating that diagnostic value of the radiographic images is highly dependent on the image quality.³⁹

Although the 2.5% of the images did not pass the quality criteria when viewed by the radiographers, the radiologists had considered them with similar diagnostic quality as the 97.5% before conversion. However, the difference in image quality between DICOM versus JPEG was not statistically significant (p = 0.66).

This study mainly considered the anatomical landmarks (image criteria) to determine the quality of the images. Future studies could incorporate other physical attributes such as image contrast, spatial resolution, and noise.

Patient turn around time and workflow processes

Our data further demonstrated good throughput attained by having radiological images shared directly on the network and made available on the ordinary PC based display systems at emergency unit resulting into a mean PTAT of 4.8 min.

The results of this study were impressive, marked by a great improvement in patient turnaround time from an average mean of 22.65 to 4.8 min after piloting the PC based display system. These changes are primarily due to the dramatic improvements of delivering the images from radiology to the clinician at emergency care points via electronic transmission compared to hand delivery of images to the clinicians before piloting PC display system. Electronic delivery of medical images has been seen to improve workflow process in the radiology department in similar studies.⁴⁰

Generally, the marked improvement in PTAT indicates a major positive impact of PC based display systems on radiology services and general emergency clinical practice. It represents an important milestone that can link affordable technological innovations to mass-use programs in low resource settings and is in agreement with insights from another study stating that sustainable innovative activities should be built based on availability and affordability measures from inception.⁴¹

Study limitations

The major limitation to the study was that it focused on clinicians at emergency care points, that is, Emergency and Intensive Care Units; other points of care such as pediatrics and OPD were not considered in this study which affects generalization. The study worked on an assumption that all participants by virtue of their experience, were in good position to adequately read the images as displayed and give an accurate assessment. Therefore, we did not incorporate training as an input which could have affected outcomes of image interpretation.

Conclusion

Use of Ordinary core i3 PC based display systems may not replace PACS workstations for image visualization and interpretation. However, their utilization is possible in a low resource limited setting where PACS workstations may not be readily available and can provide adequate image evaluation and diagnosis.

Although the DICOM format is associated with better image quality, conversion to JPEG format for viewing on ordinary core i3 PC based display systems is not associated with significant loss in image quality and can provide reliable diagnostic information exhibited by radiographic images where image viewing in DICOM is not possible.

Electronic transfer and viewing of images on ordinary core i3 computer based display systems significantly reduces waiting time in a resource-limited setting and hence improves workflow processes in the delivery of radiology services.

Footnotes

Acknowledgements

We thank the management and staff of St. Francis hospital, Nsambya who participated in this study. We express our gratitude for the time and advice provided by Ms. Logose Jasmine.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by HI-TRAIN. HI-TRAIN is a collaborative capacity building program between Moi University Kenya, University of Bergen Norway and Makerere University Uganda.

ORCID iDs

Edward Ruhuma

Josephine Nabukenya

References

Ossebaard

Van Gemert-Pijnen

eHealth and quality in health care: implementation time. Int J Qual Health Care 2016 Jun; 28(3): 415–419. Epub ahead of print 29 March 2016. DOI: 10.1093/intqhc/mzw032. PMID: 27029590.

Bradfield

OM.

Hidden waiting times of elective surgery. Health Care Manag Sci 2009. DOI: 10.1007/s10729-008-9081-8.

Zennaro

Oliveira Gomes

Casalino

, et al. Digital radiology to improve the quality of care in countries with limited resources: a feasibility study from Angola. PLoS ONE 2013. DOI: 10.1371/journal.pone.0073939.

Meier-Kriesche

Port

Ojo

, et al. Effect of waiting time on renal transplant outcome. Kidney Int 2000; DOI: 10.1046/j.1523-1755.2000.00287.x.

Conrad

Meek

Dumit

Operationalizing a One Health approach to global health challenges. Comp Immunol Microbiol Infect Dis 2013 May; 36(3): 211–6. DOI: 10.1016/j.cimid.2013.03.006. Epub 24 May 2013. PMID: 23711930.

Oche

Adamu

. Determinants of patient waiting time in the general outpatient department of a tertiary health institution in north Western Nigeria. Ann Mil Health Sci Res 2013; 3(4); 588–592. DOI: 10.4103/2141-9248.122123

Georgiou

Williamson

Westbrook

, et al. The impact of computerised physician order entry systems on pathology services: a systematic review. Int J Med Inform 2007. DOI: 10.1016/j.ijmedinf.2006.02.004.

Aparecido Nunes

de Mello

Barbosa Coelho

, et al. Analyses of budget impact considering the use of the picture archiving and communication system. Value Health Reg Issues 2015. DOI: 10.1016/j.vhri.2015.04.004.

Hains

Georgiou

Westbrook

JI.

The impact of PACS on clinician work practices in the intensive care unit: a systematic review of the literature. J Am Med Inform Assoc 2012; DOI: 10.1136/amiajnl-2011-000422.

10.

Arenson

Andriole

Avrin

, et al. Computers in imaging and health care: now and in the future. J Digit Imaging.

11.

Pianykh

OS.

Digital imaging and communications in medicine (DICOM) 2012. DOI: 10.1007/978-3-642-10850-1.

12.

World Health Organization. Classification of digital health interventions v 1.0. WHO Classification, 2018. https://apps.who.int/iris/bitstream/handle/10665/260480/WHO-RHR-18.06-eng.pdf.

13.

Meyer

Groenewald

Pitcher

RD.

Diagnostic reference levels in low- and middle-income countries: early “aLARAm” bells?

Acta Radiol 2017. DOI: 10.1177/0284185116658681.

14.

Shah

Everton

Nordvig

, et al. Increasing access to diagnostic imaging in developing countries: the Asha Jyoti Mobile Clinic. 2010. p. 169581.

15.

Andronikou

McHugh

Abdurahman

, et al. Paediatric radiology seen from Africa. Part I: providing diagnostic imaging to a young population. Pediatr Radiol 2011. DOI: 10.1007/s00247-011-2081-8.

16.

Mohd-Nor

Medical imaging trends and implementation: issues and challenges for developing countries. J Health Inform Dev Ctries 2011.

17.

Alhajeri

Aldosari

Evaluating latest developments in PACS and their impact on radiology practices: a systematic literature review. Inform Med Unlocked 2017. DOI: 10.1016/j.imu.2017.08.005.

18.

Samei

Performance of digital radiographic detectors: quantification and assessment methods 1. 2003; 27710: 37–47.

19.

Doyle

Le Fevre

Anderson

GD.

Personal computer versus workstation display: observer performance in detection of wrist fractures on digital radiographs. Radiology 2005. DOI: 10.1148/radiol.2373041439.

20.

Passadore

Isoardi

Ariza

, et al. Use of a low-cost, PC-based image review workstation at a radiology department. J Digit Imaging 2001. DOI: 10.1007/BF03190346.

21.

Robertson

Saveraid

Hospital, radiology, and picture archiving and communication systems. Vet Radiol Ultrasound 2008. DOI: 10.1111/j.1740-8261.2007.00329.x.

22.

Varma

DR.

Managing DICOM images: tips and tricks for the radiologist. Indian J Radiol Imaging 2012. DOI: 10.4103/0971-3026.95396.

23.

Tingberg

Quantifying the quality of medical x-ray images. An evaluation based on normal anatomy for lumbar spine and chest radiography, Ironmaking & Steelmaking 2000. DOI: 10.1179/174328108X301688.

24.

Ukrit

Umamageswari

Suresh

GR.

A survey on lossless compression for medical images. J Comput ppl 2011.

25.

Eyadat

Muhi

. Compression standards roles in image processing: case study. In: International conference on information technology: coding and computing, ITCC. DOI: 10.1109/itcc.2005.109.

26.

Universitair

Leuven

The impact of information technology on radiology services, 2016. DOI: 10.13140/RG.2.1.4799.9603.

27.

Lee

Peng

, et al. An economical, personal computer-based picture archiving and communication system. Radiographics 1999 Mar-Apr; 19(2): 523–30. DOI: 10.1148/radiographics.19.2.g99mr10523. PMID: 10194793.

28.

Indrajit

Alam

Computer hardware for radiologists: Part I. Indian J Radiol Imaging. 2010 Aug; 20(3): 162–167. DOI: 10.4103/0971-3026.69346. PMID: 21042437; PMCID: PMC2963745.

29.

Yamane

Statistics: An Introductory Analysis, 2nd Edition, New York: Harper and Row, 1967.

30.

European Commission. European guidelines on quality criteria for diagnostic radiographic images. Publication EUR 16260 EN. Brussels, Belgium: European Commission.

31.

Evans

Marom

Godoy

, et al. Radiologists remember mountains better than radiographs, or do they? J Med Imaging (Bellingham) 2016; 3(1), 011005. DOI: 10.1117/1.JMI.3.1.011005

32.

NEC. LCD modules for medical use deliver industry-Leading luminance, http://www.nec.co.jp/press/en/0710/1803.html (2007, accessed August, 2020

33.

McHugh

ML.

Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012; 22(3): 276–82. PMID: 23092060; PMCID: PMC3900052.

34.

Koreen

Gelman

Martinez-Perez

, et al. Evaluation of a computer-based system for plus disease diagnosis in retinopathy of prematurity. Ophthalmology 2007. DOI: 10.1016/j.ophtha.2007.10.006.

35.

Krupinski

EA.

Current perspectives in medical image perception. Atten Percept Psychophys 2010. DOI: 10.3758/APP.72.5.1205.

36.

Doyle

Gunn

MLD

Gamble

, et al. Personal computer-based PACS display system: comparison with a dedicated PACS workstation for review of computed radiographic images in rheumatoid arthritis. Acad Radiol 2002. DOI: 10.1016/S1076-6332(03)80309-4.

37.

Sensakovic

Warden

Hough

MC.

Troubleshooting Image Quality and Other Problems by Using the DICOM Header: RadioGraphics Fundamentals | Online Presentation. Radiographics. 2018 May-Jun; 38(3): 847–848. DOI: 10.1148/rg.2018170057. PMID: 29757720.

38.

Muhogora

Nyanda

Kazema

RR.

Experiences with the European guidelines on quality criteria for radiographic images in Tanzania. J Appl Clin Med Phys 2001. DOI: 10.1120/jacmp.v2i4.2601.

39.

Gharehaghaji

Khezerloo

Abbasiazar

Image quality assessment of the digital radiography units in Tabriz, Iran: A phantom study, J Med Signals Sens 2019. DOI: 10.4103/jmss.JMSS_30_18.

40.

Vreeland

Persons

Primo

, et al. Considerations for Exchanging and Sharing Medical Images for Improved Collaboration and Patient Care: HIMSS-SIIM Collaborative White Paper, Digit Imaging 2016. doi: 10.1007/s10278-016-9885-x.

41.

Anadon

Chan

Harley

, et al. Making technological innovation work for sustainable development. Proceedings of the National Academy of Sciences of the United States of America 2016; 113(35): 9682–9690. DOI: 10.1073/pnas.1525004113.