Abstract
Background:
Arteriovenous fistulae (AVF) are considered the preferred hemodialysis access but up to 50% of all AVF created in the United States never mature. Doppler ultrasound (DUS) is useful for predicting fistula maturity and impending fistula failure. DUS is resource-intensive and is associated with poor compliance rates in dialysis patients, ranging from 12% to 33%.
Methods:
EchoSure is an FDA-cleared 3D Doppler ultrasound device that automatically delivers quantitative blood flow and anatomic vascular information. The technology can be used at the bedside by personnel without formal sonographic training, nullifying limitations of traditional Duplex ultrasound imaging. This study compared the EchoSure system in the hands of inexpert personnel to a traditional expert-operated DUS for rapid assessment of a benchtop model vascular system with flow, diameter, and depth expected in a human AVF.
Results:
Both Duplex and EchoSure performed within the expected tolerance of ultrasound readings (35%) for volume flow, with the average error (AE) between the observed measurement and the ground truth being 8% for Duplex and 8% for EchoSure. However, the average coefficient of variation (CV) for Duplex pooled over all flow rate measurements was 17% versus 4% for EchoSure. Regarding diameter, Duplex measurements had AE of 15% with an average CV of 6% across all measurements versus EchoSure AE of 4% and average CV of 2%. Duplex and EchoSure measurements over all depths had the same AE of 2%. The two modalities were not statistically different for depth measurement (p = 0.05) but EchoSure measured closer to the ground truth for flow rate and vessel diameter (flow: p = 0.028, ρ = −0.07; diameter: p < 0.001, ρ = 0.69). The inexpert personnel using EchoSure acquired data 62% faster than the expert sonographers using the Duplex ultrasound (141 min for Duplex vs 87 min for EchoSure).
Conclusions:
EchoSure may offer an accurate and convenient alternative for imaging fistulas in the clinic.
Introduction
Management of vascular access remains one of the most important aspects of care for patients with end stage renal disease (ESRD) undergoing maintenance hemodialysis (HD). A mature and functioning arteriovenous fistula (AVF) is preferred over all other access modalities because it is associated with the best access survival.1–3 This benefit depends on successful maturation of the access, which typically takes a median time of 6-9 months, but can be considerably longer if interventions are required to assist maturation.3–5 Further, approximately 30%–50% of AVFs in the United States fail to mature, requiring abandonment after 6–12 months and extended catheter dependence.6,9 In turn, prolongation of catheter dependence time is associated with higher risks of infection, thrombosis, and mortality as well as higher overall cost.1,7,8
Ultrasonography is a well-known modality widely used to assess volumetric flow within a fistula and determine anatomic factors that may pose a risk to maturation failure. Doppler ultrasound imaging has demonstrated utility in predicting both fistula maturity and in detecting impending fistula thrombosis.9–18 By using previously established criteria for the prediction of maturation failure, ultrasound can provide useful information for determining the cause of non-maturation and facilitate planning for intervention. 9 Studies indicate that routine surveillance of fistulae can predict maturation and that a program of routine ultrasound surveillance decreases the rate of maturation failure significantly.9–11,15 A large, prospective multicenter study determined that fistula diameter and volumetric flow at two weeks postoperatively were significant predictors of successful or unsuccessful maturation at 6 weeks. 11 Proposed minimum ultrasound criteria for AVF maturity at 4 weeks include vessel diameter of 4–5 mm and blood flow over 400–500 mL/min.11,19–23 Use of ultrasound data can improve fistula assessment specificity from 21% with physical examination alone up to 90% or more.20–22
Duplex ultrasound (DUS) can be a resource-intensive practice. Patients are typically required to make an additional visit to an outpatient imaging laboratory, introducing both time and financial expense. Image acquisition requires a trained, formally certified professional ultrasonographer. The study itself lasts 45–60 min, after which compulsory image interpretation by a physician is performed after an additional 24–48 h. The substantial commitment of material resources, time, and personnel make formal Duplex ultrasonography ill-suited for performance of frequent examination, which is reflected in real-world practices that show significant underutilization of DUS. This offers an opportunity to improve clinical care given the well documented clinical and quality benefits that regular scanning yields. 10
Ultrasound surveillance does not necessarily require the extensive data acquisition and analysis Duplex ultrasonography is capable of. Previous work has demonstrated that a single variable, volumetric flow rate, can effectively predict both AVF maturation and thrombosis.10,12,16–18,23 Thus, the potential for regular ultrasound surveillance to decrease rates of AVF maturation failure and thrombosis may not require comprehensive examinations performed by certified expert sonographers. However, duplex flow measurement is operator dependent. When compared to actual flow rates, volume flow measured by trained sonographers can deviate from the true volume flow by up to 35%. 24 Sources of error include algorithm assumptions about vessel circularity, accurate probe positioning in the middle of the vessel, and accurate user placement of key measurements such as calipers for diameter and other parameters such as gate width and angle.
EchoSure is an FDA-cleared 3D Doppler ultrasound device that automatically delivers visual and quantitative blood flow and anatomic vascular information and may be utilized postoperatively from the bedside by a nurse, dialysis technician, or other healthcare provider without the need for special sonographic training, bypassing these limitations of traditional Duplex ultrasound imaging. EchoSure has received 510(k) clearance from FDA with general indications for measuring blood flow information for vessels marked with EchoMark, which is an echogenic, bioresorbable fiducial marker to facilitate ultrasound measurements. The EchoMark is implanted at the time of surgical fistula creation and has an expected absorption timeline of 18–24 months.
This study used the EchoSure system, in the hands of inexpert personnel (i.e. nurses and dialysis technicians rather than sonographers) for rapid ultrasound assessment of a benchtop model vascular system with flow, diameter, and depth ranges expected to be seen for a maturing fistula. Representing an ideal-world control, expert sonographers (Registered Vascular Technologists, or RVTs) performed standard of care Duplex ultrasound examinations of the same model system using conventional ultrasound technology.
We sought to compare the measurements in terms of accuracy and precision, both for inter- and between-observer differences for volumetric flow rate, diameter, and depth. We hypothesized that EchoSure examination by ultrasound naïve hemodialysis professionals would be non-inferior to traditional examination by trained sonographers in a benchtop AVF model.
Material and Methods
Study design and participants
This study was deemed IRB-exempt (Solutions Institutional Review Board, Protocol #2020/07/14) and participants provided appropriate informed consent for involvement. The design consisted of a head-to-head, prospective comparison of EchoSure in the hands of non-expert personnel, such as dialysis technicians and registered nurses, to the gold standard, Duplex ultrasonography performed by trained, certified vascular sonographers. All participants were recruited from academic and private clinics in the area with deliberate effort made to formulate a cohort of participants that represented the diversity of practice settings. All participants were assigned a unique identification number that was used to follow data acquired throughout the study. The identity of all participants was kept anonymous.
The a priori power analysis showed that a minimum of 12 participants in both the expert and inexpert arms of the trial would be needed. Inclusion criteria for participants include the following: (1) practicing registered nurse or other practicing healthcare personnel (e.g. dialysis technician), or practicing vascular ultrasonographer who has performed Duplex examinations in the preceding 12 months; and (2) ability and willingness to attend all study sessions. Exclusion criteria included the following: (1) age <18 years; (2) unable or unwilling to sign consent forms; (3) known latex allergy that prevents the participant from using the benchtop model.
The study was composed of two parts. The first part, aimed at determining the accuracy of EchoSure, required measurements to be made by the EchoSure apparatus and the data analyzed in a summative fashion. The second part featured identical measurements made by Duplex ultrasonography (gold standard).
Experimental apparatus and data acquisition
The Duplex ultrasound measurements were made using a Philips CX50. The EchoSure platform is a portable semi-automated diagnostic 3D ultrasound imaging device that includes the EchoSure Ultrasound, the EchoSure Application (software) and the EchoSure Probe. The EchoSure Probe is a mechanized linear transducer that can conduct volume acquisition of ultrasound data, such that the user simply needs to place the probe in the general region of the target rather than in a precise location or orientation. These features enable health care personnel without ultrasound experience (e.g. dialysis technicians and nurses) to quickly and reliably position the EchoSure probe for a successful scan without consideration of technique. The EchoSure Application is designed with a simple user interface and workflow that walks the user through the scanning process with step-by-step instructions. Ultrasound parameters are automatically optimized to further reduce the need for user input requiring ultrasound expertise or understanding of ultrasound physics. For example, a conventional ultrasound requires: precise positioning of the transducer aligned to the center of the vessel, precise orientation of the transducer to 60° from the vessel, precise gate setting within the wall of the vessel for velocity measurements, precise cursor placement for diameter measurements, and tuning of various parameters such as pulse repetition frequency; each of these steps is subject to user error and has a major impact on the resulting measurement, With each scan, EchoSure automatically calculates and reports volumetric flow, diameter, and depth of the fistula. Flow measurements require previous placement of an FDA-cleared implantable, echogenic, bioresorbable fiducial marker (“EchoMark”) underneath the vessel of interest. As part of the FDA regulatory clearance process, the accuracy of the EchoSure algorithms for flow measurement underwent extensive validation and verification.
The experimental setup was composed of a flow phantom of tissue-mimicking material with a segment of natural latex tubing to mimic a blood vessel, a flow pump capable of providing both continuous and pulsatile flow using a circulating blood-mimicking fluid, an ultrasound machine, and a transducer. The flow phantoms used for EchoSure testing also had an implanted EchoMark device. The model allowed for manual adjustment of several variables: volumetric flow rate, vessel diameter, and vessel depth. The EchoSure and gold standard Duplex ultrasound platforms collected measurements on various permutations of these variables in the benchtop model. The exact experimental schematic for experimental parameters is seen in Table 1. Each measurement was made five times.
Schematic representation of clinically relevant scenario parameters for data acquisition for evaluation of EchoSure and Duplex ultrasonography in the arteriovenous fistula benchtop model.
To better illustrate, consider an example participant. This individual would make EchoSure measurements of each scenario 1–12 in Table 1, with each scenario being measured five times total. This would yield a total of 60 measurements for this participant.
All study measurements were timed. At the conclusion of data acquisition, participants were asked to complete a questionnaire that included information about years of relevant clinical experience and typical clinical volume of dialysis patients cared for.
Data analysis
The study results were analyzed by both descriptive and inferential statistics. Standard descriptive statistics for numerical summaries are the number of observations (n), mean, standard deviation, median, minimum, and maximum values.
The first part of the study assessed the accuracy and precision of measurements made by inexpert personnel using the EchoSure system. A repeated measures model was used to assess the precision of the inexpert personnel against the actual flow values. The second part of the study repeated this same procedure but using certified vascular ultrasonographers and standard Doppler ultrasound platforms.
The sample size of 12 expert and 12 inexpert users was selected for this study in order to provide additional power for the second comparison between user types for the flow rate of 1500. A sample size of 12 per group provided 80% power to detect a difference of 300 (20% of 1500) assuming the standard deviation for the EchoSure users is 89 and the standard deviation for expert users is 317 based on preliminary testing. Each user collected 15 samples at each flow level which would provide adequate power (two-group Mann-Whitney U test, α = 0.05, β = 0.9, µ0 = 1500, µ1 = 1800, σ1 = 317, σ2 = 89). With a sample size of 12 EchoSure users and the same assumptions for the 1500 flow rate indicated standard deviation of 89, the study had greater than 99% power to demonstrate the mean difference is less than 300 using a single sample t-test. Hence, the study should have sufficient power to consider each flow rate separately.
Lastly, we sought to conduct a more rigorous comparison to account for the correlation of results within participants and possibility of measurement error being different at high or low flow rates. We therefore developed a mixed effects linear regression model to account for these factors. We compared the percent error from true flow for Duplex and EchoSure. The true flow and measurement modality were included as fixed effects while individual rater was included as a random effects factor. We also examined percent error within each category of flow (i.e. 250, 500, 750, 1000). All statistical analysis was performed using STATA/MP 14.0 (StataCorp Inc., College Station, TX).
Results
The study included 25 participants with 13 dialysis clinicians performing EchoSure scans and 12 sonographers performing Duplex ultrasound examinations. Each participant performed 60 examinations and each examination obtained four data points: depth, diameter, volumetric flow rate, and diameter used to calculate volumetric flow rate. This yielded approximately 5000 data points, all of which were available for computation and statistical analysis.
EchoSure measurements versus true flow rates
The first portion of the study concerned with measuring the accuracy and precision of the EchoSure device demonstrated a high degree of concordance between the EchoSure measures and actual values of depth, diameter, and volumetric flow rates (Table 2). Across all measurements for flow rate, diameter, and depth, the observed measurements fell easily within 20% of the actual value.
Summary data results for validation of the EchoSure device.
CV: coefficient of variance; SD: standard deviation.
EchoSure versus duplex ultrasonography
When comparing Duplex and EchoSure to true values, both Duplex and EchoSure performed within the expected tolerance of ultrasound readings, which has been cited by experts as up to 35% real world variability based on operator and patient specifics (Figure 1). 24 In addition to passing the criteria of <20% variation across all flow rate measurements, Duplex and EchoSure showed similar performance accuracy (Table 3). The average error between the observed measurement and the ground truth was small for Duplex and EchoSure (Table 4). While the average error was equivalent across both modalities, Duplex measurements had a much larger standard deviation than EchoSure measurements. The average coefficient of variation for Duplex pooled over all flow rate measurements was 17% (Table 3), while the same metric for EchoSure was only 4% (Table 2).

Visual representation of the proportion of measurements falling within a given percentage error from the true, pump-measured flow rate (ground truth). The horizontal axis shows the percentage error and the vertical axis provides the proportion of measurements falling within the given percentage error. Essentially all measurements obtained with EchoSure were within 20% of the true flow rate (the pre-defined clinically acceptable threshold).
Summary data results for measurements made by duplex ultrasonography.
Mixed-model regression analysis looking at true flow versus measured flow. Variation was controlled for across individual raters (random-effects factor).
Flow measurement error reaching significant difference from true flow rate (p < 0.05).
Regarding diameter, Duplex measurements had an average error of 15% with an average coefficient of variation of 6% across all measurements, while EchoSure had an average error of 4% with an average coefficient of variation of 2%. Duplex and EchoSure measurements over all depths had the same average error of 2%. Duplex had an average coefficient of variation of 5%, while EchoSure measurements had an average coefficient of variation of 2%. Overall, there was less than 10% variation between Duplex and EchoSure across all key measurements (flow, diameter, and depth). The two modalities were not statistically different for depth measurement (p = 0.05). However, when a significant difference was present, as it was for flow rate and vessel diameter, EchoSure was closer to the true value (flow: p = 0.028, ρ = −0.07; diameter: p < 0.001, ρ = 0.69). This did not meet criteria to suggest a clinically meaningful degree of discordance between Duplex and EchoSure when compared head-to-head.
Regression model flow estimation results
Ultrasound underestimated true flow by an average of 3.2% (95% CI: 8.1% underestimate to 1.8% overestimate; p = 0.21) and EchoSure underestimated by an average of 2.1% (95% CI: 8.2% underestimate to 2.9% overestimate; p = 0.49). EchoSure was, on average, 1.0% higher than ultrasound (95% CI: 6.8% below to 8.8% above). Ultrasound significantly overestimated flow at a true flow of 250 mL/min (p = 0.04) and significantly underestimated flow at a flow rate of 750 (p = 0.004) and 1000 mL/min (p < 0.001), whereas flow measured by EchoSure did not significantly differ from the true flow for any of the tested flow rates.
Technician experience and acquisition time
Finally, there was no association between the experience of the scanner and the percent error of measurements for either DUS or EchoSure (Table 5). The inexpert personnel using the EchoSure platform were able to acquire data significantly faster than the expert vascular sonographers using the Duplex ultrasonography equipment (Table 6), with Duplex taking 62% longer than EchoSure to perform the scans (141 min for Duplex vs 87 min for EchoSure for collecting 60 scans).
Linear regression analysis results analyzing the effect of technician experience (in years) on percent error in data acquisition.
Analysis of the amount of time taken for data acquisition of 60 data sets with EchoSure and Duplex ultrasonography.
Discussion
This prospective, head-to-head comparison study of a novel platform for automated AV fistula assessment by dialysis clinicians versus traditional Duplex ultrasonography for evaluation of vessel depth, diameter, and flowrate on a benchtop flow phantom model demonstrated that EchoSure was at least as precise and accurate as the gold standard modality, Duplex ultrasonography. This result was seen across flow rates ranging from 250 to 1000 mL/min. In fact, the EchoSure system tended to produce individual readings that more closely represented the actual values than did the conventional Duplex equipment.
The first element of the study demonstrated that the EchoSure is both accurate and precise in the hands of a representative dialysis clinic user, such as a registered nurse or a dialysis technician. As seen in Table 2, for all parameters measured, the coefficient of variability was less than 4% and the percent error less than 13%. We established 20% as cutoff for clinically meaningful percent error. Previous reports have shown up to a 35% variation in Duplex-measured versus actual flow, so a more challenging threshold was selected to account for possible additional error in clinical settings. 24 EchoSure produced measurements in all test runs well within that cutoff.
In the second part of the study, EchoSure was compared to the gold standard, Duplex ultrasonography performed by a certified vascular sonographer. The data obtained by the Duplex platform was also accurate and precise, with all the mean measurements being less than or equal to 18% error (Table 3). The magnitude of the percent error and coefficient of variation were equal to or greater for the Duplex system compared to the EchoSure system across all tested parameters. The EchoSure platform produced a much narrower range of volumetric flow measurements than the Duplex ultrasound setup (Figure 2). We rigorously characterized this comparison for volumetric flow rate and found that the percent error for Duplex ultrasound, at high flow rates, was significantly different than the actual value, whereas EchoSure had no significant difference from the actual value at any flow rate (Table 4). This difference may be attributed to the manual nature of Duplex measurements and the high impact that small differences in diameter or angle measurements have on the resulting output measurements.

Graphical demonstration of the measured ranges for volumetric flow rates in the benchtop model with data acquired by EchoSure in blue and those acquired by Duplex ultrasound in red. The solid, diagonally oriented black line represents perfect measurement and the paired, dashed red lines indicated the pre-ordained 20% error limits. A very high proportion of EchoSure measurements reside within the dashed red lines, moreso than for the data points representing Duplex ultrasound measurements.
Interestingly, in addition to providing data that is at least as precise and accurate as the gold standard, the EchoSure platform enabled inexpert personnel to acquire the relevant data points in a significantly shorter period of time, approximately 40% faster, on average. This excludes the substantial additional post-Duplex time for study saving, uploading to PACS, and clinician interpretation and generation of a full report, which is done in real-time on the EchoSure platform.
Our data suggests that Duplex ultrasonography is most useful when serial measurements are made and a representative value (e.g. the median or mean) is used instead of a single measured value. This phenomenon was notably not observed in the datasets produced by the EchoSure platform, in which individual measurements were more likely to better represent the ground truth; this reached statistical significance in the benchtop models of 750 and 1000 mL/min. The semi-automated nature of the EchoSure platform’s data acquisition, both in terms of the mechanized transducer array and artificial intelligence-guided vessel identification, may outweigh the stochastic effects introduced by a human sonographer, regardless of the sonographer’s experience or skill level.
Our results also suggest important considerations for the interpretation of Duplex ultrasound data. Pooled overall flow measurements showed no significant variation from true flow. However, the analysis at each flow rate showed that DUS tended to overestimate flow at low flow rates and underestimate flow at high flow rates. EchoSure did not show this bias. However, this should be placed in context insofar as DUS was still generally well within our pre-defined 20% acceptable error level.
This report is of interest to clinicians responsible for the creation, maturation, and assessment of AV fistulae, which includes vascular surgeons, interventional radiologists, medical and interventional nephrologists, and primary care physicians. Previous literature has convincingly shown the value of ultrasonography in improving the probability of successful maturation of an AVF.9–11,15 Similarly, compliance with AVF follow-up to objectively assess maturation is an important aspect of timely maturation to enable recognition of non-maturation and to facilitate proper permission to cannulate.25,26 However, the process of acquiring Duplex ultrasound information is far from easy and limits its routine use in this patient population. Patients must invest hours of their time and navigate complicated transportation logistics to and from specialty radiology centers or vascular labs which may be far from home. 27 The examination requires uniquely credentialed sonographers and physicians to perform and interpret the studies, which often do not result for several days after the exam itself takes place. Due to these challenges, dialysis patients are not compliant with visits outside of life-sustaining dialysis sessions (i.e. maturation assessment, ultrasound examinations with trained Vascular Technologists). 25
As a result, real-world Duplex compliance rates are remarkably poor, ranging from 12% to 33% even when a dedicated liaison is used to encourage compliance. 27 The costs involved can also be prohibitive, especially for large, public payers. 28 This is likely a causative factor explaining why representative national database studies such as USRDS consistently indicate that most fistulas are not used for several months after prospective cohort studies on maturation timelines would suggest they should be. This extended catheter contact time is a significant contributor to ESRD patient morbidity and mortality on a national population level.
This study has several notable limitations. This is a bench-top model of AVFs rather than a clinical study and additional clinical validation is therefore necessary. The benchtop model used in the experiment is novel and has not been validated as correlating to clinical AVF outcomes. Despite these limitations, EchoSure seems to have potential as a tool to acquire critical AVF decision-making data in the dialysis unit and offers several advantages over DUS. For patients, the examination time is much shorter and does not require extra appointments and the associated travel and financial burden. For providers, the system obviates the need for a trained vascular ultrasonographer, instead empowering practicing nurses, dialysis technicians, and other similar professionals to safely and effectively acquire data. Perhaps most importantly, the system also presents real-time data that is available to the physician making patient management decisions at the dialysis bedside.
In summary, we conducted a benchtop experiment to compare the accuracy and precision of the EchoSure platform in the hands of non-sonographers to that of gold-standard Duplex ultrasound. EchoSure was as or more accurate, showing less variable measurements at higher flow rates. Additionally, EchoSure allowed collection of the same data in a much shorter period of time. The EchoSure platform may offer a safe, accurate, and convenient alternative to Duplex ultrasonography for imaging fistulas in the dialysis clinic. Clinical studies to more rigorously define the role of EchoSure in the care of ESRD patients with AVF are warranted.
Footnotes
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Ross is a member of the Medical Advisory Board for Sonavex, Inc. The other authors have no relevant financial disclosures.
Funding
Research reported in this publication was supported by the National Heart, Lung, And Blood Institute of the National Institutes of Health under Award Number R44HL147423. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
