Augmented Reality and Ultrasonography-Guided Needle Placement: Feasibility and Accuracy in a Phantom Model

Abstract

Objective:

Augmented reality (AR) can improve ultrasonography (US)-guided procedures by placing images in the operator’s line of sight. Published Apple Vision Pro (AVP) US workflows are scarce. This study compared the feasibility and accuracy of an AR US system versus conventional sonography.

Materials and Methods:

This was designed as a prospective crossover feasibility study. The AR system used a Clarius L15 HD linear transducer streamed to an AVP headset and conventional ultrasound equipment system. Participants placed a 22-gauge spinal needle into eight clay targets (e.g., 0.5–2.0 cm) in a gelatin phantom with both systems. The primary outcome was accuracy (i.e., hits/attempts). Feasibility was assessed with an eight item, 5-point Likert-type survey. The data analyses were completed with descriptive output.

Results:

Three radiologists experienced in US guidance achieved 23/24 first-pass hits (i.e., 96%) using AR US guidance versus 24/24 (i.e., 100%) with conventional sonography. Ratings for AR US guidance were high for hand-eye coordination (e.g., 5.0 ± 0.0), image stability (e.g., 4.7 ± 0.6), and overall satisfaction (e.g., 4.7 ± 0.6), with greater ease of use than conventional sonography (e.g., 3.3 ± 0.6). The study limitations included visualization (e.g., 2.0 ± 1.7), headset comfort (e.g., 2.3 ± 1.5), and technical reliability (e.g., 2.7 ± 0.6).

Conclusion:

For this cohort of users, streaming US to AVP, enabled accurate, feasible AR US-guided needle placement with performance comparable to conventional sonography.

Augmented reality (AR) can enhance image-guided procedures by superimposing real-time and multi-modality images onto a patient’s anatomy, through head-mounted displays (HMDs). Reported benefits of AR-integrated ultrasonography (US) include improved spatial awareness, fewer gaze shifts away from the procedural field, lower cognitive load, and faster task completion in both clinical and experimental settings.^1–13

Advances in portable ultrasound equipment systems and next-generation HMDs position medicine at the beginning of a possible AR revolution.^2,3 However, to date, published examples of Apple Vision Pro (AVP) integration with US are sparse. One notable example by Lan et al ¹ described an image-fusion system that integrated laparoscopic video, intraoperative US, and a three-dimensional (3D) liver model directly to a surgeon’s AVP headset.

To the authors’ knowledge, there is no published workflow combining the AVP with a portable ultrasound transducer for experimental or clinical needle guidance. In this study, real-time images were streamed from a portable ultrasound transducer to the AVP and radiologists performed needle placements in a phantom. The objective was to compare the feasibility and accuracy of this AR US-guided system with conventional sonographic guidance.

Materials and Methods

This prospective crossover feasibility study compared the accuracy and feasibility of phantom needle placement performed with portable US streamed to an AVP headset versus conventional sonographic guidance.

The augmented reality system consisted of the Clarius L15 high definition (HD) linear transducer (Clarius Mobile Health, Vancouver, BC, Canada) streamed to an Apple Vision Pro headset (Apple Inc., Cupertino, California) via Wi-Fi Direct Mode. Users could freely position and resize the virtual sonographic display, within the headset. For comparison, the conventional ultrasound equipment system used was an Aplio i900 (Canon Medical Systems, Tustin, California).

The phantom was cast from medical gelatin (Humimic Gelatin #2) akin to subcutaneous fat (~ 2 cm below the dermis), skin, and muscle (Humimic Medical, Greenville, South Carolina). Eight clay spheres, two each measuring 2 cm, 1.75 cm, 1.25 cm, and 0.5 cm in diameter, were embedded as target lesions. The surface was draped to conceal their positions before the procedure. The phantom block measured 20 cm in width, 15 cm in length, and 6 cm in depth. To minimize memorization bias, target locations were randomized between AR and conventional trials but remain consistent for each participant within a given trial. To eliminate visual cues from prior procedures, the phantom block was reheated between participants, effectively removing any visible needle tracks. Needle placements were performed using a 22-gauge spinal needle. The AR system and experimental setup are displayed in Figure 1.

Figure 1.

An augmented reality setup is provided with a targeting phantom. An Apple Vision Pro headset, streaming the Clarius L15 sonography (inset), is shown during needle targeting. A gelatin phantom with embedded clay lesions is provided and was draped to conceal target locations.

Board-certified radiologists, each with additional training in musculoskeletal radiology and experience in ultrasound-guided procedures, were invited to participate and provided verbal assent. As this project used only a phantom model, no formal ethics review was required, nevertheless, the work was conducted in accordance with the principles of the Declaration of Helsinki.

Prior to performing needle placements, participants received a brief 5-minute orientation on connecting the Clarius Probe to the Apple Vision Pro and adjusting the depth of field. Each participant performed a single needle pass on all eight lesions. Participants were informed that eight clay spheres of varying sizes were embedded within the phantom and were allowed to locate and place a needle within the targets in any order. The two guidance sessions (e.g., AR system and conventional sonography) were completed. A needle placement was considered a “hit” if the needle intersected the target. It was not possible to anonymize the operators to the setup, as they were aware of the transducer/technology being used.

Data Analysis

The primary outcome, accuracy, was defined as the proportion of successful hits to total needle attempts. Feasibility outcomes were assessed with an eight item, 5-point Likert-type questionnaire (See Appendix 1) covering usability, screen stability, hand eye coordination, visualization clarity, discomfort, confidence for clinical use, and overall satisfaction. Results are reported descriptively as mean ± standard deviation (SD), and no formal hypothesis testing was undertaken given the feasibility scope.

Results

Three board-certified radiologists experienced in US-guided procedures participated. Using the AR system, they were able to hit 23 of 24 targets (i.e., 96%) on the first attempt; with conventional sonography and they hit all 24 targets (i.e., 100%). The lone miss occurred on a 0.5 cm lesion during AR guidance. Results from the feasibility survey are presented in Figure 2.

Figure 2.

A bar chart is provided of the feasibility survey results, based on the three radiologist involved in the study.

Discussion

In this feasibility study, three radiologists achieved near-equivalent needle placement accuracy on a phantom using AR US guidance with the Apple Vision Pro (e.g., 23/24 hits; 96%) and conventional sonography (24/24 hits; 100%). This minimal difference likely reflects a ceiling effect among experienced operators, making system performance gaps difficult to detect. ¹ Future work could involve novice users or incorporate smaller, deeper, or otherwise more challenging targets to better discriminate between systems. Additionally, repositioning the conventional US screen could help simulate the spatial constraints often encountered in procedural or operating rooms, where limited floor space may restrict optimal screen placement.

User feedback was largely favorable. Participants rated hand-eye coordination and image stability as excellent, expressed high overall satisfaction, and felt confident they could apply the system clinically after a single session. They also reported the AR system was slightly easier to use than conventional sonography for the task. These findings suggest immersive visualization with AVP could enhance selected US-guided procedures, particularly in scenarios where limited space restricts the placement of conventional ultrasound machines or when the screen is positioned too far away from the user to clearly resolve anatomical details. Key critiques included limited visualization clarity, headset discomfort, and intermittent technical glitches.

These shortcomings warrant further refinement but may be mitigated with extended training and user familiarity. The reduced image clarity may stem from the enlargement of the virtual display, as the Clarius Ultrasound App is optimized for mobile devices and was scaled up in the AR environment. This limitation likely reflects a need for app-level optimization rather than an inherent issue with the augmented reality system itself.

To the authors’ knowledge, only one other study has assessed the AVP for ultrasound guidance. During microwave ablation of hepatic hemangiomas, a fused display of laparoscopic video, intraoperative US, and a 3-D liver model was projected into the surgeon’s field of view. ¹ Compared with a conventional sonographic setup, AVP increased junior surgeons’ target hit rate from 15% to 80%, experienced surgeons’ from 70% to 100%, and overall accuracy from 43% (i.e., 17/40) to 90% (i.e., 36/40). ¹ NASA Task Load Index (NASA-TLX) scores also improved across all domains, indicating lower mental, physical, and operational workload, reduced frustration, and enhanced task performance. ¹

Other non-AVP AR US systems have shown similar promise. In a phantom study of central venous catheterization, the Foresee-X Smart Surgical Glasses reduced cognitive load and improved procedural metrics versus standard sonography. ⁴ HoloLens 2 guidance produced more precise needle trajectories in breast biopsy phantoms. ⁵ With the HoloLens US system (i.e., a novel AR US application), experts completed ultrasound-guided vascular access tasks faster and with fewer head movements while maintaining accuracy. ⁶ Even between AR modalities, upgrading from a two-dimensional (2D) to a 3D AR US display decreased puncture time by 28%. ⁷ AR US has also demonstrated ergonomic benefits compared with conventional 2D systems in simulation settings. ¹³

Limitations

This feasibility study is limited by its small sample size; however, feedback from three knowledgeable operators adds practical value. Furthermore, since testing was conducted with a phantom, the observed performance and user perceptions may not translate to real patient procedures, ¹² though this approach was useful to identify system shortcomings.

The next steps will focus on improving visualization clarity and technical reliability, although longer, more structured training (e.g., such as practice with display placement) could facilitate additional gains. ¹¹ Future work could chart the learning curve for AR-guided versus conventional ultrasound-guided tasks in larger, mixed experience cohorts within controlled educational settings. Pilot studies in point-of-care and high-throughput biopsy workflows will further clarify the system’s clinical utility.

Conclusion

In this cohort, streaming portable US to the Apple Vision Pro enabled accurate AR US-guided needle placement in a phantom, with near-equivalent accuracy to conventional sonography, among experienced operators (e.g., 96% vs 100%) and favorable usability. With further refinement and clinical validation, this approach may benefit point-of-care settings, high-throughput biopsy workflows, and selected surgical applications (e.g., hepatobiliary).

Supplemental Material

sj-pdf-1-jdm-10.1177_87564793261418020 – Supplemental material for Augmented Reality and Ultrasonography-Guided Needle Placement: Feasibility and Accuracy in a Phantom Model

Supplemental material, sj-pdf-1-jdm-10.1177_87564793261418020 for Augmented Reality and Ultrasonography-Guided Needle Placement: Feasibility and Accuracy in a Phantom Model by Noah Tregobov, Tony Jiang, Philip Edgcumbe, Abdulaziz Alhuwaymil, Jin Hui Yan, Afnan Aludaini, Peter L. Munk, Hugue Ouellette, Paul Mallinson and Adnan Sheikh in Journal of Diagnostic Medical Sonography