Autonomous Medical Officer Support Software Technology Demonstrations on the International Space Station

Abstract

Introduction:

Performance of medical procedures in spaceflight beyond low Earth orbit (LEO) requires novel solutions to replace real-time ground support, which is hampered by growing communication latencies as distance from Earth increases. The Autonomous Medical Officer Support (AMOS) Software Technology Demonstrations on the International Space Station (ISS) trialed a novel software tool that shifts the emphasis from pre-flight training and real-time remote guidance (current ISS paradigm) to a new standard of multidimensional in-flight just-in-time instruction. The AMOS platform is a skill management tool for all mission phases and currently features comprehensive training and guidance modules for urinary bladder and renal ultrasound examinations.

Methods:

Using the AMOS software, two ISS crewmembers performed in-flight bladder and kidney ultrasound examinations with no guidance from the ground and with no pre-flight exposure to the software (n = 2 sessions). Images were graded for clinical quality using a strict evaluation rubric. Software use patterns were recorded, and participants provided both structured and free response feedback.

Results:

Clinically adequate images were obtained for all but one of the attempted views. Participants rated AMOS as highly useable and primarily used linear navigation.

Conclusion:

Variability in subject anatomy, operator experience, and operator receptiveness to instruction during autonomous examinations is a persistent but manageable limitation. Despite these challenges, here we report the first successful demonstrations of autonomous imaging activities in the operational setting of spaceflight, validating this autonomous guidance proof-of-concept.

Introduction

Driven by the need to support unique medical demands of spaceflight, National Aeronautics and Space Administration (NASA) has pioneered the field of telemedicine,^1,2 gradually expanding the scope to include teleultrasound, ³ which has become of significant interest on Earth as well.^4,5 Current International Space Station (ISS) medical operations rely heavily on real-time remote guidance from Earth, which becomes impractical with communication latencies of exploration class missions. ⁶ As previously envisioned, ⁷ performance of medical procedures during these missions will require an autonomous solution to replace real-time ground support of Crew Medical Officers (CMOs).^8,9 The Autonomous Medical Officer Support (AMOS) software tool enables independent training and guidance for all crew, shifting from pre-flight training and remote guidance to in-flight just-in-time (JIT) instruction.^9–11

Autonomous guidance concepts and software design for AMOS were developed and validated by the multidisciplinary team for the ground-based Clinical Outcome Metrics for Optimization of Robust Training (COMfORT) study. ¹² The COMfORT application was subsequently rebuilt for ISS server compatibility and modular expandability and renamed AMOS. The AMOS tool contains two pilot modules (ultrasound imaging of urinary bladder and kidneys), which, in contrast to COMfORT modules, guide users through comprehensive examinations designed to collect clinically useful imaging. ¹³ Menu-driven software is organized by modules, sections, chapters, and pages, allowing both topic-directed and linear navigation, enabling users to complete complex tasks autonomously with minimal training. The platform includes integrated use tracking and evaluation features to analyze usage patterns and gather feedback.

In contrast to the current practice of generating separate products for (1) pre-flight training, (2) in-flight JIT training, and (3) procedure execution, the AMOS platform is a single tool for all mission phases of skill management, covering all training aspects and procedural applications (Fig. 1). AMOS also provides a streamlined process where multiple levels of autonomy are supported, making it ideal for deployment on the ISS as an exploration analog since it is amenable to incremental integration into operations or research. Modular design enables new content modules to be added without new software certification and is extensible to other applications beyond medical content (e.g., engineering or maintenance).

Fig. 1.

The current ISS training system uses three distinct products during training flow: (1) pre-flight training materials, (2) in-fight refresher training, and (3) in-flight procedures. Pre-fight training is 7–16 h and occurs 6–18 months pre-flight; crew medical officers (CMOs) receive an additional 4 to 5 h of specialized training. The proposed paradigm for exploration uses a single software tool (AMOS or similar) to perform all three functions, streamlining training workflow. With the proposed paradigm, trainees are familiarized with the software and trained only on select procedures that impart highly transferable skills. AMOS, Autonomous Medical Officer Support; ISS, International Space Station.

Operationally beneficial features of the AMOS platform include (1) extending telemedicine capability, (2) enabling autonomous procedure performance, (3) streamlining the process of training and skill retention, (4) reducing pre-flight training load, and (5) reducing risks from specific medical conditions by having reliable in-flight autonomous monitoring and early diagnostic and follow-up capabilities.^14,15 The AMOS platform has potential to enable new paradigms for training, skill development and retention, and on-demand performance for medical surveillance or high-priority procedures during exploration missions.

While designed to be used in full autonomy on Mars missions with high latency, this tool could be equally beneficial in low Earth orbit (low latency) or the lunar vicinity (moderate latency) since it can also be used as JIT training or a remote guidance companion. This article describes demonstrations by ISS crewmembers that confirm the effectiveness of the AMOS tool in this operational environment and its relevance to progressively Earth-independent medical operations for exploration-class spaceflight missions. ¹⁶

Methods

The AMOS project was a collaboration between KBR, NASA, and RKT Creative (Detroit, MI). The KBR/NASA team provided the structure, content, and user interface requirements, while RKT Creative was responsible for design and coding. These efforts align with NASA Standard 3001, which requires that capabilities for diagnostic and procedural imaging be available for in-mission medical care.^17,18

AMOS SOFTWARE

AMOS software is an extensible markup language-based training and procedure guidance tool that runs in any web browser; it was beta tested terrestrially with novice ultrasound users to optimize user experience for astronauts. Additional testing was conducted with NASA personnel and a physician astronaut to further improve clarity of instruction. The final version included two comprehensive ultrasound scanning modules: one for urinary bladder and one for bilateral kidney examination. AMOS was installed on the ISS server system in April 2020 and is accessible from any space station computer through a desktop shortcut that participants used during study procedures. The intended use of AMOS is full autonomy during missions with extreme latency or communication failure; therefore, no communication links are built in for flight surgeons or other experts to assist during flight.

PARTICIPANTS

The AMOS technology demonstration (Tech Demo) for ISS was deemed exempt by the NASA Institutional Review Board. The demonstration required two crewmembers (astronauts): an operator to deploy hardware and perform ultrasound examinations with guidance of AMOS, and a subject “volunteer patient” (Fig. 2). On three separate occasions, operator and subject pairs of ISS crewmembers participated. Since the Tech Demos are developmental proof of concept, ISS crew are not operationally trained on AMOS software; none of the participants had pre-flight training for these procedures, nor did they have prior experience using AMOS. However, we envision that future operational use would include pre-flight familiarization with the platform. Results from one pair of crewmembers were not included in data analysis due to significant protocol deviation in conducting critical initial instruction review.

Fig. 2.

Drs. Jessica Meir (operator) and Drew Morgan (subject) conduct an AMOS technical demonstration on the ISS during Expedition 61. (Photo courtesy of NASA; ISS062E140375). AMOS, Autonomous Medical Officer Support; ISS, International Space Station; NASA, National Aeronautics and Space Administration.

PROTOCOL

The day before each Tech Demo, the operator reviewed instructions for the activity (10 min) and foundational instruction of each of the AMOS modules (30 min total), which convey basic principles of ultrasound scanning and anatomy. The subject reviewed only instructions (10 min). AMOS modules guided the operator through steps to acquire and save ultrasound imaging scans of target organs. Ultrasound images were collected using the ISS Ultrasound 2 system-Vivid q with a 4 C-RS broadband curved array transducer (GE Health care, Milwaukee, WI). The operator collected cine-loop scans of full and empty urinary bladder in two planes and 2D and color Doppler scans of both kidneys. The ideal scanning order begins with the technically simpler bladder examination to build a basic mental model of ultrasound scanning before engaging in more technically complex renal scans. Therefore, the recommended scan order was full bladder, empty bladder, right kidney, then left kidney. Setup and configuration were allocated 30 min, with all scans scheduled for one session (70 min for the bladder and 90 min for both kidneys). Participants also completed a nine-item questionnaire within each module. Following scans, imagery was transferred to the ISS Human Research Facility laptop for later downlink. Operator and subject astronauts did not have real-time contact with the AMOS Team or Mission Control Center for assistance or clarification on AMOS content for either demo.

ULTRASOUND AND VIDEO ANALYSIS

Each demo was observed remotely via ultrasound and cabin video streams, and was recorded for later technical and human factors analysis. Images were downlinked then examined by a trained sonographer and rated for image quality and clinical utility against a rigorous, multicomponent rubric. Image components were rated from 0 to 3, with 0 being of lowest quality/utility and 3 being highest. A score of 2 was set as the threshold for clinical adequacy on the scoring rubric (Supplemental File A).

CLICK TRACKING AND SURVEYS

AMOS tracked operator usage patterns by logging inputs into an exported .csv file. Surveys regarding software usability and procedure quality were built into the modules and completed during the session to reduce recall bias. Demo 1 crew provided a postflight debrief session 48 days after the scan session; due to crew time constraints, a debrief session was not conducted for Demos 2 or 3. Summary of data used in the data review for Demo 1 appears in Supplemental File B.

Results

ULTRASOUND IMAGING

For each demo, one or more clinically adequate images (score of 2 or better) were collected for every view (with one exception, discussed below), and in most cases the average success score met or exceeded the quality threshold (Fig. 3). 2D bladder images were rated highest, followed by 2D kidney, and finally, color Doppler kidney. This follows the level of complexity for each examination, with color Doppler of the kidney being most complex due to the need to manage Doppler parameters in addition to the 2D image. Quality scores demonstrated success of all imaging studies, confirming autonomous training and guidance was achieved using AMOS by crew members with no prior training on these examinations.

Fig. 3.

Success scores (0–3) for bladder (Panel A) and kidney (Panel B) examinations. Averages of scores for each examination type are indicated by bars, and individual imaging instances are shown as triangles. The passing threshold was set at a value of two, which would be considered clinically adequate for diagnosis and measurement; only one imaging instance at two or higher is required to consider the overall examination a success for each operator.

Recommended order for scans was full bladder, empty bladder, right kidney, then left kidney. Due to the state of the subject’s bladder at the start of Demo 1, the crew performed ultrasound scans in the following order: right kidney, left kidney, full bladder, empty bladder, then returned to right kidney. Bilateral kidney imaging was executed in two segments totaling 73 min, with improved performance in the final 7-min repeated right kidney scan performed after bladder protocols. The bladder protocol was completed in 25 min, including ∼4 min break for voiding. The altered scan order and repeat of the right kidney examination provided a serendipitous internally controlled opportunity to observe substantial improvement over the course of this short demonstration activity (compare Right and Right Repeat scores in Fig. 3B), demonstrating the AMOS tool’s capability of supporting a very rapid learning curve.

HUMAN FACTORS OBSERVATIONS

ISS cabin video analysis identified human factors concerns during one of the demos, including suboptimal positioning and communication lapses. The ultrasound machine and laptop were located on the same side of the cabin, with the subject sometimes positioned between the operator and ultrasound controls. This positioning was cumbersome but effective for optimizing some imaging angles. The subject provided cooperative actions, including repositioning and managing ultrasound controls. The subject attempted self-scanning, but this proved awkward and uncomfortable. One of the demo teams had fewer human factors challenges with a more consistent “floating seated” side-by-side positioning.

During Demo 2, the operator “ended exam” after completing the prevoid bladder series, and the postvoid examination began with a default preset that was not optimized for a bladder scan. Since the operator did not reverify settings, the bladder scan was markedly prolonged with reduced image quality (Fig. 3A; Demo 2 Post-Void Horizontal). In another instance, the operator accidentally pressed the “invert” button, causing a right-left inversion of the image and increasing the difficulty of image capture. Although this highlights the utility of AMOS even during nonoptimal conditions, it also demonstrates a concern for unforeseen errors with the human–machine interface, particularly when interfaces are complex.

SOFTWARE USE PATTERN

For all demos, navigation within the AMOS software was primarily through laptop arrow key navigation rather than point-and-click actions. The software supports various learning and procedure execution styles: in one demo the operator watched only two video clips, in another demo only three, and in one demo, no clips were viewed, indicating operator media preference for text and still image material.

CREW IN-FLIGHT SURVEY AND POSTFLIGHT DEBRIEF

Operators rated the software high on usability, content organization, and overall satisfaction (Fig. 4). Free text responses highlighted the usefulness of the combination of text and videos and suggested additional instructions for adjusting color Doppler gain and acquiring Doppler images. The Demo 2 operator also noted the problem regarding ending the examination at the end of the bladder module, suggesting “if-then” wording for repeated scans in the same module. One user also noted that while AMOS software is very useful for nominal scans, more assistance would be needed if autonomous diagnosis was also a goal.

Fig. 4.

Software acceptability ratings for bladder (Panel A) and kidney (Panel B) AMOS modules; individual ratings are shown for operators of both demonstrations. AMOS, Autonomous Medical Officer Support.

In a post-flight debrief, both operator and subject reported no need for ground support during the study procedures and stated that the AMOS guide was clear and intuitive. Operator and subject affirmed that AMOS would be useful for medical procedures in spaceflight. The crew also provided positive feedback and constructive suggestions for future enhancements to support crew autonomy during medical procedures, including integration with portable or wearable screens and incorporation with artificial intelligence (AI) platforms for real-time guidance.

Discussion

Autonomous ultrasound image acquisition was successfully accomplished during the AMOS Tech Demos, and clinically adequate images were obtained for nearly all views, validating the concept of autonomous guidance in spaceflight. Especially remarkable is the learning curve observed in Demo 1 kidney imaging segments; the operator was able to recognize deficiencies, optimize probe manipulation technique during difficult imaging, and reacquire data with much higher quality. In this instance, the crew chose to perform kidney scans first, then return to bladder. Had the bladder protocol been performed first, time savings and better initial proficiency would likely have been demonstrated when scanning kidneys. Being more straightforward, the bladder scan is ideal for becoming familiar with basic ultrasound scanning concepts such as marker orientation, sweeping the probe through the organ volume, and saving cine-loops. Experience with basic ultrasound tasks on less technically challenging scans before attempting more complex examinations like kidney may be an important step in learning, building, and maintaining skills. This reversal of scan order did, however, provide a clear example of rapid skill acquisition that translated into higher image quality scores (Fig. 3B), demonstrating effectiveness of this JIT methodology.

Crew ergonomics and teamwork seemed to influence imaging success. For many spaceflight activities, ground teams suggest certain positioning, but ultimately, crew are best suited to discover arrangements that work best in microgravity. Subjects and operators demonstrated excellent teamwork during Demos, enhancing resultant images. Sometimes optimal positioning for imaging meant the operator leaned across the subject’s body to adjust ultrasound settings or advance through AMOS software; in many instances this was solved by the subject operating ultrasound controls.

Crew motivation to collect useable, diagnostic-quality images would be enhanced during an actual on-orbit medical event. Therefore, integrated subject–operator team training should be considered for medical tasks, particularly complex imaging. The operator could direct the subject to make adjustments on the ultrasound in this scenario, but even more preferred is a subject who is fully engaged and aware of procedures and task timelines; this preferred scenario was observed during AMOS Tech Demos. Two caveats to these points are (1) in these Demos, the subjects were physicians and particularly engaged in the activity, and (2) during a medical emergency, the subject may not be able to fully contribute to scanning procedures depending on general medical condition and degree of distress.

During Demo 1 the subject attempted self-scanning, which was initially considered as a nominal mode for kidney scanning procedures. Although the left kidney was clearly identified on scanhead video, arm positioning and grip on the probes were awkward. Degree of discomfort and lack of probe control will vary with individual anatomy and imaging window for each subject. For most, however, this positioning is not feasible for a full session of scanning and could only be used for some views as a secondary option.

Operators primarily navigated AMOS software through laptop arrow keys rather than direct-to-section content through cursor point-and-click action. This is an expected use mode, especially for first-time users who would likely page through all content linearly (similar to book style navigation). In addition, given that the operator’s hands are occupied with scanning tasks, simpler arrow key navigation is anticipated. Many navigation actions occurred within seconds while accessing desired material, indicating a rapid scan of most information. Once at the desired content page and examinations had begun, there were lengthy time gaps between software navigation actions. Crew feedback highlighted the usefulness of combined text and video instructions, but minimal reliance on embedded videos suggests flexibility for different learning styles.

The potential for the AMOS platform to enhance skill management during long-duration spaceflight was clearly recognized by the crew, and high usability ratings were coupled with suggestions for future enhancements. Eventual integration and deployment of the AMOS platform on a more portable and/or wearable screen would be preferred once these technologies are more readily available in flight. Simplified hardware interfaces and integration with machine learning or AI platforms for real-time guidance will need to be leveraged for accomplishing more crew autonomy in medical and other procedures. Guidance procedure platforms like AMOS will need to be more seamlessly integrated with hardware being used for each procedure. For example, newer handheld ultrasound devices have substantially simplified interfaces with AI software guidance; future products that merge in-platform display of live imagery with detailed procedure and reference material would be an ideal solution to fulfill recommendations made by crew. Handheld systems could also lighten system operation aspects of imaging procedures, increase flexibility of positioning, and facilitate self-scanning as an alternative to operator-subject scanning.

The bladder and kidney protocols within the AMOS platform are limited standardized protocols and not designed to fully satisfy formal clinical imaging criteria. While the principles and key solutions appear effective, additional trials and modifications should be conducted to best meet the clinical imaging needs of spaceflight. Clinically oriented modifications of autonomous protocols must rely on best terrestrial and spaceflight evidence and be fully coordinated with operational medical organizations to maximize benefits of risk perception as well as current and future clinical practice.

NASA has repeatedly demonstrated that the crew can reliably perform complex medical procedures remotely,^3,19–23 but enhanced contextual knowledge and AI-assisted interpretation of diagnostic data will be critical for self-medical care on exploration missions to Mars. Similarly, medical skill management platforms will need to be fully integrated with medical support architecture from the perspective of hardware, companion software, data analytics, and data transmission and storage systems. ²⁴ Finally, personalized and adaptive training should be considered for all skill management applications intended for exploration spaceflight.

Space-based autonomous medical systems such as AMOS and terrestrial telemedicine initiatives are increasingly interdependent, with each domain offering critical insights that inform and accelerate the other. For example, needs and advances are being made that are common to both spaceflight and terrestrial applications (e.g., renal disease management,^14,25 ophthalmology,^26,27 and thrombosis/cardiovascular conditions^23,28,29), which can contribute to resolution of significant health risks of spaceflight. ³⁰ Development and demonstration of AMOS in spaceflight not only advances frontiers of autonomous care delivery in extreme environments but also provides valuable frameworks for enhancing remote and resource-limited healthcare on Earth. ³¹

Conclusion

Based on demo data and crew participants’ feedback, the AMOS platform’s potential to enhance skill proficiency during exploration spaceflight was established, with high crew satisfaction. As NASA continues to develop progressively Earth-independent medical operations, platforms like AMOS will be required to support CMOs. This proof of concept provides a valuable initiative for future applications of the AMOS platform and the next generation of JIT guidance for exploration-class spaceflight.

Authors’ Contributions

D.E.: Conceptualization, writing—original draft, and analysis supervision; M.W.: Conceptualization, analysis, and writing—review and edit; V.E.B.: Software, resources, and data curation; A.M.N.: Writing—review and editing and visualization; A.S.: Data curation, analysis, and writing—review and editing.