Anatomically Integrated In-Place Visualization of Patient Data for Cooperative Tasks with a Case Study on a Neurosurgical Ward

Abstract

The workflow in modern hospitals entails that the medical treatment of a patient is distributed between several physicians and nurses. This leads to intensive cooperation, which takes place under particular time pressure and requires efficient conveyance of relevant patient-related medical data to colleagues. This requirement is difficult to achieve with traditional data representation approaches. In this paper, we introduce a novel concept of anatomically integrated in-place visualization designed to engage with cooperative tasks on a neurosurgical ward by using a virtual patient’s body as spatial representation of visually encoded abstract medical data. Based on the findings of our field studies, we provide a set of formal requirements and procedures for this kind of visual encoding. Moreover, we implemented a prototype on a mobile device that supports the diagnosis of spinal disc herniation and has been evaluated by 10 neurosurgeons. The physicians have assessed the proposed concept as beneficial, especially emphasizing the advantages of the anatomical integration such as intuitiveness and a better data availability due to providing all information at a glance. Particularly, four of nine respondents have stressed solely benefits of the concept, other four have mentioned benefits with some limitations and only one person has seen no benefits.

Introduction and prior work

Patient-related medical data are stored and maintained in order to support different diagnostic and treatment processes in a hospital and are shared within and between professional groups, such as physicians, nurses and sometimes technicians. Here, the increasing complexity of these processes due to, for instance, new diagnostic methods and the expanding interdisciplinary as well as inter-hospital collaboration, is amplified by the challenges in organizing clinical work due to shift rotations and understaffing. Therefore, the timely access of the medical personnel to the currently relevant patient data is crucial for treatment quality. In this paper, we present a visualization approach to support clinical cooperation by building on the visual tradition in medicine. Compared to prior work, we focus on the specific requirements of such cooperative tasks, that is the need of a synoptic-like representation of the current patient’s status for a fast information transfer. In particular, our approach allows to visualize multiple clinical parameters in an intuitive, anatomically integrated fashion. In the following, we describe our main contributions and the delimitation to other methods more in detail.

Various visualization approaches have been proposed for intuitively accessing patient’s medical data in order to improve clinical support and quality assurance. ¹ In general, we can distinguish two main categories, depending on the aim of such visualizations. The first category is characterized by the exploratory nature of the approaches and commonly refer to electronic health records (EHRs). That is, the visualization provides an overview of a patient’s complete set of medical and potentially administrative data and offers functionality for data selection including refinement, reconfiguration of layout and visual data encoding, and detection of data correlations or outliers. ² The second category comprises task-oriented visualization approaches that address medical tasks with specific requirements and with a focus on efficient information access and transfer. Examples are, for instance, communication to patient of the prostate cancer health risk ³ or reading of imaging diagnostic records. ⁴ For the purpose of such visualization systems a relatively small subset of patient data, in some cases not documented in the EHR, ⁵ is relevant.

Visualization tools for patient data predominantly utilize concepts from information visualization, for example, encoding time-related medical data as colorized geometric primitives that are placed at a timeline.^6–8 Some newer works intend to enhance the comprehensibility and navigability of the timeline-based EHR visualization, using it in combination with a data abstraction layer ⁹ or a generic data model such as the Observational Medical Outcomes Partnership Common Data Model. ¹⁰ Still, the physicians are in general not very familiar with abstract visual representations, which can impair their understanding. ¹¹ Therefore, some visualization approaches additionally utilize a virtual human body in order to provide a more intuitive data access. For instance, Kirby and Rector ¹² presented one of the first systems that uses a visualization of the human body as the central entry point for an efficient access of medical data items that are presented as text. Sundvall et al. ¹³ presented a prototype of a 2D visualization framework for patient’s medical data based on Google Earth. It supports so-called placemarks, the standard Google Earth approach to position body-referenced information as icons and text. An et al. ¹⁴ developed a 3D navigation and visualization method for medical data that uses different levels of detail (LOD) in combination with different visualization strategies applied to data at different levels. Particularly, on the two topmost levels a virtual human body is used to refer to affected organ systems or organs, which serve as access points to respective disease descriptions. Shi et al. ⁴ proposed a visualization tool that uses a 3D anatomical human model in combination with a three state color encoding of pathological severity in order to facilitate for radiologists the access to the current and historical medical status of a patient. The user can access the original data, such as DICOM images or imaging diagnostic reports, by selecting the highlighted organs. Jaatun et al. ⁵ developed a tablet-based digital pain body map that allowed patients with cancer to visually express their pain by drawing its location on a 2D virtual body and applying a color encoded scale for its intensity. The tool is aimed at an improvement of the communication between patients and medical personnel and supports the information exchange between different physicians and nurses by a patient handover. The existing anatomy-based approaches have in common the focus on the on-body visualization of few, mainly a single parameter, while further data are presented using abstract visualization methods or text.

The overall goal of our interdisciplinary research project is the analysis of cooperative clinical work practices on a neurosurgical ward and the development, implementation and evaluation of appropriate visualization concepts to improve these practices by providing efficient and task-oriented access to the health status, i.e., the relevant medical data, of a patient. The focus is on pre- and post-operative medical tasks that are distributed among several doctors and nurses due to shift rotations and occasional understaffing and therefore rely on their cooperation in the form of circulation of patient’s status. As this cooperation takes place in the strict time slots of the main medical tasks, e.g. ward rounds, it is characterized by the particular time pressure under which the patient’s status must be conveyed from one physician or nurse to the next.

In the initial analysis, our field studies revealed major findings that are relevant for our visualization concept (further details are given in Sec. Field Study). (1) Only a specific and relatively small subset of data in a concise and objectified form has to be conveyed, depending on the usage context. (2) A significant part of such subsets are diagnostic data related to the anatomy, e.g., clinical symptoms. (3) Neurosurgical personnel strongly rely on their visual tradition and can therefore easily get used to anatomical visualization metaphors.

In this paper, we present a new visualization concept based on our findings regarding the cooperative clinical work practices on a neurosurgical ward. Our approach focuses on transferring diagnostic medical data with anatomical reference. Inspired by the prior work that partially utilizes a virtual human body for visualizing medical data^4,5,12–14 and encouraged by the positive feedback from the neurosurgical personnel regarding anatomy related visualization techniques, we exploit the visual tradition in medicine as much as possible in order to achieve an anatomically integrated in-place visualization. This means that several, task specific data are encoded with appropriate visual attributes, which then are applied by displaying the respective anatomical structures of a 3D avatar.

Following the basic visualization principle of the previous body-based approaches, we especially focus on providing a visual synopsis of patient’s task-specific status, i.e. a consistent, simultaneous representation of multiple key data. Here we take advantage of the fact that, as observed during our field studies and stressed in the interviews by our medical partners, in the considered cooperative constellations only relatively small subsets of patient diagnostic data are relevant for the physicians (see the major finding one above), which allows us to minimize the necessity of manual online data selection and filtering. Still, we have to face the challenging task of mapping multiple quantified medical data to visual attributes – except spatial location, which is pre-defined by the anatomy – in a way that they can be effectively perceived in a simultaneous visualization.

The main contributions of this paper are as follows:

1. The concept of an anatomically integrated in-place visualization of medical data with anatomical reference including the definition of requirements and visual encoding procedures for this kind of visualization.

2. The design goals and data categories for clinical neurological symptoms related to the differential diagnosis for patients with spinal disc herniation. This serves as case study for applying our visualization concept and the formal requirements.

3. A prototypical implementation of the visualization concept and its evaluation by 10 neurosurgeons.

Our evaluation shows a strong consensus among the assessing physicians about the usefulness of applying our visualization concept to the diagnostic data of patients with spinal disc herniation.

It should be emphasized that our concept is designed for visualization of a specific kind of medical data that refer to anatomical structures. Therefore, for a more advanced use it may need to be combined with information visualization techniques to incorporate medical data without anatomical reference.

Design method and goals

In the context of our interdisciplinary long-term project on a neurosurgical ward, we investigate fundamental questions of analyzing and in part reconfiguring cooperative medical work practices. From a sociological perspective, articulation work, ¹⁵ i.e. the ongoing work of integrating distributed tasks and maintaining a coherent treatment trajectory by collection, processing and organizing patient data, is at the core of clinical cooperation. However, this information flow is hampered by gaps and distortions due to the limited availability or time-consuming access to patient-related medical data, which in turn is at least partly related to their inadequate visual presentation.

In our project, we aim at the design and development of an integrated in-place visualization of patient-related medical data in order to support and potentially even modify specific cooperative workflows. We address the fundamental problem of the acceptance, effectiveness and efficiency of new technologies in working practices using a participatory design approach, similar to Belden et al. ¹⁶ and Sedlmair et al. ¹⁷ That is, we set up a design and implementation process that involves visualization researchers, sociologists and neurosurgeons in order to analyze specific cooperative real-world situations on the neurosurgical ward. Our design and implementation process (see Figure 1) comprises high-frequency interaction between these groups of experts, in particular during the initial definition of the overall goals of the intended visualization tool and the field studies that comprise observations of cooperative workflow situations and interviews. This interaction is based on the mutual engagement in the design process following a symmetrical approach (Schubert and Kolb). ¹⁸ The insights of the field study are depicted in Sec. Field Study, whereas the deduced design goals are presented in Sec. Design Goals.

Figure 1.

The design and development process for the visualization prototype. The rectangles represent the three main stages. The last stage is an iterative prototype development, which leads to a gradual concept refinement on the basis of experts’ feedback.

Moreover, we utilize participatory refinement of design and implementation similar to the principle of agile software development (see, e.g., Martin), ¹⁹ in which visualization researchers, sociologists and neurosurgeons jointly advance and refine the visualization design on the basis of the formulated design goals (see Sec. Design Goals) by utilizing prototype implementations of a visualization tool. For reasons of efficiency and in order to not bias the evaluation, three physicians (expert group) are involved in this stage, whereas the evaluation involves a distinct set of 10 physicians (test group; see Sec. Prototype Evaluation).

Field study

In this first phase of data collection and processing, field observations and interviews were conducted on the neurosurgical ward of a local, 480 bed hospital. Here, the sociologists, and partially visualization researchers, observed a total of 14 neurosurgeons for 22 days during their routine ward work to become acquainted with the workflows and relevance structures of their daily work. An important finding was the fact that the physicians rely on the patient’s anatomy as spatial reference of the diagnostic results not only using medical imaging for instance, but also by handwriting information, e.g., on anatomical sketches in examination forms. Furthermore, during these observations, specific cooperative work constellations have been identified. Due to the distributed character of the clinical work, the patient-related medical data collected by one physician, has to be conveyed to the colleague who continues with the patient’s treatment. In this context, the task-related subset of medical data that describes the patient’s current status and its change over time is of particular importance. A significant part of this subset constitute diagnostic data related to the anatomy, e.g., clinical symptoms. Moreover, the timely efficient assessment of the relevant medical data and its temporal change plays a crucial role for decision making and treatment quality. An effective conveyance of the patient’s status under these particular conditions requires a data representation in an objectified and concise form.

The cooperative work constellations can occur in synchronous or asynchronous mode. An example of synchronous cooperation is the shift handover or, more precisely, the presentation of patient during the morning ward meetings. In these cooperative constellations the information is passed on personally, mostly in verbal, i.e., volatile, form. However, a personal shift handover is not always possible due to the inherent irregularity of the clinical workflow that results from possible emergencies. In these cases, the required information has to be passed on to colleagues asynchronously. This kind of asynchronous cooperation frequently occurs when dealing with unfamiliar patients, e.g., during ward rounds or when writing of discharge reports. Currently, the physicians have to recover the patient’s status from different data sources. Furthermore, even when treating a patient they are familiar with, the physicians often need to recollect the patient-related medical data they acquired themselves.

Design goals

Following the project motivation and the findings of the field studies, the overarching goal of the intended visualization system is to facilitate efficient transfer of the patient’s status in the context of distributed medical tasks. The resulting design goals listed below have been jointly deduced by the visualization researchers, sociologists and neurosurgeons from findings of the field studies, using a symmetrical approach (see above Design Method and Goals):

DG1 familiarity in novelty

The goal is to exploit the existing visual tradition, i.e., the usage of the anatomy as spatial reference, as far as possible and to prevent abstract visualizations in order to achieve a high degree of intuitiveness.

DG2 visual discriminability

A data mapping that allows the user to clearly discriminate between corresponding visual attributes in a given clinical usage context is crucial for the correct understanding of a visualization, and thus for an effective information transfer. The preference for anatomical visualization concepts defines the spatial location as a visual attribute a-priori, which makes such a mapping even more challenging to achieve.

DG3 context-related synopsis

In cooperative settings, it is important to provide a simultaneous visual access to all the data that are relevant in the given clinical usage context for conveying the current patient status at a glance.

DG4 intuitive comprehensibility

The visualization design has to focus on an intuitive and comprehensive interpretability in order to address the overall goal of efficient and effective transfer of the patient’s status.

DG5 concise visualization

In order to avoid clutter and distraction by secondary details, we aim to provide only the data that are relevant in the current clinical usage context and to focus on medically relevant, i.e., abnormal, value constellations.

DG6 visualization of chronological changes

The visualization should provide access to the evolution of the patient’s data.

DG7 mobility

The goal is to make the visualization available in all places using a mobile device.

The visualization concept

General considerations

This section describes our visualization concept, which aims at the achievement of the design goals (see Sec. Design Goals) in consideration of practical insights into the hospital workflow gathered in our field studies (see Sec. Field Study).

At the core of the concept is an anatomically integrated in-place visualization of medical data relevant to specific cooperative tasks. According to this visualization principle, an anatomical model serves as spatial representation of medical data that inherently refer to its structures, e.g., the clinical symptom paresis referring the affected muscle. Particularly, the visual attributes that appropriately encode the medical data to be visualized are applied by rendering of the corresponding anatomical structures, changing their default or “natural” appearance.

Using the human body as spatial reference, we directly address the design goal DG1 of exploiting the existing visual tradition, which is tightly bound to the human anatomy. On the one hand, this approach reduces the freedom in assigning medical data to visual attributes compared to Information Visualization (InfoVis) (see, e.g., Bertin, ²⁰ Carpendale), ²¹ since the attributes ‘spatial position’ and ‘form’ are defined by the model and are no longer free parameters. On the other hand, the focusing on a specific cooperative context and on data relevant therein (cf. DG5), significantly reduces the amount of data to be visualized and, consequently, the number of required visual attributes. This data preselection in combination with in-place visualization provides a context-related synopsis at a glance (cf. DG3). Moreover, we expect that the spatial embedding of medical data in the anatomical context makes its relation to the underlying real phenomena more evident and facilitates its interpretability (cf. DG4).

However, the anatomically integrated in-place visualization poses a very fundamental challenge: How to design the mapping from abstract medical data to visual attributes under the restriction of a preset location/geometry, such that visual discriminability (DG2), context-related synopsis (DG3) and intuitive comprehensibility (DG4) can be achieved in practice? We cope with this challenge by providing the formal requirements that any specific realization of the anatomically integrated in-place visualization has to fulfill and the corresponding procedure of visual encoding (Sec. Formal Definition and Requirements) as well as by discussing further visualization concepts (Sec. Further Visualization Concepts).

Formal definition and requirements

Medical data

In order to develop an anatomically integrated in-place approach to visualize a synopsis of task-specific patient data, we need a formal definition of the medical data and visual encoding attributes. Our data organization links the formalism of the Unified Medical Language System (UMLS) ¹ to the data required in the visualization process. As UMLS itself is organized as ontology, it covers all notions and terms, as well as spatial and diagnostic relations. During the implementation process, UMLS serves as knowledge background in the discussion with the physicians to extract task-specific medical data (see Sec. Mapping of Spinal Disc Herniation Data). Additionally, we require data types and domains, as they play a central role in the visual encoding of the patient data. The medical data to be visualized is organized in categories, each of them can have one or several properties. The categories are grouped into usage-dependent views. Below we give the precise definition of these concepts and their relations.

• Property. A property p is defined as triple

$p=\left(\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{T}\mathrm{y}\mathrm{p}\mathrm{e},\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}\mathrm{T}\mathrm{y}\mathrm{p}\mathrm{e},\mathrm{d}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{i}\mathrm{n}\right),$

where p. propType refers to the underlying medical concept, p. dataType is a nominal, ordinal or numerical data type and p. domain is the domain set of elements of the corresponding p. dataType.

• • Category. A category c is a pair

$c=\left(\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l}\mathrm{R}\mathrm{e}\mathrm{f},\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{s}\right),$

where c. spatialRef is the reference to the corresponding anatomical structure and c. props = {x|x is a Property} are all properties of c. We denote the set of all categories as C.

• • View. A view V ⊆ C is defined as the categories relevant to the given usage context. $\mathcal{V}$ is the set of all views.

Note: Views automatically reduce the amount of information to be visualized, and thus the necessity of user’s navigation, by preselecting the context relevant data.

The following examples illustrate the structural concept introduced above.

The category ‘muscle strength’ refers to the anatomical structure ‘muscle’, comprising a single property with the type ‘intensity’ that documents the strength a patient can create in a specific muscle, quantified in six numerical values.

$\begin{array}{l}\mathrm{m}\mathrm{u}\mathrm{s}\mathrm{c}\mathrm{l}\mathrm{e}\mathrm{S}\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{g}\mathrm{t}\mathrm{h}=(\mathrm{m}\mathrm{u}\mathrm{s}\mathrm{c}\mathrm{l}\mathrm{e},\\ \left\{\left(\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y},\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l},\left[0,\dots ,5\right]\right)\right\},\end{array}$

The second example describes the category ‘radicular pain’, i.e., pain caused by irritation of a nerve root and related to the skin region that is associated with the latter, i.e., dermatome. This category comprises one numerical and one nominal property.

$\begin{array}{l}\mathrm{r}\mathrm{a}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{P}\mathrm{a}\mathrm{i}\mathrm{n}=(\mathrm{d}\mathrm{e}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{o}\mathrm{m}\mathrm{e},\\ \{\left(\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y},\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l},\left[0,\dots ,10\right]\right),\\ (\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{g}\mathrm{g}\mathrm{e}\mathrm{r},\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l},\{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t},\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{s}\}).\end{array}$

There are several fundamental relationships between views, categories and properties:

• A property type is unique within a category. Property types obtain their medical meaning only in combination with the category they are used in.

• A property type can be shared between properties of several categories. This expresses similarity of medical concepts, such as ‘intensity’ of clinical symptoms in the prior examples, even if the respective property domains can be distinct.

• A category is unique within a view.

• A category can be shared between several views, as it can be relevant in different usage contexts.

In oder to be able to properly specify the mapping of medical data to visual attributes, we define the set T of all property types as

$T=\left\{p.\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{T}\mathrm{y}\mathrm{p}\mathrm{e}|p\in c.\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{s},c\in C\right\}.$

Visual attributes

Shape and position are mainly predefined as the anatomical structures that are used for visualization in our concept, the geometric visual attributes cannot be used for data encoding. ² The remaining visual attributes are color components, namely hue, brightness and saturation, textures and transparency, as well as time using, e.g., animations.

Let a denote a visual attribute, A the set of all visual attributes, and a. range the discrete and finite set of distinctively perceivable values of a ∈ A. Note, that $\left|a.\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\right|$ is commonly smaller than the number of displayable visual attributes. For example, the visual attribute ‘hue’ is a floating point value, however, the human vision is able to distinguish only up to eight hue values w/o external reference (see, e.g., Kuehni and Schwarz ²² ).

Visual encoding/mapping

The mapping of medical data to visual attributes is represented in Figure 2 and comprises three levels, i.e.(a) encoding of categories (step 1–2), (b) encoding of property types (step 3), (c) encoding of property values for all properties of each category (step 4).

Figure 2.

The three level visual encoding of medical data.

Injectivity requirement

In general, injectivity is a pre-requisite for any visual encoding/mapping in order to lead to an unambiguously comprehensible representation (see, e.g., Ziemkiewicz and Kosara). ²³ We distinguish the following situations where either the mapping injectivity is strongly required or its violation has to be recognized and appropriately tackled.

Local Injectivity

In the case the mapping M ^c : C → a ^C .range or M ^t : T → A \{a ^C } (see Figure 2) is not injective within a given view V or category c, respectively, it is impossible to trace back the categories or property types from their visual representation. Local injectivity can be guaranteed if (1) all categories of a view are mapped to distinct values of a ^C , and (2) all property types of a category are mapped to distinct visual attributes.

Global Injectivity

While local injectivity guarantees the visual distinctiveness of categories and properties inside each single view and category, respectively, the global injectivity ensures the uniqueness of visual encoding across respective contexts, i.e. categories and views for property types and views for categories. Due to the restricted distinctiveness of visual attributes, global injectivity is hardly achievable, still it should be pursued as far as possible.

Spatial Injectivity

Obviously, several categories of a view can refer to the same anatomical structure. The violation of spatial injectivity rules out simultaneous visualization of the respective medical data and needs to be handled explicitly (see Sec. Further Visualization Concepts).

Property Domain Injectivity

A non-injective mapping of a property domain to the respective visual attribute range causes quantization and, consequently, leads to a loss of information. In some cases a quantized visualization can be acceptable, in particular in the context related synopsis (cf. DG3), in which a qualitative overview is sufficient. In any case, quantization needs to be detected and reported. In general, quantization cannot be prevented and needs to be handled explicitly (see Sec. Further Visualization Concepts).

Visibility restrictions

A 3D visualization with free camera motion may intrinsically affect the visibility of geometric objects representing anatomical structures. In our case, we have two causes for restricted visibility. 1. An anatomical structure that relates to relevant medical data (target) may be occluded by another anatomical structure. For example, a muscle affected by paresis is hidden under the skin. 2. The spatial extension of an anatomical structure is too small in relation to the entire human body, so that the visualization cannot be clearly recognized. An example would be a tendon.

Both aspects are handled in our visualization concept as described in Section Further Visualization Concepts.

Further visualization concepts

Independent from the specific mapping that we introduce in Sec. Prototype Implementation, there are further aspects that we added to our anatomically integrated in-place visualization concept in order to achieve the design goals postulated in Sec. Design Goals.

Data selection

Besides two already described data selection mechanisms, which apply automatically, i.e., focusing on the abnormal and usage-dependent views, we provide the user with the possibility to additionally filter the data by their categories. Note that the usage-dependent preselection reduces the available categories to a manageable amount. This filter allows to tackle, inter alia, the visualization of multiple medical data for same anatomical structure, i.e., spatial non-injectivity (see Sec. Injectivity Requirement).

Alternating visualization

In the constellations where a simultaneous visualization of multiple medical data on the same anatomical structure is not possible, i.e., the spatial injectivity is not fulfilled (see Sec. Injectivity Requirement), an alternating visualization with additional user control to select one of the alternatives, e.g., through the data category filter, can be applied (see Figure 3(a), (b)).

Figure 3.

Visualization of temporal changes. Figures 3(a) to 3(b) show the visualization of the preoperative neurological status of a patient with an asymmetric patellar reflex, a moderate paresis in the right extensor hallucis longus muscle, a mild paresis in the tibialis anterior muscle, a severe pain and a hypoesthesia in the left L5 dermatome, whereby the pain (see Figure 3a) and the hypoesthesia (see Figure 3b) are visualized alternately; this visualization is used in Task 2. Figure 3c shows the postoperative status of the same patient with symmetry in patellar reflex, a decreased hypoesthesia in combination with paresthesia and a remaining mild paresis in the right extensor hallucis longus muscle (see Task 3).

Textual overlays

In some situations, the physician may want to access the underlying information explicitly, i.e., in textual form. We enable this by textual overlays on top of the corresponding anatomical structure in order to, for example, resolve the quantization problem (see Sec. Injectivity Requirement) that results in a simplified visual representation or to support the physicians in the getting acquainted with our visualization tool.

Proxies

The small object extension problem (see Sec. Visibility Restrictions) can be tackled by means of an appropriately scaled proxy that is displayed on the body surface over the location of the target anatomical structure and rendered with the corresponding visual attributes.

Transparencies

In order to handle depth occlusions of anatomical structures carrying relevant information by other anatomical structures (see Sec. Visibility Restrictions), we use view dependent (semi-)transparency for the occluder, in case the occluder itself is not carrying relevant information, while preserving the surrounding anatomical context.

The specific technical approaches taken to implement textual overlays, proxies and transparencies are described in Sec. Rendering Implementation Details.

Prototype implementation

Based on the design goals stated in Sec. Design Goals and the visualization concept introduced in Sec. The Visualization Concept, we present a prototype implementation on a mobile device. As a proof of concept, we focus on patients with spinal disc herniation and the relevant medical data for assessing a patient’s neurological status.

Development environment

As geometric model for the anatomically integrated visualization we use plasticboy’s 3D human avatar ³ , which fulfils the substantial requirement of an exact and faithful representation of the human anatomy. Even though the model already includes the main organ systems subdivided into the corresponding anatomical structures, this subdivision is not sufficiently detailed for the use with our visualization approach. Thus, we map the initial plasticboy anatomical human model to the hierarchy of the Foundational Model of Anatomy Ontology (FMA) ⁴ . Furthermore, dermatomes, which constitute very important anatomical structures in the context of neurosurgery, are not reflected in the model. Hence, we set up an anatomy refinement procedure based on indexed texture maps that allowed us to define substructures on the existing geometries.

Our prototype was developed in C++, using Qt 5.12 for Android for the platform integration and our own rendering framework, which is based on the Vulkan 1.1 API. The latter allows an efficient resource management, which is important especially on mobile devices.

We developed the prototype in Qt Creator natively, using the Qt Android integration and our own Vulkan Framework.

Prototype features

Mapping of spinal disc herniation data

Raw Data Categories

In collaboration with the expert group of neurosurgeons (see Sec. Design Method and Goals), we determined seven data categories and their properties (see Table 1) that are of high relevance for cooperative tasks with respect to the representation of neurological status of a patient with spinal disc herniation, i.e., in the respective view (cf. Sec. Medical Data). The type of the main property to be visualized, common for all these categories, is the intensity with which the respective symptom manifests, whereas its domain is individual for each symptom, i.e., category.

Table 1.

Data Categories: The raw data categories (left block), the final categories after discussion with the physicians (center block), and the visual attributes incl. Anatomical reference (right block). Only the abnormal states are listed. Specific aspects are indicated as ¹ : quantization, ² : usage of proxy geometry.

Raw data categories			Final data categories			Visual attribute	Anatom. Reference
Category	Prop. Type	Domain	Category	Prop. Type	Domain	(| range|)
Radicular pain			Radicular pain			Red
	Intensity	{1, …, 10}		Intensity	{1, …, 10}	Saturat.-brightn. (3) 1	Dermatome
	Trigger	Binary		Trigger	Binary	Texture Normal Pert. (1)
Muscle strength			Paresis			Purple
	Intensity	{1, …, 5}		Intensity	{Mild, moderate, severe}	Saturat.-brightn. (3)	Muscle
T-reflex			T-reflex			Green
	Intensity	{1, …, 5}		Intensity	{1, …, 5}	Saturat.-brightn. (5)	Tendon 2
Excretion disorder			Excretion disorder			Orange
	Intensity	Binary		Intensity	Binary	Saturat.-brightn. (3)	Urethra or anus 2
Paresthesia			Sensory disorder			Cyan
	Intensity	{1, …, 3}		Intensity	{1, …, 4}	Saturat.-brightn. (4)	Dermatome
Hypoesthesia				Paresthesia	{1, …, 3}	Texture Noise (3)
	Intensity	{1, …, 3}
Anaesthesia
	Intensity	Binary

The category radicular pain has a further property with the type trigger, which states if the pain is constant or only occurs under stress, e.g. during movements, whereby the former is assumed to be the normal, i.e., default situation for pain that does not need any visual indication.

Data Category Refinement

The above raw categories mainly reflect concepts from the Unified Medical Language System (UMLS) vocabularies, related to neurological findings, which then have been partially refined with the participation of domain experts, in order to better map the application logic to the given context of cooperative tasks.

During the first trials with the prototype and discussions with neurosurgeons, two changes to the initial raw data categories have been applied.

First, the three categories related to sensory disorder, i.e., paresthesia, hypoesthesia and anaesthesia have rather complex interrelations. For example, hypoesthesia and anaesthesia can be considered as different stages of sense decrease, whereas paresthesia is in a certain sense orthogonal because it does not describe a decrease of sensation but rather its abnormality, e.g., tingling, and, thus, it can occur in combination with hypoesthesia. Therefore, the new category sensory disorder represents anaesthesia and hypoesthesia as a joint property with the type ‘intensity’ and has an additional property with the type ‘paresthesia’.

Second, insufficient muscle strength, measured in the Medical Research Council scale 0, …, 5, is used in daily clinical practice as indication for paresis that trigger for potential urgent actions such as emergency surgery. Therefore, we adopted this practice by using the category paresis with the ordinal data type comprising the values ‘mild’, ‘moderate’, ‘severe’.

Mapping of Data to Visual Attributes

The final medical data categories in Table 1 are visually encoded in consideration of the rules deduced in Secs. Visual Encoding/Mapping and Injectivity Requirement, that is, the mapping functions M ^c and M ^t are at least locally injective. We selected hue as the visual attribute a ^C to encode category (cf. Step 1 in Figure 2). The category mapping M ^c (cf. Step 2 in Figure 2) takes into account the distinctiveness of the resulting five hues with regard to each other as well as to their context in the anatomical model, e.g., a red color is not suitable for visualization on muscles because it highly coincides with their natural, i.e., healthy, appearance. The shared property type intensity is mapped in all categories to a composite visual attribute saturation-brightness in the HSV color space, i.e., M ^t (cf. Step 3 in Figure 2) is also globally injective. We decided to combine two respective visual attributes in a single one to increase the visual discriminability (DG2). Additional properties are encoded by means of textures. We select texture such that they are visually as complementary to the visual attribute color as possible in order to allow for a simultaneous visualization. In case the additional property is non-binary, e.g., the property with the paresthesia type in sensory disorder (cf. Table 1), the required range of visual values corresponds to the texture’s frequency and amplitude (cf. Step 4 in Figure 2); see also Figure 4.

Figure 4.

The legend of the prototype visual encoding. Figure 4a depicts the active tab of the sensory disorder category: the right L2-L5 dermatomes visualize four intensity levels, the left L3-L5 dermatomes show three levels of the paresthesia, i.e. Noise texture; Figure 4b shows the active radicular pain tab: the right L3-L5 dermatomes visualize three intensity levels, the left L3-L5 dermatomes show same levels in combination with the stress trigger, i.e., Normal Pert. Texture (cf. Table 1); the exact property values can be looked up by means of overlays similarly to Figure 6c.

There are two additional features of the mapping to visual attributes that need to be mentioned (cf. Table 1):

(1) The 10 intensities of the pain category result in a visual quantization, i.e., the corresponding property domain mapping is not injective (see Sec. Injectivity Requirement). In this case, the technical reason for the quantization coincides with the physician’s suggestion for a reduction of the number of intensities for this symptom to account for the subjectivity of the patient’s sensation (cf. Jaatun et al., ⁵ who encoded pain intensity using three colors instead of the 1-to-10 scale, which was evaluated as suitable by most users; cf. also the 0-to-3 Digital Palpation pain scale in Masulo et al. ²⁴ ). (2) the anatomical reference for the categories T-reflex and excretion disorder are too small and require a proxy geometry (see also Sec. Rendering Implementation Details and, e.g., Figure 3).

Rendering implementation details

In this section, we briefly describe some implementation details related to the realization of specific visualization features, partially mentioned in Sec. Further Visualization Concepts.

Anatomical Proxies

The main idea in generating anatomical proxies is to utilize projective textures, ²⁵ e.g., appropriately scaled circles, on the skin surface above the anatomical structure that is too small for a direct visualization (cf. Sec Visibility Restrictions and the patellar reflex, e.g., in Figure 3).

Occlusion Handling

In order to visualize hidden anatomical structures in an integrated overview of the most relevant information without requiring specific selection, navigation or zooming efforts, we dynamically decrease the opacity of areas above the occluded target structure, depending on the current camera transformation (cf. Sec. Visibility Restrictions). Similar to Viola et al. ²⁶ and Burns and Finkelstein, ²⁷ we use an image space approach that allows an efficient detection of occluding fragments in real time on mobile hardware (see Figure 5).

Figure 5.

Dynamic transparency area with hierarchical information: the right gastrocnemius muscle with paresis data (purple) is visible through the skin of the same body region and partially occluded by the left leg.

Context Menu

The prototype includes a context menu for accessing the advanced features such as the data category filter (see Figure 6a) and the overlays with textual data (see Figure 6c). The filter provides for the user the possibility to hide/show data visualizations by their category as described in Sec. Further Visualization Concepts, e.g., for handling of the spatially non-injective cases (see Sec. Injectivity Requirement). The overlays display textual data corresponding to the visualization on a anatomical structure selected by the user.

Figure 6.

Usage of further prototype features in Task 2. Figure 6a illustrates the use of the data category filter. Figure 6b shows a close-up of paresis after filtering, i.e., without the occluding dermatome. Figure 6c exemplifies the textual overlay for one of the affected muscles comprising its anatomical name and associated raw muscle strength data.

Visualization of Temporal Changes

For assessment of healing progress a slider with dates of available clinical examinations is integrated below the main 3D view (see Figure 3). Moving the slider, the user can navigate to the date of interest or scroll consecutive examination results. By selecting a date, the entries with the corresponding timestamp are retrieved from the database and the virtual body is rendered with updated visual attributes.

Prototype evaluation

Objectives

With the goal to evaluate the potential of our visualization concept and its limitations, we conducted a user study, addressing the following objectives.

EO1 Perceived usefulness of the anatomically integrated in-place visualization concept in comparison with the conventional data representation, i.e., hospital information system, examination forms etc. We separately evaluate the following core concepts

(a) Use of anatomy for spatial data representation (cf. DG2);

(b) Synopsis-like visualization of multiple data (cf. DG3);

(c) Narrowing the amount of data to be visualized according to the given usage-dependent view (cf. DG5);

(d) Use of a mobile platform (cf. DG7).

EO2 Effectiveness of the information transfer. In particular, we evaluate the visual encoding of the medical data, i.e., that the medical data can be read from the visualization unambiguously regarding the following aspects:

(a) Reading of a neurological patient status;

(b) Reading of the healing progress by comparison of different status snapshots (cf. DG6);

(c) Usage of interactive mechanisms, for instance data category filter or zooming (see Sec. Rendering Implementation Details), where the direct reading is impeded.

EO3 Perceived efficiency in comparison with the conventional data representation. Since currently the relevant data are scattered in different sources, a direct quantitative comparison is impossible. Instead, users should estimate the expected speed-up due to our visualization approach on the background of their long-term experience in using the traditional data sources.

EO4 Completeness/meaningfulness of the implemented data selection for the view of neurological status (see Table 1). In particular, the aim is to assess the convergence between the usage-dependent view defined in collaboration with the expert group (see Sec. Design Method and Goals) and the data considered relevant in the given context by the test group.

Setup

The evaluation comprises three practical tasks and a questionnaire. Figure 7 gives an overview of the evaluation process, including the evaluation objectives to achieve in each specific stage.

Figure 7.

The evaluation process of the visualization prototype. The left and middle column show the numbers of practical tasks or questionnaires and their short descriptions, respectively. The right column displays the evaluation objectives, as described in EO1-EO4, targeted by the corresponding task or questionnaire.

The practical tasks are designed as follows.

Task 1

In the first task, the physicians need to give their intuitive interpretation of two visualizations (see Figure 8) without previous knowledge about the meaning of the visual attributes. The assumption to be tested is that embedding in the anatomy (see EO1a) and usage-dependent preselection of medical data (see EO1c) increase the intuitiveness of the visualization.

Figure 8.

Two visualization samples used for spontaneous interpretation in Task 1. Figure 8a shows right C6 dermatome with radicular pain. Figuure 8b shows several symptoms visualized in parallel: radicular pain in the left L4 dermatome, paresis of the left quadriceps muscle, asymmetric patellar reflex by means of the proxies.

Task 2

The visualization of a neurological patient status (see Figures 3a and 3b) has to be assigned to the corresponding textual description of clinical symptoms. The participants need to choose the correct status out of seven real-life descriptions. The task can be seen as succession of two sub-tasks, in which different evaluation objectives are addressed. (a) The direct reading of visual attributes (see EO2a) narrows down the possible choices to the subset of two very similar descriptions. ⁵ (b) Separating the remaining two similar cases requires some user interaction (see EO2c), namely applying data category filter in order to uncover the muscles, and, potentially, zooming and textual overlays (see Figures 6a–c).

Task 3

The last task is to read the healing progress (see EO2b) from two visualizations (see Figures 3a–3c) that show successive patient states. The first one equals Task 2 and corresponds to the preoperative status of a patient. The second one represents the postoperative neurological status of the same patients. Switching between both visualizations by means of the timeline, the physicians have to recognize (cf. DG2) and to describe the respective changes. Analogously to Task 2, there is an additional level of difficulty due to the partially remaining paresis that is still occluded in the postoperative status (see Figure 3c), which again requires the use of the data category filter (see EO2c).

The questionnaire comprises three main sections:

QNR 1

The first section is dedicated to the evaluation of data in the usage-dependent view related to patients with spinal disc herniations (see EO4). Particularly, the physicians assess the relevance of the pre-selected data categories in different cooperative constellations as described in Sec. Field Study as well as their completeness (see Figure 9a). The considered cooperative constellations are 1) shift handover; 2) visiting a familiar patient during ward rounds, i.e., a patient who already has been examined by the same physician; 3) visiting an unfamiliar patient during ward rounds, i.e., a patient who has been admitted and examined by a colleague; d) writing a discharge report, especially for patients who were treated by another, currently absent colleague.

Figure 9.

Quantitative results of the questionnaire. 9a: the x-axis represents the symptom categories. 9b: the x-axis shows the prototype features to be assessed. 9c: the x-axis shows possible answers to the question: Would the proposed visualization concept yield the following benefits in your work routine?.

QNR 2

In the second section the physicians assess the usefulness of the anatomically integrated in-place visualization concept, including the simultaneous visualization of multiple data and the availability on a mobile device (see EO1).

QNR 3

The third section includes questions regarding the effectiveness (cf. EO2) and efficiency (cf. EO3) of the prototype that implements our anatomically integrated in-place visualization concept (see Figure 9c).

Procedure

The exact evaluation procedure is structured as follows.

(1) After a brief introduction to the overall context, the participants are directly confronted with Task 1.

(2) Afterwards the visual encoding legend (see Figure 4) is introduced and the participants are familiarized with the most important interaction features of the prototype.

(3) Next, the physicians have to perform Task 2 and Task 3, explaining their choices.

(4) Finally, the participants have to reflect on the practical experience answering the questionnaire, i.e., QNR 1, QNR 2 and QNR 3.

Ten neurosurgeons from the same department took part in the evaluation, none of which was involved in the participatory refinement process described in Sec. Design Method and Goals, i.e., this test group had only a very basic understanding of the overall aim of the visualization prototype. The group covers very different professional experiences and hierarchical positions, namely it consists of three assistant physicians, one specialist, five senior physicians and one chief physician (for more details see Table 2). The evaluation study was conducted in the hospital, in groups of two persons or, partially, individually, and has been filmed for documentation and analysis. The complete procedure for a single group took about an hour. A total of nine (out of 10) completed questionnaires was submitted, because one of the interviews was interrupted due to a medical emergency. The total number of answers varies, since a few participants skipped some questions.

Table 2.

Professional experiences as well as age and gender structure of the test group.

Prof. Experience		Age		Gender
Years	# Persons	years	# Persons		# Persons
<10	3	30–40	4	Female	4
11–20	3	41–50	2
21–30	2	51–60	2	Male	6
>30	2	$>$ 60	2

Quantitative results

In the following we present an overview of the quantitative evaluation results, structured according to the aforementioned tasks and questions. The qualitative results and lessons learned are discussed in Sec. Discussions.

In Task 1, reading the visualization without any specific pre-knowledge, most physicians (8/10) correctly interpreted the red dermatome (see Figures 8a and 8b) as pain and the rest of the participants (2/10) as sensory disorder. The purple color in the muscle (see Figure 8b) was consistently associated with paresis (10/10), which is the correct meaning of this visual attribute. The green proxy geometries at the knees (see Figure 8b) have often been misinterpreted (6/10), e.g., as “problems with knee joints” or “hematoma”. Nevertheless, four participants (4/10) recognized them as visualization of patellar reflex.

In Task 2, assigning the visualization to a textual description, all physicians (10/10) could easily solve step 1, i.e., they narrowed down the conceivable variants to two similar cases. By step 2, three of the participants (3/10) independently found the correct case and the other seven of them could solve the task after our suggestion to use data category filter and zooming function.

The healing progress in Task 3 was interpreted correctly by all participants (10/10). Several of them (6/10) also could independently find the occluded remaining paresis in the postoperative status (see Figure 3c), hiding the occluding dermatome by means of the category filter without external suggestions.

The quantitative results regarding the questionnaires QNR 1-QNR 3 are summarized in Figures 9a–9c, and are discussed in Sec. Discussions.

Discussions

Evaluation findings

In the following we discuss the evaluation results according to the objectives determined in Sec. Objectives

Usefulness

The results of QNR 2 (see Figure 9b) and the physicians’ comments during the evaluation study show that the proposed visualization concept was perceived as very beneficial. In particular, the use of the anatomy (cf. EO1a) was well received. Several physicians pointed out that, as a positive collateral effect, the 3D body model also serves as anatomical quick reference guide. The results of Task 1 underline the increased intuitivity of our visualization, due to the anatomically integrated visual attributes in combination with a usage-dependent view (cf. EO1c). Both allow for the comprehension of the visualized symptom even without legend (cf. Interpreting muscle color as paresis) or significantly reduce the possible interpretations (cf. Color in a dermatome as pain or sensory disorder). However, the interpretation of visual attributes corresponding to the patellar reflex in the same task was perceived as less intuitive. According to the above claim, this can be explained by a lower importance of the data category “T-reflex” (see Figure 9a and the discussion regarding EO4 below), and the use of a proxy geometry, i.e., a weaker anatomical integration.

The physicians appreciated the availability of “a lot of information at a glance” thanks to the synopsis-like visualization (cf. EO1b). At the same time, some of them expressed concern about a potentially too complex appearance due to such an approach and the required user interaction, e.g., to hide an occluding anatomical structure. Also Task 2 revealed difficulties of the users to utilize the prototype’s interaction functionalities. However, this can be at least partially explained by an extremely short learning phase of about 20 minutes and the lack of experience with the corresponding prototype features, e.g., the data category filter. This conjecture is confirmed by the observation that in the next task (Task 3) the physicians used the interaction mechanisms for reading (partially) occluded information much more reliably.

Most physicians see a clear benefit in the mobility of the prototype (cf. EO1d), since it allows to save time in making the information available immediately in any location. However, it was mentioned that the current device is still too large and heavy to be carried in a coat pocket, and one participant also commented that a medical doctor with a mobile may be perceived by patients as “less professional”.

Effectiveness

The evaluation tasks demonstrates that the proposed visualization allows to effectively convey the neurological status of the patient (cf. EO2a) and its development over time (cf. EO1b). The observed limitations refer to complex situation with impaired visibility, which require user interaction (cf. EO1c), as discussed above.

The prototype’s capability to reduce the loss of information in the hospital workflow (see Figure 9c) has been rated controversially. The participants who remain rather skeptical, explained their attitude by the fact that the information still can be lost before reaching the prototype, due to a deficient collection and input of raw data by the examining physician. Surprisingly, only few physicians assessed the prototype as advantageous for communication with patients (see Figure 9c). This, however, is due to the characteristics of the workflow itself, i.e., all physicians admitted that the proposed visualization would allow to explain diagnosis to the patient more easily, but this would stimulate further discussions with the patients, which is in conflict with the strongly limited time budgets.

Efficiency

Comparing with the currently used textual representation of the respective medical data, the physicians attest to the prototype’s potential for accelerating the transfer and evaluation of the patient’s status (see EO3), which is due to the visualization of “all symptoms at a glance”, i.e., synopsis, and to the fact that the information is “always available”, i.e., mobility (see also Figure 9c). Only one participant claimed that simple lists are as good as visualizations.

Data selection

As follows from the rating in Figure 9a, all defined data categories are of high importance for assessment of neurological status of a patient with spinal disc herniation (cf. EO4), i.e., there is a high coincidence in the selection of relevant data within the test group as well as between the test group and the expert one.

Considerable variations occur in the category T-reflex only. The latter is caused by differences in the individual diagnostic methods: several participants actively apply reflex test during a clinical examination, whereas other physicians consider this test less relevant (cf. also the above discussion about effectiveness). In relation to the completeness of the view, six (6/9) physicians proposed further data categories to be added. While some of these proposals reflect individual variations in the diagnostic practices, referring to, for instance, additional pathological reflexes or pain qualities, other suggestions address the extended use of the prototype by integrating, for example, prior deceases or instability signs, required for decision making or surgery planning.

Limitations

Primarily, the prototype supports the evaluation of our core concept, i.e., anatomically integrated in-place visualization, and, thus, focuses on the visualization of usage-dependent medical data with anatomical reference, more precisely, neurological symptoms of spinal disc herniation, in the form of a synopsis, i.e., a consistent, simultaneous representation of multiple key data. The preference for using anatomy as a spatial reference and for the synoptic form of data visualization are major findings from our field studies, and their effectiveness, efficiency and acceptance by the physicians have been verified in our evaluation. Nevertheless, these visualization principles are potentially less useful in other medical application scopes, such as trend analysis of extensive medical data records, visualization of a single data category with a high domain cardinality, or the handling of medical data without anatomical reference. There is, however, the option to integrate other visualization approaches such as abstract timeline-based approaches into our visualization concept.

The use of human anatomy as spatial representation of abstract medical data, facilitating a more intuitive visualization semantic, at the same time implies some additional challenges with regard to visualization design. The general problem of mapping of a large amount of data to the limited number of visual attributes becomes in this case even more significant, because the spatial position and the form of anatomical structures are fixed. As discussed in Sec. The Visualization Concept, we address this problem with such concepts as data selection at different levels and alternating visualization. Nevertheless, by further increasing the number of patient data associated with same anatomical structure, the current solutions are not sufficient. Therefore, we aim to approach such complex situations in future work, e.g., by developing quantifiable textures that allow for encoding of a larger amount of quantified data.

Moreover, the on-body visualization can lead to configurations that are challenging in terms of visibility (cf. Visual Attributes). The proposed solutions are described in Sec. Further Visualization Concepts and Rendering Implementation Details. However, they potentially require additional user interaction, which can impair the information reading, as demonstrated in the practical tasks during the evaluation. In this respect, we consider the evaluation results as intermediate. A more substantial evaluation of the efficiency of user interaction mechanisms requires a greater familiarity of physicians with the prototype.

Finally, the current prototype does not allow for a personal adaptation to individual variations in diagnostic practices (see Sec. Evaluation findings). This suggests to introduce some personalization options by the view definition, on top of the core data categories.

Conclusion and future work

In this paper we presented a novel concept for anatomically integrated in-place visualization of medical data. The concept is designed in accordance with the requirements arising from specific tasks in cooperative clinical workflow, namely conveying of cooperation relevant patient data between colleagues, and it aims at closing the existing gap in current visualization approaches. Our approach allows for a spatially integrated comprehensive visualization of medical data, such as clinical symptoms, on a 3D human avatar using their inherent references to affected anatomical structures and an appropriate visual encoding. Preselecting patient data as a function of their relevance in the given clinical usage context, i.e., view, provides an at-a-glance synopsis of relevant information to physicians.

The evaluation of the prototypical implementation of the visualization concept by a group of neurosurgeons revealed positive feedback, in particular concerning the use of anatomy as spatial representation of data and the potential speed-up of information assessment. The current prototype covers the application scenarios in the context of distributed clinical cooperation related to the diagnosis of spinal disc herniation. Still, having a sufficient level of abstraction, the visualization concept is also transferable to other cooperative tasks and diagnoses, where a) data with anatomical reference are used, and b) a quick patient status overview is beneficial.

For a better integration in the existing workflow, each specific implementation would require field studies and participation of medical experts in the design process, particularly in regard to selection of task-specific medical data and intuitive visual encoding. In the next project phase, we plan to extend the prototype, aiming at the cooperation among nursing personnel on a neurosurgical ward, which, even though working with the same patients as the physicians, use different patient data to accomplish their tasks. An example of relevant data with anatomical reference in this context is information about the healing state of surgical wounds.

At the same time, the evaluation revealed some limitations of the current solution in the situations where the target anatomical structures are not sufficiently distinguishable without user interaction.

Beyond the extension of our prototype to further clinical usage contexts, we intend to address several of the issues that have arisen in the evaluation, such as the flexibility to adapt to individual professional procedures and the problem in distinguishing adjacent anatomical structures.