Memory for health information: Influences of age,hearing aids,and multisensory presentation

Abstract

We investigated how presenting online health information in different modalities can influence memory, as this may be particularly important for older adults who may need to make regular decisions about health and could also face additional challenges such as memory deficits and sensory impairment (hearing loss). We tested whether, as predicted by some literature, older adults would disproportionately benefit from audio-visual (AV) information compared with visual-only (VO) or auditory-only (AO) information, relative to young adults. Participants were 78 young adults (aged 18–30 years old, M = 25.50 years), 78 older adults with normal hearing (aged 65–80 years old, M = 68.34 years), and 78 older adults who wear hearing aids (aged 65–79 years old, M = 70.89 years). There were no significant differences in the amount of information remembered across modalities (AV, VO, AO), no differences across participant groups, and we did not find the predicted interaction between participant group and modality. The older-adult groups performed worse than young adults on background measures of cognition, with the exception of a vocabulary test, suggesting that they may have been using strategies based on prior knowledge and experience to compensate for cognitive and/or sensory deficits. The findings indicate that cost-effective, text-based websites may be just as useful as those with edited videos for conveying health information to all age groups and hearing aid users.

Keywords

Online health information recall cognition multisensory information

Background

The number of people seeking health information online has increased in recent years (Chu et al., 2017) with 54% of people over the age of 75 years using the internet (Office for National Statistics [ONS], 2020). Older adults also report the internet as the most used and trustworthy source for medical information after health care professionals and pharmacists (Medlock et al., 2015). However, there are several barriers faced by older adults seeking health information online. First, accessing the information may be difficult as the current generation of older adults may have difficulty with navigating websites due to inexperience with information technology (IT) and less exposure to digital technology over their lifetime (Age UK, 2018), although there has been an increase in the amount of older adults using the internet since the COVID-19 pandemic (Age UK, 2021). Cognitive decline such as deficits in working memory, problem-solving, and attention can also make it difficult for older adults to use websites (Strong et al., 2001). Second, sensory deficits may inhibit the ability to comprehend health information. In the United Kingdom, over 80% of older adults aged 70 and above have hearing loss (Royal National Institute for Deaf People, 2024), and ~80% of older adults aged 65 and above have visual impairments, including those with corrected vision (glasses) and those with uncorrected sight loss (Royal National Institute of Blind People, 2022). Third, health information must be remembered before it can be acted upon, and this may be difficult for older adults who experience cognitive deficits. Working memory and processing speed, which are needed for comprehension, have been found to decline in older adults compared with young adults (e.g., Luo & Craik, 2008). Finally, older adults encounter more physical health problems than young adults (Jaul & Barron, 2017) and may therefore have to remember multiple pieces of complex medical information, resulting in increased cognitive load. Given these challenges, it is important to understand how best to present online health information to older adults.

There is converging evidence that suggests older adults may benefit more than young adults from multiple sources of sensory information, compared with information in just one modality (see de Dieuleveult et al., 2017 for a systematic review). For example, audio-visual (AV) stimuli (images and audio) have been found to facilitate problem-solving for older adults compared with visual-only (VO) stimuli (text and images) through reducing cognitive load (van Gerven et al., 2006). AV information has also been found to improve recall for older adults. Frieske and Park (1999) presented news items in different modalities: auditory-only (AO; radio), VO (newspaper), and AV (TV). Although young adults had better recall than older adults in all conditions, the AV stimuli improved recall for older adults compared with unisensory conditions. In addition, reduced auditory and visual acuity, as well as processing speed, accounted for age differences in recall. AV (pictures with spoken words) stimuli have also been found to enhance recall for words compared with sounds or spoken words alone for both young and older adults (Heikkilä et al., 2018). This improvement was more apparent for older adults compared with young adults. This is in keeping with Mayer’s (2009) modality principle of multimedia learning, which suggests that learning is improved when information is multimodal, for example, written text (visual information) combined with spoken words (auditory information).

Although considerable evidence suggests that older adults should benefit from multisensory information compared with young adults, it is also important to acknowledge emerging evidence that suggests a lack of age differences in multisensory perception. Atkin et al. (2023) found no evidence of age differences when replicating an established multisensory ageing effect (Laurienti et al., 2006) using a speeded perceptual discrimination task. In addition, Badham et al. (2024) found convincing evidence for a lack of age differences in multisensory processing in several experiments that measured associative memory. Therefore, it is important to explore the specific tasks/contexts in which older adults may benefit from multisensory information.

A multisensory benefit for older adults has been found in studies that focus on memory for health information. Bol et al. (2015) investigated the influence of modality and narration style (formal vs informal) on recall. They found that AV information increased recall of health information compared with VO (written text) for both young and older adults. The combination of AV stimuli and conversational narration style resulted in better recall for all participants. These results are supported by research in clinical settings where patients with lung cancer remembered more medical information when presented with video and text compared with text alone (Bol et al., 2013). Young adults also recalled more information compared with older adults, but not when the authors controlled for internet use.

AV stimuli may also be particularly relevant for older adults with hearing aids (HAs). McCoy et al. (2005) asked older adults with normal hearing (NH) and those with hearing loss to recall words in a list. They found that those with hearing loss could recall less words compared with NH listeners. However, correct identification of the words by the hearing loss group suggests that the deficit in recall was due to more effortful listening, which resulted in reduced ability to encode and recall information. Indeed, sensory deficits have been shown to be linked with cognitive deficits, whereby degraded visual or auditory information increases cognitive load, which, in turn, limits the cognitive resources available, and if this persists, may result in cognitive decline (see Roberts and Allen (2016) for a review). There is also evidence of multimorbidity with hearing loss and chronic health conditions including but not limited to; cancer, cardiovascular risk factors, diabetes, and stroke (see Besser et al., 2018 for a review) indicating that older adults with hearing loss may be more at risk of developing other health conditions. This emphasises the need for delivering health information in a format that people with hearing loss are able to access. Furthermore, Ferguson et al. (2015) found that a multimedia intervention (DVD for TV or computer) improved recall of specific HA information for hearing-aid users (after 6 weeks) compared with a control group who received standard care.

Taken together these findings suggest that multisensory stimuli may be a solution to overcoming the cognitive or sensory deficits associated with ageing. However, no study has investigated the influence of unisensory and AV information and recall of online health information in older adults with NH and older adults with HAs.

The current study

Given the evidence that suggests older adults benefit from multisensory information, we wanted to exploit this advantage and use AV information to enhance older adults’ recall of online health and well-being information.

Objectives

We also aimed to compare older adults with NH and older adults who wear HAs to see how sensory deficits affect recall. We aimed to compare a multisensory condition with two different unisensory conditions: a VO condition that used written words only, as this is similar to prominent health websites in the United Kingdom and may facilitate self-paced reading, which is beneficial for older adults who have slower processing speed (Frieske & Park, 1999); and an AO condition in which the information is spoken, as this could be relevant for people with visual impairments and/or those who would normally use text-to-speech software. The goals of the research are important for designing online health information on websites to help older adults overcome cognitive and sensory deficits and help them stay healthy into older age.

Hypotheses

H1. Young adults will have better recall than older adults regardless of modality.

H2. All groups will have better recall in the multisensory condition compared to unisensory conditions.

H3. There will be an interaction between age group and modality: the difference in recall between the young adult group and the older adult groups (NH; NH & HA; HA) will be smaller in the multisensory condition compared to the unisensory conditions.

H4. Older adults with HAs will benefit the most from the audio-visual information.

Method

Transparency and openness

Details of the sample size calculation are included in section “Participants.” All measures and reasons for data exclusion have been reported. In our original pre-registration document, we stated that we would compare a group of young adults with a group of older adults. After data collection, we observed null results and made the decision to collect a further participant group comprising HA users, which is reflected in the update to the pre-registration document. The analyses that follow relate to the updated pre-registration plan. The study’s original pre-registration, updated pre-registration, and data can be found on Open Science Framework (OSF) https://osf.io/jbqhc/. The research materials can be found in the Gorilla.sc repository (https://app.gorilla.sc/openmaterials/591791).

Design

The study comprised a 3 × 3 mixed design with between-subjects factor Group (young, older adults with NH, older adults with HAs) and within-subjects factor Modality (VO; AO; AV). The dependent variables were two measures of memory for health information: scores on a quiz (cued recall) and percentage correct free recall.

Participants

This study was approved by the School of Social Sciences Research Ethics Committee at Nottingham Trent University, approval no. 2020/311. Informed consent was obtained from participants. The sample size calculation was conducted in R using the pwr (Champely, 2020) package. The calculation was performed for the 3 (Group) × 3 (Modality) interaction (analysis of variance [ANOVA]) using a medium effect size based on previous literature. A sample size of N = 156 (78 young adults and 78 older adults) was required. We later updated our pre-registered data analysis plan to include a sample of HA users and so we aimed to recruit an additional 78 older adults with HAs making a total of 234 participants. The sample calculation was based on a regression with four predictors so that we could assess background measures of cognitive performance against recall performance.

The inclusion criteria were as follows: English as a first language and age range 18–30 years old (Young group) or 65–80 years old (Older groups). Participants were screened for the exclusion criteria via Prolific. For mild cognitive impairment or dementia, participants were asked “Have you ever been diagnosed with mild cognitive impairment or dementia?” Only those who reported no were invited to participate in the study. Participants were also asked “Do you experience colour blindness?” They were not invited to participate if they answered yes.

Four participants were excluded (three because they did not meet the inclusion criteria, and one because the audio portion of the study did not work) and four replacement data sets were collected. Two-hundred and thirty-four participants were included in the final data set; participant characteristics are reported in Table 1. The experiment was designed and hosted on Gorilla Experiment Builder (www.gorilla.sc) (Anwyl-Irvine et al., 2018). Data were collected between October 2021 and December 2022. Young and older adults with NH were recruited through Prolific; older adults with HAs were recruited through the Nottingham Biomedical Research Centre participant panel. Participants were paid £10 via Prolific or given a £10 shopping voucher.

Table 1.

Participant demographics.

	Young	Older NH	Older HA
Mean age in years	25.5	68.3	70.8
		N
Sex
Female	33	40	39
Male	45	36	39
Education
GCSES/O-levels	6	16	17
A-levels or equivalent, e.g., Scottish Highers	19	16	7
National Vocational Qualification (NVQ)		9	8
Degree/degree apprenticeship	32	18	14
Masters’/PhD/postgraduate diploma	16	12	12
Visual acuity
Excellent	43	13	5
Fair	2	12	21
Good	28	50	51
Poor	4	0	0
Glasses/contacts worn	20	60	62
Hearing Screening 1
Yes	1	14	75
No	77	62	3
Hearing Screening 2
Yes	8	18	72
No	70	58	6
Hearing Screening 3a
Cannot hear at all	0	0	3
With great difficulty	0	0	9
With moderate difficulty	1	3	24
With no difficulty	73	64	4
With slight difficulty	4	9	30
Hearing Screening 3b
Cannot hear at all	0	0	2
With great difficulty	0	0	16
With moderate difficulty	1	2	19
With no difficulty	73	62	5
With slight difficulty	4	12	28

Stimuli

Health and well-being information was adapted from National Health Service (NHS) websites. The NHS is the publicly funded health care system in the United Kingdom, and the main NHS website is one of the key places people seek health information with an average of 28 million views per week (NHS Digital, 2022).

Pilot study

Topics were determined by what is readily available on NHS websites according to what this health organisation considers to be important. Older adult participants (n = 5) recruited via the Nottingham Biomedical Research Centre participant panel answered questions on six health and well-being topics (14 questions on each) without being given any information; this was to test their prior knowledge. As participants scored on average 6.4 out of 14 (almost half on the topic “How to sit at your desk correctly”), this topic was omitted from the study. The remaining topics, healthy eating (M = 3.4) example question “A portion of fruit is approximately __ grams,” Vitamin D & Sunlight (M = 4.0) example question “Who might need to take vitamin D supplements?,” mindfulness (M = 3.9) example question “Where has evidence shown that mindfulness works?,” time management (M = 3.6) example question “The three Ds are: _______, ______ and ______,” and Power of attorney (M = 3.5) example question “If the Enduring Power of Attorney has been registered, who do you need to get permission from to cancel it?,” were included. Power of attorney is relevant for all age groups, as an individual may become incapacitated at any point in their life and may need someone to manage their finances. All information was replicated from the relevant NHS websites except for health advice relating to children which was omitted. For the AV condition, we replicated the information on the NHS websites, which is presented in a question and answer format, and created videos using actors designed to simulate a GP and patient consultation, in which the patient asked the GP questions using a formal speech style.

Video

The video stimuli were 24 videos (4–5) per topic in .mp4 format, approximately 20 s each in duration each, resolution 1920 × 1080 pixels, and filled ~85% of the screen as presented to participants.

Audio

The audio stimuli were the audio track taken from the video file, sample rate 48,000 Hz, stereo, .mp3 files.

Visual

The content of the VO stimuli consisted of the script from the videos in black font on a white background. Html was used to denote font size which varied according to the screens on participants’ devices.

The content was the same regardless of modality, the duration of content varied across the different stimuli as the websites involved text only and reading is self-paced whereas the audio and video stimuli were the same length.

Participants also completed a background battery of measures described in detail in the following sections. We included both hearing and vision screening to gather demographic information; we also included a subjective measure of hearing and an objective measure of hearing (perceptual measures), as well as several cognitive tasks which we planned to use both these perceptual and cognitive measures for further analysis.

Questionnaires

Self-reported vision

Self-reported vision was a single item question “Please rate your present eyesight with glasses/contact lenses if you use them” rated on a scale of: very poor, poor, fair, good, and excellent. Participants who wore glasses/contacts also confirmed that they were wearing glasses/contacts while completing the study.

Hearing screening questionnaire

The questionnaire (Davis et al., 2007) includes four questions: (1) “Do you have any difficulty with your hearing?” (2) “Do you find it very difficult to follow a conversation if there is background noise (such as TV, radio, children playing)?” These questions require a yes or no response: (3a) “How well do you hear someone talking to you when that person is sitting on your right side in a quiet room?” and (3b) “How well do you hear someone talking to you when that person is sitting on your left side in a quiet room?” Possible responses were with no difficulty, with slight difficulty, with moderate difficulty, with great difficulty, cannot hear at all.

Speech, spatial and qualities of hearing scale

The Speech, Spatial and Qualities of Hearing Scale (SSQ12) (Noble et al., 2013) measures hearing and listening in different situations and includes 12 questions which are rated on a scale from 0 to 10. A higher score on this questionnaire indicates greater listening difficulties.

Cognitive tasks

The following tasks were chosen because previous work has found differences between young and older adults. In particular, vocabulary tends to increase with age (Kavé, 2024; Verhaeghen, 2003); this allows us to measure the possibility of testing an unusually less able group of older adults if their vocabulary is worse than the young group. The remaining measures speed (letter comparison task), executive function (cued task switching), and working memory (N-back) are all cognitive measures known to decline with age (e.g., see Murman, 2015, for review). Therefore, these are most likely to correspond to the age differences in episodic memory being measured in the current study.

Mill Hill Vocabulary test

Similar to the paper version of the Mill Hill Vocabulary test (Raven et al., 1988), words are listed on screen at the same time, and for each word, the participant must identify the word with the closest meaning from a choice of six words and show their response by highlighting a circle next to the word of their choice. The task is scored out of 33.

Letter comparison task

To measure visual processing speed, an online version of the letter comparison task (Salthouse & Babcock, 1991) was created for this study. Participants were given 30 s to identify whether pairs of strings were the same or different by pressing “J” on the keyboard for same or “F” for different. For example, a pair that was the same would be “RXL RXL” and a pair that was different might be “RFL RXL.” The strings would stay on the screen until a key was pressed. There were six practice trials with three letter strings. For the main task, there were 20 × 3 letter strings, 20 × 6 letter strings, and 20 × 9 letter strings, 60 trials in total. On half of the trials, the strings were the same, and on the other half, they were different. The stimuli for this task were created by generating random strings, which were then checked and omitted if they contained double characters, words or well-known abbreviations, as this may make them easier to distinguish. The letters were displayed in Courier Sans Serif font (size varied according to participant devices) and displayed in the centre of the screen.

Cued task switching

In the Cued task switching task (Rogers & Monsell, 1995, adapted by Gorilla.sc), participants are asked to respond to either colour or shape. A rectangle or square was displayed, which was either green or blue. If asked to respond to the shape, participants would press on the keyboard “F” for square and “J” for rectangle. If asked to respond to the colour, participants would press “F” key for blue and “J” for green. There were four practice trials and 16 experimental trials. At the start of a trial, the word colour or shape would appear in the centre of the screen for 500 ms, followed by a fixation cross for 500 ms; the shape would then appear and remain on the screen until the participant responded.

N-back (two-back)

To assess visual working memory, we used the N-back task (Kirchner, 1958, adapted by Gorilla.sc) in which single letters appear on the screen; the participants’ task is to press “J” on the keyboard when the letter is the same as the letter displayed two places before. If the letter is not the same, they press “F” on the keyboard. There were 10 practice trials and 100 experimental trials. Feedback was displayed in the form of a thumbs up (correct) or thumbs down (incorrect) for 400 ms; if there was no response, the screen advanced automatically after 2,000 ms. The participant’s score was displayed at the end of the task.

Adaptive speech-in-noise listening task: coordinate response measure variant

We used the coordinate response measure (CRM) variant of the adapted speech in noise task (Bianco et al., 2021). Evidence suggests adaptive listening in noise tasks is a valid measure of hearing loss as they produce speech reception thresholds (SRTs), which have been associated with traditional measures of hearing loss such as the digit triplet test and audiometric thresholds (Semeraro et al., 2017). Compared with the original task, we increased the luminance of the green colour and used two blocks of trials. In this task, the talker states a colour and a number, for example, “show the dog where the red six is,” the participant then has to identify the number they heard from 1 to 9 (excluding 7 because it has two syllables) by clicking on a coloured number. Participants were given visual feedback after every trial in the form of a happy or sad face, and an overall score at the end of each block. There were two blocks in total. The speech was presented in a one-up one-down adaptive track using a threshold of 50% correct (Levitt, 1971). Two-talker babble was presented at fixed signal-to-noise ratios (SNRs) starting at 20 dB. The first two reversals were in steps of 9 dB, after the first two reversals, this decreased by 2 dB and then by 3 dB for the remaining trials. There were seven reversals in total or 25 trials, whichever was reached first. The SRTs were calculated as in Bianco et al. (2021) by averaging across the last four reversals.

Procedure

All participants were provided with an electronic information sheet and consent form and were asked to provide a unique identifier between 1 and 8 characters long and containing letters and numbers. The demographics collected included age, highest level of education, what hearing devices they are using to complete the study if any, and what glasses or contacts they are wearing to complete the study, if any, and if they wear hearing devices/glasses/contacts on a daily basis. Participants then completed the self-reported vision, hearing screening, and SSQ12 questionnaires. Prior to the main tasks, a speaker check was completed which allowed participants to play an audio file to check that their speakers were working and adjust the volume to a comfortable level.

The recall task consisted of three different conditions in which information to be remembered was presented either AO (voice recording), VO (text), or AV (video). The information included in these conditions consisted of three randomly selected topics out of five possible topics: healthy eating, Vitamin D & Sunlight, mindfulness, time management, and Power of attorney. The order of modality (AO, VO, AV) was randomised, and the order of topics was counterbalanced with five possible condition orders and participants were assigned to each condition order in groups of 5. The recall stage proceeded after each topic and included two parts; first participants answered 10 comprehension questions relating to the information provided, followed by a free recall task in which participants could type out as much of the information as they remembered.

The cognitive tasks were then completed in the following order: Mill Hill Vocabulary test, letter comparison task, cued task switching, N-back (two-back), and adaptive speech-in-noise task. After the final test, participants were thanked and paid for their time. The whole experiment took approximately 45 min to complete.

Results

Table 2 reports the results of the one-way independent groups ANOVA used to test for differences in performance on each of the cognitive tasks. Significant results were explored with t-tests using the Holm adjustment for multiple comparisons. Older adults with HAs reported worse self-reported listening difficulties compared to young adults and older adults with NH (all ps < .001). Older adults with NH and older adults with HAs scored significantly higher on the vocabulary test compared with young adults (all ps < .001). Older adults with HAs had the highest SRTs (i.e., needed less noise to understand speech) followed by older adults with NH, and then young adults (all ps < .001). Young adults scored higher on the cued task switching compared to older adults with HAs (p = .002). Young adults scored higher on the N-back task compared to older adults (p = .027) and older adults with HAs (p < .001), and older adults with NH scored higher than older adults with HAs (p = .027). Young adults were more accurate on the letter comparison task compared to older adults (p < .001) and older adults with HA (p < .001).

Table 2.

Descriptive statistics for the questionnaires, scores on the cognitive tasks, and results of the independent groups one-way ANOVA tests with effect sizes.

	Young			Older NH			Older HA
	M	SD	N	M	SD	N	M	SD	N	Group difference
Age	25.00	3.69	78	68.34	3.09	76	70.89	3.72	78	F	p	η²
SSQ12^a	7.87	1.13	78	7.44	1.59	76	4.73	1.71	77	99.71	< .001^b	.467
Mill Hill Vocabulary	18.45	4.59	78	22.80	3.67	76	24.36	3.80	69	42.80	< .001^b	.280
SRT (SNR)^a	−11.50	3.72	76	−7.06	4.77	74	−3.98	7.48	68	34.38	< .001^b	.242
Task switch	13.50	2.64	78	12.64	2.92	76	11.76	3.42	68	6.09	.003^b	.053
N-back	80.01	15.28	78	73.57	16.58	76	66.37	21.15	77	11.32	< .001^b	.090
Letter comparison	13.85	4.03	78	11.03	3.36	76	9.84	4.79	77	19.51	< .001^b	.146

SRT: speech reception threshold, SNR: signal-to-noise-ratio, SSQ12: Speech, Spatial and Qualities of Hearing scale.

Significance remains the same after Bonferonni’s adjustment; unadjusted p-values are reported.

Data coding

Free recall

Data were coded using a method in which video transcripts were condensed into units of information, where each unit relates to an item of semantic information to be recalled. Units were scored from 0 to 2. Answers were assigned a score of 2 if the text was remembered verbatim, 1 if some information was missing or altered, and zero if the information was completely inaccurate or missing. For example, a score of 2 would be: “Aim for 5 fruit and veg a day (400 g).” A score of 1 could be: “5 fruit & veg a day (300 g)”. A score of zero could be: “3 fruit & veg a day.” The scores were then totalled and converted into a percentage. Ten percent of the data (N = 24) were coded by a second rater. Inter-rater reliability was assessed by intra-class correlations (ICCs; Koo & Li, 2016) which showed that the ICC was .92 (95% confidence intervals [CIs] = [0.54, 0.99]) indicating excellent reliability. An example of the free recall coding is provided in the online Supplementary Material.

Quiz score

An example comprehension question was: Q. The government recommends that we eat five fruit and veg a day, which is the equivalent of ___ grams. Half points were awarded for partially correct information. Scores on the quiz were totalled (maximum score of 10) and converted to a percentage.

Analysis

The Bonferroni correction was applied for multiple comparison; unadjusted p-values are reported unless otherwise stated. Results were analysed using JASP (JASP Team, 2022) version 0.11.1. Plots were created using ggplot2 (Wickham, 2016) in R version 1.2.5042 (R Core Team, 2021).

To test the three hypotheses, we conducted a 3 × 3 mixed measures ANOVA with between-subject factor Group (young, older + NH, older + HA) and within-subjects factor Modality (AV, AO, VO) with the dependent variable scores on comprehension questions. Median scores for the comprehension questions are shown in Figure 1 which shows participants scored approximately the same in the AO and VO conditions (scores were not at ceiling). Table 3 shows the results of the ANOVA with accompanying effects sizes. The Bayes factors are provided and interpreted using the classification scheme developed by Lee and Wagenmakers (2014). Results showed that there were no significant effects of modality or age group with strong evidence in favour of the null hypothesis. There was no significant interaction (Modality*Group) and extreme evidence in favour of the null hypothesis.

Figure 1.

Box plots of scores on the comprehension questions in each modality error bars show 95% confidence intervals.

Table 3.

Results of the repeated measures ANOVA quiz scores and free recall.

Quiz scores	Sum of squares	df	Mean square	F	p	η²	BF₁₀
Modality	8.161	2	4.080	1.394	.249	.003	.064
Modality * Group	20.787	4	5.197	1.776	.133	.007	.005
Group	9.03	2	6.285	0.976	.378	.008	.084
Free recall
Modality	79.027	2	39.514	0.766	.466	.001	.035
Modality * Group	796.007	4	56.756	1.100	.356	.004	.015
Group	227.025	2	398.00	2.537	.081	.022	.435

For the free recall data, we conducted a 3 × 3 mixed measures ANOVA with between-subjects factor Age group (young, older + NH, older + HA) and within-subjects factor Modality (AV, AO, VO) and percentage of free recall as the dependent variable; results are reported in Table 3. The median free recall scores in the different modalities are depicted in Figure 2 which shows that participants remembered a similar amount of information on average in each condition (scores not at ceiling). We found no significant effect of modality and no significant interaction (Modality * Group) with strong evidence in favour of the null hypothesis, and no significant effect of Group with anecdotal evidence in favour of the null hypothesis.

Figure 2.

Box plots of percentage correct free recall in each modality error bars show 95% confidence intervals.

Exploratory analyses

Our pre-registered data analysis plan stated that if the interaction was significant, we would conduct a regression analysis for older adults only using the outcome variable AV benefit (AV − [AO + VO]/2; Dias et al., 2021) and perceptual and cognitive test scores as predictors. As the results did not support our hypotheses, we did not proceed with our regression analysis. Instead, we conducted some exploratory analyses. First, we investigated how much information each group reported in each condition. The mean number of words recalled is shown in Table 4. There were no significant differences in the amount of words recalled between young adults, older adults with NH, and older adults with HA with anecdotal to moderate evidence in favour of the null hypothesis.

Table 4.

Word counts on the free recall task in each modality.

	Young			Older			HA			Group differences
	M	SD	N	M	SD	N	M	SD	N	F	df	p	BF
AV	79.436	50.458	78	66.474	35.841	76	64.351	40.761	77	2.81	2	.062	0.553
AO	75.013	54.474	78	67.368	38.972	76	67.256	46.666	78	0.69	2	.503	0.765
VO	71.154	50.399	78	63.487	42.001	76	60.833	40.540	78	1.13	2	.326	0.123

Exploratory correlations

To better understand the relationship between hearing (SRTs) and performance on the recall task in each modality (AV, VO, AO), we conducted Spearman’s correlations and found a significant weak negative correlation between SRTs (better hearing corresponds to better comprehension, high SRTs indicate poorer hearing) and comprehension scores in the AO condition r = −.38, p = .002, BF₁₀ = 321.55 and the VO condition r = −.26, p = .037, BF₁₀ = 0.40. There was no significant relationship between SRTs and comprehension scores in the AV condition (p = .26). There were no significant relationships between SRTs and free recall scores in any of the conditions and strong evidence in favour of the null hypothesis (all BF₁₀ = 0.1)

Discussion

The aim of the current research was to investigate whether AV information improved older adults’ recall of health and well-being information, compared with VO or AO information. In our pre-registered hypotheses, we expected young adults to remember more information than older adults with NH and older adults with HA in all modality conditions. We expected that all groups would recall more health and well-being information in the multisensory condition compared with the unisensory conditions. We also expected to find a greater multisensory benefit for older adults with NH and, in particular, older adults with HA, compared with young adults. If a multisensory benefit was found, we had planned to explore this using scores on the cognitive tasks as predictors of recall. We found that young adults outperformed both groups of older adults on all background cognitive tasks, but we were surprised to find that there was no evidence of differences in recall between young adults and older adults with NH or older adults with HA. These results are at odds with the literature which suggests that older adults will show a deficit in recall of health information (Bol et al., 2013) and may disproportionally benefit from multisensory information compared with young adults (e.g., Heikkilä et al., 2018). We expected to observe a deficit in the unisensory conditions and that a multisensory benefit would improve older adult performance making it akin to that of young adults; however, in the current study, performance was similar in both age groups meaning there was no observed improvement for the older adult group.

We suggest several reasons why we may have found no differences in recall between young and older adults in the current study. The present results are in line with McGillivray et al. (2015) who found no age difference in recall of trivia. They asked young and older adults to rate their interest in the answers to trivia questions and found that for both age groups; interest was related to memory. In addition, the predictive ability of interest increased when recall was delayed from 1 h to 1 week for older adults, but this decreased for young adults. This suggests that interest in topics is important for older adults’ memory and that this effect may only be apparent over time, whereas in the present study, we used immediate recall.

The type of health information we used was replicated from NHS websites covering a broad range of topics intended to provide enough information for people to look after their physical and emotional well-being. However, the type of health and well-being information used in the present study differed to that of pervious research. A systematic review (Stacey et al., under review) found that AV information improved knowledge of patients’ treatment options compared with AO or VO information. Similarly, Bol et al. (2018) used information regarding a new treatment of lung cancer. These types of health information may include more complex or novel information, and in this context, multisensory information may facilitate recall.

In the present study, we piloted the content to check that information to be remembered was sufficiently challenging and to measure familiarity with the information. As participants randomly completed three out of five possible topics, this should have decreased the likelihood that participants would have prior knowledge of all topics. However, older adults may have had more prior health knowledge compared to young adults due to their personal health experience or health experience from friends or family (Jaul & Barron, 2017). Chin et al. (2015) investigated the role of health literacy (understanding and acting on health information) and the ability of older adults to remember self-care information. They found that general knowledge and health knowledge mediated the relationship between health literacy and recall of health information. The authors (Chin et al., 2015) suggest that prior knowledge can offset deficits in processing capacity experienced by older adults. Indeed, Badham et al. (2016) found that prior knowledge disproportionately benefitted older adults when they were asked to recall semantically logical or illogical sentences.

Consistent with the possibility that prior health beliefs may affect the amount of information recalled, participants remembered on average ~14% of information in the free recall condition, which was lower than expected. Several studies have found that participants recall less health information when they are given conflicting information (Barnwell et al., 2022; Rice & Okun, 1994). There is some indication in the present study that the information provided may have conflicted with some participants’ prior health beliefs. For example; one participant wrote that they disagreed with the information stating “as you can tell I’m a sceptic.” This may have caused confusion and affected the participant’s ability to recall the health and well-being information.

Prior experience may also be important in relation to the VO condition which included online written text in a website format. In our sample, older adults scored higher on the Mill Hill Vocabulary task compared to young adults which is to be expected as older adults have more literacy experience (Verhaeghen, 2003). Payne et al. (2012) found that older adults with higher literacy experience (print exposure) were able to recall more sentences compared to those with lower literacy experience. Therefore, increased print exposure appears to provide a compensatory mechanism for older adults with working memory deficits and facilitates recall. This may explain why older adults recalled the same amount of information as young adults in the VO condition. As both older adult groups performed worse on all the other cognitive tasks and hearing tests compared with the young adult group, we tentatively suggest that older adults may have been using strategies such as prioritising information, note-taking, rehearsal, or association to compensate for their sensory and cognitive decline, although we did not test for this.

Finally, we would like to propose an optimistic interpretation of our results, which is that for the older adults in our sample, age-related deficits in short-term working memory did not impair their ability to recall health and well-being information. This is consistent with Badham (2024), who evidenced that age deficits are smaller now, than just a few decades ago. Furthermore, Verhaeghen et al. (1993) have argued that the constraints of experimental work involve designing a task, which avoids ceiling and floor effects to demonstrate age differences and that this is not reflective of real-life scenarios in which age deficits may not be present. Castel (2007) also emphasises the importance of using naturalistic tasks, as this allows older adults to employ strategies for recall that they would use in their everyday lives. As participants completed the study online and in their own homes, using similar material as encountered in everyday life, perhaps this provided enough of a realistic environment for them to use familiar recall strategies. This suggestion warrants further investigation and could form the basis of future studies to compare familiarity/unfamiliarity of topics and presentation types, for example, self-paced reading, and if these relate to recall strategies that influence age differences in memory.

Furthermore, a report from Age UK (2021) suggests that older adults are using the internet more frequently since the COVID-19 pandemic providing further opportunity to hone their technical skills, and this may have had a positive impact on their ability to use online information. There may have been no differences observed between the older adults with HAs and the older adults with NH in the AV condition and AO condition as the task was completed in quiet listening conditions and differences in recall may only be apparent when the task is more effortful (c.f., Verhaeghen et al., 1993). Our findings are important for older adults with listening difficulties, as they may demonstrate the benefits of adopting a HA.

Limitations

Several limitations of the current study should be noted. Although the focus of the present work was recall of health and well-being information and cognitive ability, there may be other important factors that could influence the recall of health information such as motivation to engage with online information. Bol et al. (2018) found that motivation was related to recall of online cancer information in a sample of older adults with cancer. They suggest that older adults who might not have much time left in life may add more weight to relevant health information, which subsequently leads to better recall. Although the health and well-being information included in the present study is important for everyday self-care, perhaps participants would be extrinsically motivated to recall health information that is directly relevant to a health issue they have. The perceived emotional valence of the health information may also be a motivating factor as older adults favour positively valenced stimuli over negatively valenced stimuli (positivity effect; e.g., Lockenhoff, 2018). Therefore, older adults may be more motivated to remember health information if it is framed in a positive way.

Future directions

Different studies use different time-frames for recall; therefore, it would be pertinent to investigate how people’s memory of health information changes over time. The present research used immediate recall to assess young and older adults’ short-term memory of health information. McGuire (1996) showed participants a video consultation with a doctor talking about osteoarthritis and found that young adults recalled more information during an immediate free recall task compared to older adults; however, when recall was delayed at two time points (1 week, 1 month), there were no differences in recall between young and older adults at either time point suggesting further research is required. Delayed recall may be more relevant for real-life contexts, for example, receiving information at a doctor’s appointment and then having to recall it later at home.

An extension of the present work could be to examine the influence of tailored health information on recall. Vromans et al. (2020) found that videos increased recall of cancer information only when they were tailored to the individual. Future research could tailor the health information to each age group. For example, one of the videos in the present study contained information on Vitamin D consumption for adults, but recommendations may change according to age as people over the age of 70 need more Vitamin D than those under 70 years of age (Meehan & Penckofer, 2014).

Implications

The finding that the modality of health and well-being information did not affect recall contributes to knowledge through understanding the most effective way to present health information to the public. The findings are also important for health care providers because they suggests that cost-effective; text-based websites may be just as useful as those with edited videos for conveying health and well-being information to all age groups.

Conclusion

We found that older adults with NH and older adults who wear HAs could recall as much online health and well-being information as young adults. We suggest that either age deficits in short-term memory were not present in the current sample or that older adults were able to use prior knowledge and experience to compensate for any age deficits in memory.

Supplemental Material

sj-docx-1-qjp-10.1177_17470218241295722 – Supplemental material for Memory for health information: Influences of age, hearing aids, and multisensory presentation

Supplemental material, sj-docx-1-qjp-10.1177_17470218241295722 for Memory for health information: Influences of age, hearing aids, and multisensory presentation by Jemaine E Stacey, Christopher Atkin, Katherine L Roberts, Helen Henshaw, Harriet A Allen and Stephen P Badham in Quarterly Journal of Experimental Psychology

Footnotes

Acknowledgements

The authors thank David R. Connolly for participating in the stimuli preparation and Sandra Smith for participant recruitment.

Author contributions

All authors were involved in the conceptualisation and designing the methodology of the work. J.E.S. was responsible for investigation, analysis, and preparation of the original draft. C.A. and K.L.R. assisted with data processing. S.P.B. supervised the project and administered funding. All authors reviewed and edited the manuscript drafts.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the Economic and Social Research Council (ESRC; grant no. ES/V000071/1, Evaluating Multisensory Stimuli as a Mechanism to Boost Cognition and Wellbeing in Old Age).

ORCID iD

Jemaine E Stacey

Christopher Atkin

Kate Roberts

Helen Henshaw

Stephen Badham

Data accessibility statement

The study was pre-registered on Open Science Framework, data can be found here https://osf.io/jbqhc/ the stimuli, and tasks used can be accessed here .

Supplementary material

The supplementary material is available at qjep.sagepub.com.

References

Age UK. (2018). Digital inclusion evidence review. https://www.ageuk.org.uk/siteassets/documents/reports-and-publications/reports-and-briefings/active-communities/rb_sept13_age_uk_digital_inclusion_evidence_review.pdf

Age UK. (2021). Digital inclusion and older people—How have things changed in a Covid-19 world? https://www.ageuk.org.uk/siteassets/documents/reports-and-publications/reports-and-briefings/active-communities/digital-inclusion-in-the-pandemic-final-march-2021.pdf

Anwyl-Irvine

A. L.

Massonnié

Flitton

Kirkham

Evershed

J. K.

(2020). Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods, 52, 388–407.

Atkin

Stacey

J. E.

Roberts

K. L.

Allen

H. A.

Henshaw

Badham

S. P.

(2023). The effect of unisensory and multisensory information on lexical decision and free recall in young and older adults. Scientific Reports, 13(1), 16575. https://doi.org/10.1038/s41598-023-41791-1

Badham

S. P.

(2024). The older population is more cognitively able than in the past and age-related deficits in cognition are diminishing over time. Developmental Review, 72, 101124.

Badham

S. P.

Atkin

Stacey

J. E.

Henshaw

Allen

H. A.

Roberts

K. L.

(2024). Age deficits in associative memory are not alleviated by multisensory paradigms. The Journals of Gerontology, Series B: Psychological Sciences and Social Sciences, 79, gbae063.

Badham

S. P.

Hay

Foxon

Kaur

Maylor

E. A.

(2016). When does prior knowledge disproportionately benefit older adults’ memory? Aging, 23(3), 338–365. https://doi.org/10.1080/13825585.2015.1099607

Barnwell

P. V.

Fedorenko

E. J.

Contrada

R. J.

(2022). Healthy or not? The impact of conflicting health-related information on attentional resources. Journal of Behavioral Medicine, 45(2), 306–317. https://doi.org/10.1007/s10865-021-00256-4

Besser

Stropahl

Urry

Launer

(2018). Comorbidities of hearing loss and the implications of multimorbidity for audiological care. Hearing Research, 369, 3–14. https://doi.org/10.1016/j.heares.2018.06.008

10.

Bianco

Mills

De Kerangal

Rosen

Chait

(2021). Reward enhances online participants’ engagement with a demanding auditory task. Sage. https://doi.org/10.1177/23312165211025941

11.

Bol

Smets

E. M. A.

Burgers

J. A.

Samii

S. M.

Hanneke

C. J. M.

De Haes Van Weert

J. C. M.

(2018). Older patients’ recall of online cancer information: Do ability and motivation matter more than chronological age? Journal of Health Communication, 23(1), 9–19. https://doi.org/10.1080/10810730.2017.1394400

12.

Bol

Smets

E. M. A.

Rutgers

M. M.

Burgers

J. A.

de Haes Hanneke

C. J. M.

Loos

E. F.

van Weert

J. C. M.

(2013). Do videos improve website satisfaction and recall of online cancer-related information in older lung cancer patients? Patient Education and Counseling, 92(3), 404–412. https://doi.org/10.1016/j.pec.2013.06.004

13.

Bol

van Weert

J. C.

de Haes

H. C.

Loos

E. F.

Smets

E. M.

(2015). The effect of modality and narration style on recall of online health information: Results from a Web-based experiment. Journal of Medical Internet Research, 17(4), e4164.

14.

Castel

A. D.

(2007). Metamemory: A theoretical framework and new findings. Psychology of Learning and Motivation, 26, 125–173. https://doi.org/10.1016/S0079-7421

15.

Champely

(2020) PWR: Basic Functions for Power Analysis. R Package Version 1.3-0. https://CRAN.R-project.org/package=pwr

16.

Chin

Madison

Gao

Graumlich

J. F.

Conner-Garcia

Murray

M. D.

. . .Morrow

D. G.

(2015). Cognition and health literacy in older adults’ recall of self-care information. Oxford University Press. https://doi.org/10.1093/geront/gnv091

17.

Chu

J. T.

Wang

M. P.

Shen

Lam

T. H.

Viswanath

Chan

S. S. C.

(2017). How, when and why people seek health information online: Qualitative study in Hong Kong. Interactive. Journal of Medical Research, 6, e7000. https://doi.org/10.2196/ijmr.7000

18.

Davis

Smith

Ferguson

Stephens

Gianopoulos

(2007). Acceptability, benefit and costs of early screening for hearing disability: A study of potential screening tests and models. Health Technology Assessment, 11(42), 1–294.

19.

de Dieuleveult

A. L.

Siemonsma

P. C.

van Erp

J. B. F.

Brouwer

A. M

. (2017). Effects of aging in multisensory integration: A systematic review. Frontiers in Aging Neuroscience, 9, 80. https://doi.org/10.3389/fnagi.2017.00080

20.

Dias

J. W.

McClaskey

C. M.

Harris

K. C.

(2021). Audiovisual speech is more than the sum of its parts: Auditory-visual superadditivity compensates for age-related declines in audible and lipread speech intelligibility. Psychology and Aging, 36(4), 520–530. https://doi.org/10.1037/pag0000613

21.

Ferguson

Brandreth

Brassington

Wharrad

(2015). Information retention and overload in first-time hearing aid users: An interactive multimedia educational solution. American Journal of Audiology, 24(3), 329–332. https://doi.org/10.1044/2015_AJA-14-0088

22.

Frieske

D. A.

Park

D. C.

(1999). Memory for news in young and old adults. Psychology and Aging, 14(1), 90–98. https://doi.org/10.1037/0882-7974.14.1.90

23.

Heikkilä

Fagerlund

Tiippana

(2018). Semantically congruent visual information can improve auditory recognition memory in older adults. Multisensory Research, 2018(3–4), 213–225. https://doi.org/10.1163/22134808-00002602

24.

JASP Team (2022). JASP (Version 0.11.1)[Computer software].

25.

Jaul

Barron

(2017). Age-related diseases and clinical and public health implications for the 85 years old and over population. Frontiers in Public Health, 5, 335. https://doi.org/10.3389/fpubh.2017.00335

26.

Kavé

(2024). Vocabulary changes in adulthood: Main findings and methodological considerations. International Journal of Language & Communication Disorders, 59(1), 58–67. https://doi.org/10.1111/1460-6984.12820

27.

Kirchner

W. K.

(1958). Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55(4), 352–358. https://doi.org/10.1037/h0043688

28.

Koo

T. K.

M. Y.

(2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.

29.

Laurienti

P. J.

Burdette

J. H.

Maldjian

J. A.

Wallace

M. T.

(2006). Enhanced multisensory integration in older adults. Neurobiology of Aging, 27(8), 1155–1163.

30.

Lee

M. D.

Wagenmakers

E.-J.

(2014). Bayesian cognitive modeling: A practical course. Cambridge University Press.

31.

Levitt

H. C. C. H

. (1971). Transformed up-down methods in psychoacoustics. The Journal of the Acoustical Society of America, 49(2B), 467–477. https://doi.org/10.1121/1.1912375

32.

Lockenhoff

C. E.

(2018). Aging and decision-making: A conceptual framework for future research—A mini-review. Gerontology (Basel), 64(2), 140–148. https://doi.org/10.1159/000485247

33.

Luo

Craik

F. I.

(2008). Aging and memory: A cognitive approach. Sage. https://doi.org/10.1177/070674370805300603

34.

Mayer

R. E.

(2009). Modality principle. Multimedia Learning, 2, 200–220.

35.

McCoy

S. L.

Tun

P. A.

Cox

L. C.

Colangelo

Stewart

R. A.

Wingfield

(2005). Hearing loss and perceptual effort: Downstream effects on older adults’ memory for speech. The Quarterly Journal of Experimental Psychology Section A, 58(1), 22–33. https://doi.org/10.1080/02724980443000151

36.

McGillivray

Murayama

Castel

A. D.

(2015). Thirst for knowledge: The effects of curiosity and interest on memory in younger and older adults. Psychology and Aging, 30(4), 835–841. https://doi.org/10.1037/a0039801

37.

McGuire

L. C.

(1996). Remembering what the doctor said: Organization and adults’ memory for medical information. Experimental Aging Research, 22(4), 403–428. https://doi.org/10.1080/03610739608254020

38.

Medlock

Eslami

Askari

Arts

D. L.

Sent

de Rooij

S. E.

Abu-Hanna

(2015). Health information-seeking behavior of seniors who use the internet: A survey. Journal of Medical Internet Research, 17(1), e10. https://doi.org/10.2196/jmir.3749

39.

Meehan

Penckofer

(2014). The role of vitamin D in the aging adult. Journal of Aging and Gerontology, 2(2), 60–71. https://doi.org/10.12974/2309-6128.2014.02.02.1

40.

Murman

D. L.

(2015, August). The impact of age on cognition. In Seminars in hearing (Vol. 36, No. 03, pp. 111–121). Thieme Medical Publishers.

41.

NHS Digital. (2022). https://digital.nhs.uk/news/2022/1.2-billion-visits-to-nhs-website-in-last-12-months

42.

Noble

Jensen

N. S.

Naylor

Bhullar

Akeroyd

M. A.

(2013). A short form of the speech, spatial and qualities of hearing scale suitable for clinical use: The SSQ12. International Journal of Audiology, 52(6), 409–412. https://doi.org/10.3109/14992027.2013.781278

43.

Office for National Statistics. (2020). https://www.ons.gov.uk/businessindustryandtrade/itandinternetindustry/bulletins/internetusers/2020#:~:text=While%20there%20has%20been%20little,%2C%20to%2054%25%20in%202020

44.

Payne

B. R.

Gao

Noh

S. R.

Anderson

C. J.

Stine-Morrow

E. A. L.

(2012). The effects of print exposure on sentence processing and memory in order adults: Evidence for efficiency and reserve. Aging, Neuropsychology, and Cognition, 19, 122–149. https://doi.org/10.1080/13825585.2011.628376

45.

R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/

46.

Raven

J. C.

Raven

Court

J. H.

(1988). The Mill Hill vocabulary scale. H. K. Lewis.

47.

Rice

G. E.

Okun

M. A.

(1994). Older readers’ processing of medical information that contradicts their beliefs. Journal of Gerontology, 49(3), 119–128. https://doi.org/10.1093/geronj/49.3.P119

48.

Roberts

K. L.

Allen

H. A.

(2016). Perception and cognition in the ageing brain: A brief review of the short- and long-term links between perceptual and cognitive decline. Frontiers in Aging Neuroscience, 8, 39. https://doi.org/10.3389/fnagi.2016.00039

49.

Rogers

R. D.

Monsell

(1995). Costs of a predictible switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124(2), 207.

50.

Royal National Institute of Blind People. (2022). https://www.rnib.org.uk/professionals/health-social-care-education-professionals/knowledge-and-research-hub/key-information-and-statistics-on-sight-loss-in-the-uk/

51.

Royal National Institute for Deaf People. (2024). Prevalence of deafness and hearing loss - RNID

52.

Salthouse

T. A.

Babcock

R. L.

(1991). Decomposing adult age differences in working memory. Developmental Psychology, 27(5), 763.

53.

Semeraro

H. D.

Rowan

van Besouw

R. M.

Allsopp

A. A.

(2017). Development and evaluation of the British English coordinate response measure speech-in-noise test as an occupational hearing assessment tool. International Journal of Audiology, 56(10), 749–758. https://doi.org/10.1080/14992027.2017.1317370

54.

Stacey

J. E.

Atkin

Badham

S. P.

Roberts

Allen

H. A.

Henshaw

(2022). Does audio-visual information result in improved health-related decision-making compared with audio-only or visual-only information? Protocol for a systematic review and meta-analysis. BMJ Open, 12, e059599.

55.

Strong

A. J.

Walker

Rogers

W. A.

(2001). Searching the World Wide Web: Can older adults get what they need? In Human Factors Interventions for the Health Care of Older Adults, Rogers

W. A.

Fisk

A. D.

, Eds. Lawrence Erlbaum Assoc., Inc., Mahwah, NJ, pp. 255–269.

56.

van Gerven

P. W. M.

Paas

van Merriënboer

J. J. G.

Schmidt

H. G

. (2006). Modality and variability as factors in training the elderly. Applied Cognitive Psychology, 20(3), 311–320. https://doi.org/10.1002/acp.1247

57.

Verhaeghen

(2003). Aging and vocabulary score: A meta-analysis. Psychology and Aging, 18(2), 332–339.

58.

Verhaeghen

Marcoen

Goossens

(1993). Facts and fiction about memory aging: A quantitative integration of research findings. Journal of Gerontology, 48(4), 157–171. https://doi.org/10.1093/geronj/48.4.P157

59.

Vromans

Pauws

Bol

van de Poll-Franse

L. V.

Krahmer

(2020). (Not) taking it personal: An experimental study into the effects of personalization and message format on the interpretation of cancer treatment side effect risks. In Abstract from 42nd Annual Meeting of the Society for Medical Decision Making. Relationships in Medical Decision Making.

60.

Wickham

(2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. ISBN 978-3-319-24277-4, https://ggplot2.tidyverse.org.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB