inFLUence: Development of a self-administered risk stratification tool for patients with influenza-like illness using a user-centered approach

Abstract

Background

Influenza is often self-limiting but leads to avoidable emergency department visits. This study describes the development of inFLUence, an online self-administered tool for stratifying influenza-like illness risk.

Methods

The tool was developed using a user-centered design approach, informed by a systematic review of existing risk stratification tools. A prototype was iteratively refined across three rapid prototyping cycles with 18 participants and evaluated by seven clinical experts through an online survey.

Results

Across three testing cycles, nearly all participants (5–6 out of 6 per round) found the tool easy to use, understandable, useful, and effective. Feedback informed consensus-based adjustments at each iteration. Clinical experts rated the relevance of the proposed questions on a 9-point scale, with median scores ranging from 5 to 9, where 9 indicated extreme relevance. The final product is a bilingual, self-administered web application for individuals aged ≥5 months, providing personalized recommendations for users presenting with influenza-like illness, regardless of the underlying viral pathogen.

Conclusion

inFLUence is a promising digital tool that could enhance patient autonomy and support more efficient use of healthcare services.

Keywords

emergency service hospital influence influenza human self assessment tool user-centered design

1. Introduction

Influenza is an acute respiratory infection responsible for seasonal epidemics and occasional pandemics, representing a major cause of medical consultations. It is primarily caused by influenza A and B viruses, and more rarely by type C.¹ Transmission occurs through respiratory droplets, aerosols, and contact with contaminated surfaces. The incubation period is usually short, ranging from one to four days. Typical clinical manifestations include abrupt fever, cough, sore throat, myalgia, headache, chills, and marked fatigue.^1,2 In most cases, the illness is mild and resolves spontaneously within seven to ten days, but serious complications such as viral or secondary bacterial pneumonia, acute respiratory distress, or decompensation of chronic conditions can occur.^2,3 These complications particularly affect young children, older adults, pregnant women, and individuals with chronic diseases or immunosuppression, and the presence of comorbidities is a major predictive factor for poor outcomes.^4–6

Beyond its clinical burden, influenza also poses a major economic challenge. In the United States, the annual cost has been estimated at more than 10 billion dollars in direct medical care and nearly 87 billion dollars in total.^4,7–9 In Canada, influenza is responsible for more than 12,200 hospitalizations and 3,500 deaths annually, along with approximately 227,935 emergency department (ED) visits, making it one of the leading causes of hospital-based consultations.^10,11 Periods of intense viral circulation exacerbate ED overcrowding, reducing the ability of healthcare facilities to respond effectively to patient needs. To optimize the resource management, health authorities encourage patients at low or moderate risk to practice self-management at home, thereby reducing avoidable contacts and viral transmission.^12,13 However, many patients with influenza-like illness (ILI) struggle to judge the severity of their symptoms and the need for medical evaluation, leading both to unnecessary ED visits and, conversely, to delays in care for those truly at risk.

Several approaches have been proposed to address this challenge, including clinical prediction rules, nurse triage algorithms, electronic decision-support systems integrated into electronic medical records, and risk stratification models.^14,15 These tools classify patients according to their likelihood of complications, thereby helping direct them toward the appropriate level of care.^5,16 Yet most of them are designed for clinicians and rarely adapted for autonomous use by the public, especially across both pediatric and adult populations.¹⁷

In this context, we developed inFLUence, a bilingual, self-administered online tool designed for both adults and children presenting with ILI symptoms, regardless of the underlying viral pathogen. Its originality lies in integrating pediatric and adult pathways within a single application, offering a user-friendly interface accessible to the public, and generating personalized recommendations based on symptoms, severity indicators, comorbidities, and regular medications. The objective of this study is to describe the development process of this tool and its operational framework.

2. Methods

The development of the inFLUence tool followed a user-centered design approach and was structured around four key steps: a systematic review, prototype creation, rapid prototyping, and an expert survey (Figure 1).

Figure 1.

Development process of the inFLUence tool.

2.1. Phase 1: Related works (systematic review)

The first step in the development of inFLUence consisted of conducting a systematic review, which had been carried out and published previously by our team.¹⁸ The objective of this review was to identify risk stratification tools developed to predict complications, in particular hospitalization and mortality, among patients presenting with ILI in ambulatory or ED settings.

The literature search was performed in Medline, Embase, Cochrane Library, and CINAHL from their inception until July 20, 2023, restricting the selection to publications in French and English. Eligible studies were required to report the development or validation of a risk stratification score or algorithm for influenza-related complications. Studies focusing on untested algorithms, undifferentiated composite outcomes, exclusively hospitalized populations, or methodological designs that did not allow an evaluation of tool performance were excluded.

Article selection was first conducted by four reviewers who screened titles and abstracts, followed by full-text analysis by two independent reviewers. Disagreements were resolved by a third reviewer. The risk of bias of included studies was assessed using the PROBAST tool. The results were summarized descriptively and displayed with plots illustrating the sensitivity and specificity of each tool. No meta-analysis was performed due to heterogeneity across studies.

In total, fourteen tools (8 predictive scores and 6 predictive models) were identified. They provided the conceptual basis for selecting candidate variables, including symptoms, severity indicators, comorbidities, and medications, and for guiding prototype development in the subsequent phases. The full methodology and detailed results of this review have been published elsewhere and are available in open access.¹⁸

2.2. Phase 2: Prototype creation

Based on the tools identified through the systematic review, an initial prototype was developed using a decision-tree structure. The selected questions formed the foundation for designing algorithms tailored to all age groups, enabling identification of infected individuals, assessment of disease severity, and stratification of both adult and pediatric patients distinguishing those requiring urgent medical attention from those likely to recover without intervention.

To ensure linguistic and conceptual consistency, the questions were first translated into English and then back-translated into French by two independent professional translators. A collaborative effort involving the two translators, a computer scientist, an information designer, a graphic designer, a research coordinator, and a clinical expert led to the creation of a digitized prototype designed to deliver personalized risk stratification for ILI based on user input. Upon completing the questionnaire, the tool instantly generates a tailored recommendation indicating whether a consultation in the ED or a primary care setting is necessary.

2.3. Phase 3: Rapid prototyping

Three iterative rapid prototyping cycles were conducted with potential users of the inFLUence tool. This approach enables prototype testing through user involvement and feedback, aiming to refine both the design and content, and to identify potential usability issues. This phase was conducted in the ED of the Centre Hospitalier de l’Université Laval (CHUL) from September 2019 to February 2020. CHUL is an academic hospital located in Québec City that provides general and specialized tertiary care to both adults and children. As part of its tertiary mission, it serves the greater Québec City area, as well as eastern Québec and northern New Brunswick. Its ED handles over 77,000 visits annually, including approximately 33,000 stretcher cases, with children under 18 accounting for 44% of all visits.

In each testing round, two adult patients, two parents of pediatric patients, and two triage nurses were invited to use the tool on tablets. Nurses were included as potential end-users, given that the tool could complement the ED triage process by supporting the initial assessment of patients with ILI. Previous evidence indicates that a minimum sample of 15 participants is sufficient to detect more than 90% of usability and ergonomic issues.¹⁹

Patients and parents with suspected ILI were identified via the ED management software (SIURGE, Logibec, Montréal, Canada) and recruited by a research assistant after triage, during weekdays. Eligibility criteria included being at least 18 years old, consulting for themselves or their child, presenting with symptoms of acute respiratory infection, and remaining ambulatory throughout their ED stay. Exclusion criteria comprised severe respiratory distress requiring urgent care, visual impairments preventing tablet use, language barriers limiting comprehension of the tool, and any cognitive or consciousness impairment.

Triage nurses were recruited a few days before their scheduled shifts and asked to use the tool twice per session on convenience-selected patients with acute respiratory infection: once for an adult and once for a child. These patients were not considered study participants, as they did not interact directly with the tool or provide feedback.

During each cycle, participants used the tool independently on a tablet without external assistance, verbalizing their reflections throughout the process (think-aloud technique). A research assistant observed and documented both verbal and non-verbal reactions. At the end of each session, individual semi-structured interviews were conducted using a pretested interview guide (Supplemental Material) with adult patients and parents of pediatric patients. For nurses, interviews were conducted at the end of their shift. Interviews explored four predefined usability criteria: (1) ease of use (simplicity, navigation, format, length), (2) comprehension (clarity of vocabulary and instructions), (3) appreciation (perceived usefulness and overall acceptability), and (4) perceived purpose achievement (extent to which the tool was considered effective for risk stratification). Participants were also invited to suggest improvements. To avoid influencing clinical care, the tool’s outputs were concealed from both participants and healthcare providers. This methodological approach was informed by previous studies.²⁰

All interviews were audio-recorded, transcribed, and analyzed using a descriptive qualitative approach. Participation in the study was voluntary, and written informed consent was obtained from all individuals prior to their involvement. The study received approval from the Research Ethics Committee of CHU de Québec-Université Laval (reference number: 2020-4562). After each test cycle, a multidisciplinary team meeting was held to review participant feedback and improve the tool before the next cycle. The team consisted of a computer scientist, designer, linguist, research coordinator, and the principal investigator.

2.4. Online expert survey

To refine the prototype, a panel of seven medical experts was assembled based on recommendations or personal contacts of the research team. The panel included pediatricians, infectious disease specialists, emergency physicians, and internists. Each expert was invited to assess the relevance of the prototype’s questions via an online survey (REDCap),^21,22 using a 9-point scale where 1 indicated not at all relevant and 9 indicated extremely relevant.

A videoconference was subsequently held to review items with median scores between 4 and 7, indicating uncertainty about their relevance, with the aim of reaching a final classification or making modifications to improve their usefulness.²³ The evaluation focused on severity signs of ILI, comorbidities and medical conditions likely to influence disease progression, as well as regular medications that may justify a medical consultation. Experts were also invited to suggest additional elements, such as other relevant severity signs, comorbidities, or medications. The prototype was further revised based on their comments and suggestions.

3. Results

3.1. Prototype creation

A series of questions was developed to address age, ILI symptoms, severity signs, health conditions, comorbidities that could worsen disease progression or support the initiation of antiviral treatment, as well as the regular use of medications that may influence prognosis.²⁴ The questions about medical history and medications were specifically designed to identify patients who might benefit from antiviral treatment, based on local guidelines in effect at the time of the tool’s development. The initial prototype was divided in 3 different algorithms for the following age groups: 5 months to 5 years, 5 to 12 years, 12 years and older. Immediately after the first classifying question, patients under 5 months of age are advised to consult a healthcare provider. The final recommendations provided to patients upon completing the tool were based on the Québec Ministry of Health’s guidelines for self-care in cases of ILI.²⁵

3.2. Rapid prototyping

A total of 18 participants took part in the three iterative testing cycles of the tool, with 6 participants per cycle: 2 adult patients, 2 parents of pediatric patients, and 2 triage nurses in each round. The mean age of participants was 36.1 years, with 12 women among the 18 participants.

Regarding ease of use and overall appreciation, across all cycles, all participants, except one in the second cycle, found the tool easy to use and intuitive at first attempt. Only one older participant, unfamiliar with tablet technology, experienced some hesitation but successfully completed the test. In the first cycle, 5 out of 6 participants appreciated the tool, while all participants found it satisfactory in the subsequent two cycles. The format, presentation, and user-friendliness were widely praised. Understanding of the tool also improved from 5 out of 6 participants during the first two cycles to 6 out of 6 in the third cycle. Some technical terms were unclear to certain users, leading to adjustments favoring more accessible language for the public.

Participants’ evaluation of the tool’s ability to meet its objective followed a similar trend, increasing from 5 out of 6 in the first cycle to 6 out of 6 in the final cycle (Tables 1 and 2). In each round, participants provided comments and suggestions, which were analyzed by the research team and experts to assess their relevance and incorporate them as needed. Across the first, second, and third cycles, 14, 21, and 35 comments were collected respectively, resulting in 8, 4, and 4 modifications to the prototype. For example, concrete improvements implemented throughout the iterations included increasing font size to facilitate reading for visually impaired users, simplifying overly technical terms to enhance comprehension for non-specialist users, and adjusting the interface to render it more intuitive and user-friendly. These refinements helped ensure that each new version of the tool was more accessible, ergonomic, and tailored to user needs.

Table 1.

Summary of participants’ assessments across three iterative testing cycles of the inFLUence tool.

Evaluation criteria	Cycle 1	Cycle 2	Cycle 3
Number of participants who found the tool easy to use	6/6	5/6	6/6
Number of participants who appreciated the tool (usefulness, simplicity, format, presentation, length, etc.)	5/6	6/6	6/6
Number of participants who understood the tool	5/6	5/6	6/6
Number of participants who stated that the tool achieved its intended purpose	5/6	6/6	6/6
Number of comments/suggestions made by participants	14	21	35
Number of changes made by the team after each round	8	4	4

Table 2.

Participant feedback on the inFLUence tool across three user testing rounds.

Evaluation criteria	Participant feedback
1st round
Ease of use	All participants found the tool easy and simple to use, except for one who felt it was not intuitive and hesitated during completion. A triage nurse noted that people today are fond of mobile apps and could easily imagine using this tool on a phone. One nurse said: “People are really into apps nowadays, and I could picture this on a phone. Users could just click on their symptoms; anyone can do that. The questions are simple, and even as they progress and feel sicker, it still stays easy to follow.”
Appreciation of the tool (usefulness, simplicity, format, layout, length, etc.)	All participants found the tool useful and simple with a good interface, except for one who felt that clicking “next” to proceed between pages was not intuitive. A 29-year-old patient remarked: “It’s not intuitive to have to press “next”. I was expecting it to go directly to the next page, but the format was fine overall.”
Understanding of the tool	Overall, participants understood the terminology used in the questions, except for one who had difficulty with terms like “cyanosis” and “drowsiness.” A triage nurse commented: “The wording needs to be simple, something the average person can understand.”
Ability to achieve intended purpose	All participants felt the tool fulfilled its intended purpose, except for one nurse who considered it too simplistic to support clinical decisions. She felt the questions might be masking other conditions and were not specific to ILI. The nurse stated: “I find it too simplistic for a decision to be made. It could also be hiding something else.”
2nd round
Ease of use	In the second round, all participants found the tool easy to use. One patient stated: “Easy-to-use tool. It matched my symptoms.”
Appreciation of the tool (usefulness, simplicity, format, layout, length, etc.)	All participants appreciated the tool. One patient remarked: “The tool wasn’t too long.”
Understanding of the tool	One participant reported difficulty understanding the term “shortness of breath” and noted that the default language of the application was not French.
Ability to achieve intended purpose	All participants agreed that the tool achieved its intended objective.
3rd round
Ease of use	All participants found the tool easy to use. One parent stated: “It’s really super easy. It’s easy to use. The questions are clear, fast, and the visuals are good.” A triage nurse commented: “Same questions as triage, easy to use.”
Appreciation of the tool (usefulness, simplicity, format, layout, length, etc.)	All participants gave positive feedback. One patient stated: “The questions are easy, the choices are clear, and it’s well done.”
Understanding of the tool	All participants reported understanding the tool.
Ability to achieve intended purpose	All participants felt the tool met its intended purpose, although one triage nurse noted that some important questions might be missing.

Moreover, numerous positive comments emerged, highlighting that the tool could help reduce ED workload and spare patients’ unnecessary travel. Triage nurses noted that the questions included in the tool corresponded to standard triage criteria, which reinforced their endorsement of the project.

3.3. Online expert survey

Nine severity signs specific to children aged 5 months to 12 years and nine others specific to individuals aged 12 years and older were evaluated by clinical experts. For children aged 5 months to 12 years, median scores ranged from 7 to 9, with three signs rated 9 and five signs rated 8. Among individuals aged 12 years and older, median scores ranged from 7 to 9, with four signs achieving a median score of 9 (Table 3).

Table 3.

Expert evaluation of the relevance of severity signs of influenza-like illness.

Severity signs of influenza-like illness	Median relevance score (Q1; Q3)*
Children aged 5 months to 12 years
Is the child having serious difficulty breathing, rapid breathing, or chest retractions?	9 (9; 9)
Does the child have severe or persistent vomiting?	8 (7; 9)
Is the child feeling pain or pressure in the chest or abdomen?	7 (6; 8)
Does the child experience dizziness when he/she stands up?	8 (8; 9)
Are the child’s lips bluish or purple?	9 (8; 9)
Is there blood in the child’s sputum?	8 (7; 9)
Has the child gone more than 8 hours without urinating?	8 (8; 8)
Is the child unusually drowsy or showing a marked decrease in activity level?	8 (8; 9)
Is the child experiencing convulsions (e.g. sudden and involuntary muscle contractions)?	9 (8; 9)
Individuals aged 12 years and older
Do you currently feel more short of breath than usual?	9 (9; 9)
Do you have severe or persistent vomiting?	7.5 (7; 8.5)
Do you have chest or abdominal pain or pressure?	7.5 (6.5; 8.5)
Do you feel dizzy when standing up?	8.5 (7.5; 9)
Do you have blue or purple lips?	9 (5; 9)
Is there blood in your sputum?	7 (6.5; 8)
Have you not urinated for more than 12 hours?	8.5 (8; 9)
Do you experience unusual drowsiness or confusion?	9 (6.5; 9)
Are you experiencing convulsions (sudden, involuntary muscle contractions)?	9 (8.5; 9)

*Q1 = first quartile; Q3 = third quartile.

Median relevance scores assigned by clinical experts to severity signs of influenza-like illness in children aged 5 months to 12 years. Scores were rated on a 9-point scale (1 = not at all relevant, 9 = extremely relevant).

Two separate lists of comorbidities (one for children aged 5 months to 12 years and one for individuals aged 12 years and older) likely to complicate ILI were submitted for expert evaluation. For children aged 5 months to 12 years, median scores ranged from 5 to 9. Notably, 11 out of 18 comorbidities received a median score of 9, while three were rated below 7. Similarly, in individuals aged 12 years and older, median scores ranged from 6 to 9, with six comorbidities achieving a score of 9 and three falling below 7 (Table 4). For instance, comorbidities such as epilepsy, spinal cord disease, and developmental delay received lower median scores. However, following a discussion between the experts and the research team, it was agreed to retain these comorbidities in the tool due to their potential to significantly impact disease progression in certain subgroups, despite their lower scores.

Table 4.

Expert evaluation of the relevance of comorbidities and regular medications.

Comorbidities potentially complicating influenza-like illness	Median relevance score (Q1; Q3)*
Children aged 5 months to 12 years
Renal dialysis	9 (9; 9)
Sickle cell anemia	9 (9; 9)
HIV (AIDS) or immunodeficiency	9 (9; 9)
Organ transplant recipient	9 (9; 9)
Active cancer	9 (9; 9)
Asthma	9 (8; 9)
Cystic fibrosis	9 (9; 9)
Heart disease	9 (9; 9)
Diabetes	8 (8; 8)
Epilepsy	5 (5; 8)
Cerebral palsy	8 (7; 9)
Muscular dystrophy	9 (9; 9)
Spinal cord disease (e.g., spina bifida)	5 (4; 7)
Kidney insufficiency	9 (7; 9)
Liver disease	7 (7; 9)
Metabolic syndrome	9 (9; 9)
Developmental delay	5 (5; 6)
Moderate to serious intellectual deficiency	7 (6; 8)
Individuals aged 12 years and older
Renal dialysis	9 (7.5; 9)
Sickle cell anemia	9 (8; 9)
HIV (AIDS) or immunodeficiency	9 (8.5; 9)
Organ transplant recipient	9 (9; 9)
Active cancer	9 (9; 9)
Pregnant woman	8 (6.5; 8.5)
Asthma	8 (7.5; 8.5)
Emphysema	8.5 (7.5; 9)
Chronic bronchitis, COPD (Chronic Obstructive Pulmonary Disease)	8.5 (7.5; 9)
Cystic fibrosis	8.5 (8; 9)
Cardiac insufficiency	9 (8; 9)
Angina or past myocardial infarction	6.5 (5.5; 7.5)
Diabetes	8 (6.5; 8)
Kidney insufficiency	7 (6.5; 8)
Liver cirrhosis	8 (6.5; 8.5)
Multiple sclerosis	7 (6; 8)
Spinal cord disease	6 (5; 6.5)
Cognitive dysfunction	6.5 (4.5; 7)
Alzheimer’s disease	7 (4.5; 7.5)
Extreme obesity (BMI** >40)	8 (6; 8)
Regular Medications (All age groups)
Daily aspirin	7 (7; 9)
Corticosteroids (e.g., Prednisone, Decadron)	8.5 (8; 9)
Chemotherapy within the last 30 days	9 (9; 9)
Biologic agents (e.g., Enbrel, Remicade, Humira)	9 (9; 9)
Methotrexate	9 (8.5; 9)

*Q1 = first quartile; Q3 = third quartil; ** BMI = Body Mass Indexe.

Median relevance scores assigned by clinical experts to comorbidities and regular medications potentially complicating influenza-like illness. Scores were rated on a 9-point scale (1 = not at all relevant, 9 = extremely relevant).

Regarding regularly used medications, median scores ranged from 7 to 9, with three medications receiving a median score of 9 (Table 4). Additionally, the experts’ comments and suggestions were reviewed by the research team to assess their relevance and consider their incorporation into the tool.

3.4. Description of the final tool

Following all design stages, the final tool, titled inFLUence, was developed as a web application offering a self-administered digital questionnaire. Its programming ensures compatibility across all devices, including computers, smartphones, and tablets. The hosting and deployment of the application were managed through the OpenShift platform,²⁶ while server infrastructure and data storage were handled by Valeria. The beta version of the application in both French and English can be accessed online via the following link: https://ul-val-prj-influence-frontend.dev.valeria.science/.

The interface, intentionally minimalist with black text on a light background and enhanced by dynamic animations, facilitates navigation. Questions are presented sequentially and tailored to the selected age group: under 5 months, 5 months to 5 years, 5 to 12 years, or 12 years and older.

Based on the user’s responses and the underlying algorithm, the application can stop at any point to provide a recommendation tailored to the user’s risk profile, regardless of the causal virus. These recommendations include home rest, calling 811, seeking medical consultation, visiting the ED, or calling 911 (Table 5 and Figure 2). For example, patients under 5 months of age or over 75 years are systematically directed to seek professional medical advice. Similarly, the presence of any severity sign, comorbidity, regular medication, or persistent fever lasting more than eight days triggers a specific recommendation.

Table 5.

Questions and recommendations of the inFLUence tool.

Questions	Recommandations
1- Your age (or age of the child with the illness):	If 5 months or less: This questionnaire is not intended for infants. If you are worried, call Info-Santé at 811 or consult a healthcare professional.
5 months or less
5 months to 5 years	If 5 months to 5 years, go to Question 2
5 years to 12 years	If 5 years to 12 years, go to Question 22
12 years or older	If 12 years or older, go to Question 41
Questions concerning infants or young children (5 months to 5 years old)
2- Does the child have a fever? Yes/No	No matter the answer, go to Question 3.
3- Does the child have any of the following symptoms?	If yes to at least one symptom, go to Question 5.
Cough Yes/No	If no to all symptoms and yes to Question 2, go to Question 4.
Sore throat Yes/No	If no to all symptoms and no to Question 2: The child is at minimal risk of complications. Make sure the child rests and drinks sufficiently. For additional advice, consult a self-care guide. If you are worried, call Info-Santé at 811 or consult a healthcare professional.
Stuffy or runny nose Yes/No
Muscle ache, backache, pain in the arms or legs Yes/No
4- Has the child been feverish for at least 8 days? Yes/No	If yes: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
4- Has the child been feverish for at least 8 days? Yes/No	If no, go to Question 6.
5- Have the child’s symptoms persisted for at least 8 days? Yes/No	If yes: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
	If no, go to Question 7.
6- Is the child dealing with any of the following conditions:	If yes to at least one health condition: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Kidney dialysis Yes/No	If no to all of these conditions, go to Question 21.
Sickle-cell anemia Yes/No
HIV (AIDS) or immunodeficiency Yes/No
Organ transplant Yes/No
Active cancer Yes/No
7- Is the child having serious difficulty breathing (e.g. difficulty saying single words, eat or drink)? Yes/No	If yes to any of these warning signs: The child is at considerable risk of complications. Immediately, consult a healthcare professional or take the child to a hospital emergency ward or call 911.
8- Is the child vomiting violently or frequently? Yes/No	If no warning signs, go to Question 17.
9- Is the child drinking less than 50% of his usual intake when nursing? Yes/No
10- Has the child gone more than 8 hours without urinating? Yes/No
11- Does the child cry without tearing? Yes/No
12- Is the child lethargic (e.g. excessive drowsiness and decreased level of awareness?) Yes/No
13- Is the child feeling pain or pressure in the chest or abdomen? Yes/No
14- Are the child’s lips bluish or purple? Yes/No
15- Is the child experiencing convulsions (e.g. sudden and involuntary muscle contractions)? Yes/No
16- Did the child’s symptoms improve and then deteriorate again? Yes/No
17- Does the child have any of the following diseases?	If yes to at least one of these comorbidities: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Asthma Yes/No	If no to all, go to Question 18.
Cystic fibrosis Yes/No
Heart disease Yes/No
Diabetes Yes/No
Active cancer Yes/No
18- Does the child have any of the following conditions?	If yes to at least one of these health conditions: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Epilepsy Yes/No	If no to all, go to Question 19.
Cerebral palsy Yes/No
Muscular dystrophy Yes/No
Spinal cord condition (e.g. spina bifida) Yes/No
19- Does the child have any of the following diseases:	If yes to at least one of these comorbidities: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Sickle-cell anemia Yes/No	If no to all, go to Question 20.
Kidney insufficiency Yes/No
Liver disease Yes/No
HIV (AIDS) or immunodeficiency Yes/No
Metabolic disease Yes/No
20- Does the child have any of the following conditions:	If yes to at least one of these health conditions: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Slow development and/or moderate to serious intellectual deficiency Yes/No	If no to all, go to Question 21.
Organ transplant Yes/No	If no to all, go to Question 21.
21- Is the child being given any of the following medications:	If yes to at least one of these regular medications: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Aspirin every day Yes/No
Corticosteroids (e.g. Prednisone, Decadron) Yes/No
Chemotherapeutic agents during the past 6 months Yes/No	If no to all medications: The child is at minimal risk of complications. Make sure the child rests and drinks sufficiently. For additional advice, consult a self-care guide. If you are worried, call Info-Santé at 811 or consult a healthcare professional.
Biological response modifiers (e.g. Enbrel, Remicade, Humira) Yes/No
Methotrexate Yes/No
Questions concerning children 5 to 12 years old
22- Does the child have a fever? Yes/No	No matter the answer, go to Question 23.
23- Does the child have the following symptoms?	If yes to at least one symptom, go to Question 25.
Cough Yes/No	If no to all symptoms and yes to Question 22, go to Question 24.
Sore throat Yes/No	If no to all symptoms and no to Question 22: The child is at minimal risk of complications. Make sure the child rests and drinks sufficiently. For additional advice, consult a self-care guide. If you are worried, call Info-Santé at 811 or consult a healthcare professional.
Stuffy or runny nose Yes/No
Headache Yes/No
Muscle ache, backache, pain in the arms or legs Yes/No
24- Has the child been feverish for at least 8 days? Yes/No	If yes: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
24- Has the child been feverish for at least 8 days? Yes/No	If no, go to Question 26.
25- Have the child’s symptoms persisted for at least 8 days? Yes/No	If yes: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
	If no, go to Question 27.
26- Is the child dealing with any of the following conditions:	If yes to at least one health condition: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Kidney dialysis Yes/No	If no to all these health conditions, go to Question 40
Sickle-cell anemia Yes/No
HIV (AIDS) or immunodeficiency Yes/No
Organ transplant Yes/No
Active cancer Yes/No
27- Is the child having serious difficulty breathing (e.g. shortness of breath at rest or difficulty saying single words)? Yes/No	If yes to at least one of these warning signs: The child is at considerable risk of complications. Immediately, consult a healthcare professional or take the child to a hospital emergency ward or call 911.
28- Is the child vomiting violently or frequently? Yes/No	If no to all these warning signs, go to Question 37.
29- Is the child feeling pain or pressure in the chest or abdomen? Yes/No
30- Does the child experience dizziness when he/she stands up? Yes/No
31- Are the child’s lips bluish or purple? Yes/No
32- Is the child spitting up blood? Yes/No
33- Has the child gone more than 8 hours without urinating? Yes/No
34- Is the child lethargic (e.g. excessive drowsiness and decreased level of awareness)? Yes/No
35- Is the child experiencing convulsions (e.g. sudden and involuntary muscle contractions)? Yes/No
36- Did the child’s symptoms improve and then deteriorate again? Yes/No
37- Does the child have any of the following diseases:	If yes to at least one of these comorbidities: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Asthma Yes/No	If no to all comorbidities, go to Question 38.
Cystic fibrosis Yes/No
Heart disease Yes/No
Diabetes Yes/No
Active cancer Yes/No
38- Does the child have any of the following neuromuscular conditions:	If yes to at least one of these health conditions: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Epilepsy Yes/No	If no to all these conditions, go to Question 39.
Cerebral palsy Yes/No
Muscular dystrophy Yes/No
Spinal cord condition (e.g. spina bifida) Yes/No
39- Does the child have any of the following diseases:	If yes to at least one of these comorbidities: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Sickle-cell anemia Yes/No	If no to all comorbidities, go to Question 40.
Kidney insufficiency Yes/No
Liver disease Yes/No
HIV (AIDS) or immunodeficiency Yes/No
40- Is the child being given any of the following medications:	If yes to at least one of these regular medications: The child is at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Corticosteroids (e.g. Prednisone, Decadron) Yes/No	If no to all medications: The child is at minimal risk of complications. Make sure the child rests and drinks sufficiently. For additional advice, consult a self-care guide. If you are worried, call Info-Santé at 811 or consult a healthcare professional.
Chemotherapeutic agents during the past 6 months Yes/No
Biological response modifiers (e.g. Enbrel, Remicade, Humira) Yes/No
Methotrexate Yes/No
Questions concerning individuals aged 12 years or older
41- Do you have a fever? Yes/No	No matter the answer, go to Question 42.
42- Do you have any of the following symptoms?	If yes to at least one symptom, go to Question 44.
Cough Yes/No	If no to all symptoms and yes to Question 41, go to Question 43.
sore throat Yes/No	If no to all symptoms and no to Question 41: You are at minimal risk of complications. Get lots of rest and drink sufficiently. For additional advice, consult a self-care guide. If you are worried, call Info-Santé at 811 or consult a healthcare professional.
Stuffy or runny nose Yes/No
Headache Yes/No
Muscle ache, backache, pain in the arms or legs Yes/No
43- Have you had a fever for at least 8 days? Yes/No	If yes: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
43- Have you had a fever for at least 8 days? Yes/No	If no, go to Question 45.
44- Have the symptoms been present for at least 8 days? Yes/No	If yes: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
	If no, go to Question 46.
45- Are you managing any of the following conditions?	If yes to at least one of these health conditions: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Kidney dialysis Yes/No	If no to all these health conditions, go to Question 62.
Sickle-cell anemia Yes/No
HIV (AIDS) or immunodeficiency Yes/No
Organ transplant Yes/No
Active cancer Yes/No
46- Are you experiencing serious difficulty breathing (e.g. shortness of breath at rest or difficulty saying single words)? Yes/No	If yes to at least one of these warning signs: You are at considerable risk of complications. Immediately, consult a healthcare professional or go to a hospital emergency ward or call 911.
47- Are you experiencing severe or persistent vomiting? Yes/No	If no to all these warning signs, go to Question 58.
48- Do you feel pain or pressure in the chest or abdomen? Yes/No
49- Do you feel dizziness when you stand up? Yes/No
50- Are your lips bluish or purple? Yes/No
51- Are you spitting up blood? Yes/No
52- Have you gone for at least 12 hours without urinating? Yes/No
53- Do you feel lethargic (e.g. sluggish or indifferent)? Yes/No
54- Are you experiencing convulsions (e.g. sudden and involuntary muscle contractions)? Yes/No
55- Did your symptoms improve and then deteriorate again? Yes/No
56- Are you pregnant? Yes/No
57- Are you 75 years old or older? Yes/No
58- Do you have any of the following respiratory or heart conditions?	If yes to at least one of these comorbidities: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Asthma Yes/No	If no to all comorbidities, go to Question 59.
Emphysema, chronic bronchitis, chronic obstructive pulmonary disease (COPD) Yes/No
Cystic fibrosis Yes/No
Cardiac insufficiency Yes/No
Angina or history of myocardial infarction Yes/No
59- Do you suffer from any of the following diseases?	If yes to at least one of these comorbidities: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Diabetes Yes/No	If no to all comorbidities, go to Question 60.
Kidney insufficiency Yes/No
Cirrhosis of the liver Yes/No
Sickle-cell anemia Yes/No
HIV (AIDS) or immunodeficiency Yes/No
60- Do you suffer from any of the following neurodegenerative diseases	If yes to at least one of these comorbidities: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Multiple sclerosis Yes/No	If no to all comorbidities, go to Question 61.
Spinal cord disease Yes/No
Cognitive dysfunction Yes/No
Alzheimer’s disease Yes/No
61- Are you dealing with any of the following conditions?	If yes to at least one of these comorbidities: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Active cancer Yes/No	If no to all comorbidities, go to Question 62.
Organ transplant Yes/No
Extreme obesity (BMI > 40) Yes/No
62- Are you taking any of the following medications:	If yes to at least one of these regular medications: You are at low to moderate risk of complications. Consult a healthcare professional or call your family doctor.
Corticosteroids (e.g. Prednisone, Decadron) Yes/No	If no to all these regular medications: You are at minimal risk of complications. Get lots of rest and drink sufficiently. For additional advice, consult a self-care guide. If you are worried, call Info-Santé at 811 or consult a healthcare professional.
Chemotherapeutic agents during the past 6 months Yes/No
Biological response modifiers (e.g. Enbrel, Remicade, Humira) Yes/No
Methotrexate Yes/No

Figure 2.

Decision-making algorithm of the inFLUence tool.

It should be emphasized that, in its current form, inFLUence is intended solely as a self-assessment tool to guide patient orientation. It does not provide a clinical diagnosis and is not designed to support or replace medical decision-making at this stage.

4. Discussion

Using a user-centered design approach and rapid prototyping, we developed inFLUence, a self-stratification tool intended to help patients determine whether they should seek medical care for an ILI. Adapted for patients 5 months and older, the tool recommends either to go immediately to the ED, to consult a care provider in a community clinic in the next 48 hours or to self-care at home. inFLUence was designed as a web-based application accessible anytime, anywhere.

In contrast to inFLUence, PMEWS and CURB-65, identified as the two best-performing tools for predicting hospitalization in our previous systematic review, remain essentially clinician-oriented scores. Although they demonstrated good discriminative ability and are available as online calculators, their use requires either clinical evaluation in the case of PMEWS or biological data such as blood urea measurement for CURB-65. These requirements make autonomous use by lay patients unrealistic and limit their applicability outside clinical settings. Self-stratification tools truly designed for the public remain rare, particularly those developed through an iterative user-centered design approach.^15,27 Among the few precedents, the SORT (Strategy for Off-Site Rapid Triage) tool, implemented in the United States during the 2009 H1N1 pandemic, is a notable example.²⁸ Like inFLUence, it aimed to alleviate ED overcrowding by directing low-risk patients toward self-management at home. Its design was inspired by the CURB-65 score²⁹ and its algorithm shared several similarities with inFLUence: it began with ILI symptom screening, assessed severity signs and comorbidities, stratified patients into low, intermediate, and elevated risk categories, and provided recommendations based on risk level. However, it relied on two separate tools for adults and children.

inFLUence therefore distinguishes itself by relying exclusively on self-reported information, covering a broader range of symptoms and comorbidities, incorporating regular medication use into risk assessment, and unifying pediatric and adult triage pathways within a single bilingual interface.

inFLUence also presents several additional advantages. Its simplicity and accessibility make it easy to integrate into daily life, without time or location constraints. Its clean interface, available in both French and English, was designed to accommodate varying levels of digital literacy. Furthermore, its algorithms can be updated by the development team to reflect new scientific evidence or clinical guidelines, ensuring that the tool remains relevant and reliable in a constantly evolving healthcare context. By collecting anonymized data on usage and self-reported symptoms, inFLUence could also contribute to the epidemiological surveillance of ILI in the general population, particularly among individuals who do not seek healthcare services.

The next step will be to conduct large-scale prospective studies to evaluate the tool’s performance, including sensitivity, specificity, predictive values, task completion time, and error rates, as well as user trust in and adherence to its recommendations. The objective will be to confirm that the application effectively fulfills its triage function without compromising patient safety. If results are favorable, integration into existing digital health systems, whether governmental or organizational, could be considered. In this context, developing a mobile version of the application appears essential, since smartphones are now the predominant means of accessing the web for most of the population. Such a strategy would encourage adoption by both healthcare professionals and the public, while enhancing interoperability with other frontline digital tools. Finally, future research should also examine whether the tool’s performance varies depending on the viral agent responsible for the respiratory infection that prompted its use.

Some limitations of this study must be acknowledged. First, the content validation process relied on a relatively small panel of seven experts, all based in Québec City. This may limit the diversity of perspectives and the adaptability of the tool to other socio-healthcare contexts. Second, user testing was conducted with 18 participants, a modest sample size which, although useful for improving the tool’s ergonomics and clarity, may not fully capture all usability challenges encountered by a broader and more heterogeneous population. Nevertheless, according to Virzi’s work, a sample of approximately 15 participants is typically sufficient to identify over 90% of usability issues, which lends robustness to the findings obtained.¹⁹

In addition, the use of self-assessment tools carries inherent risks. Patients with severe symptoms may underestimate the seriousness of their condition or be falsely reassured by the recommendations provided, potentially delaying appropriate care.³⁰ As with any self-reporting tool, this limitation highlights the importance of deploying the application with clear messages encouraging users to seek medical attention in case of doubt or persistent symptoms. Conversely, some users may overestimate the severity of their symptoms due to anxiety or lack of knowledge, leading the tool to systematically direct them toward consultation as a precaution. While this conservative approach helps ensure patient safety, it may reduce the tool’s effectiveness in limiting avoidable healthcare visits.

Ultimately, inFLUence represents a meaningful advancement in the field of digital risk self-assessment for respiratory infections, with anticipated benefits for both patients and the healthcare system.

5. Conclusion

The inFLUence tool was designed through a participatory, user-centered approach, integrating user needs and expectations throughout the development process. This approach promotes optimal adoption of the tool by patients and contributes to its future acceptability. By offering an innovative, accessible solution focused on patient autonomy, inFLUence empowers individuals to assess their own risk of complications related to ILI. To validate its effectiveness and performance under real-world conditions, a large-scale prospective study is the essential next step before its widespread deployment to the public.

Supplemental material

Supplemental Material - inFLUence: Development of a self-administered risk stratification tool for patients with influenza-like illness using a user-centered approach

Supplemental Material for inFLUence: Development of a self-administered risk stratification tool for patients with influenza-like illness using a user-centered approach by Judicael Adadja, Myriam Mallet, Sean Johnston, Maude Ballou, Éric Kavanagh, Maurice Boissinot and Simon Berthelot in Health Informatics Journal.

Footnotes

ORCID iDs

Judicael Adadja

Simon Berthelot

Ethical considerations

This study was approved by the Research Ethics Committee of Québec (reference number: 2020-4562) on 2019-07-24. Respondents gave written consent for review and signature before starting interviews.

Consent to participate

Written informed consent was obtained from all participants prior to their inclusion in the study. Participants were provided with all necessary information, had the opportunity to ask questions, and voluntarily agreed to the collection and use of their personal data for research purposes.

Author contributions

SB and MM designed the study. SB, MM, MBa, SJ, EK and MBo supervised its conduct and contributed to the tool development. JA and SB were involved in the analysis and interpretation of the data. JA drafted the initial version of the manuscript. All authors critically reviewed the manuscript and approved the final submitted version. SB assumes full responsibility for the integrity of the data and the accuracy of the analyses performed.

Funding

This work was supported by the Fonds d’accélération des collaborations en santé of the Ministère de l’Économie et de l’Innovation du Québec (Bergeron-FACS-004). The authors declare having received private funding from Meridian Bioscience to develop point-of-care molecular tests; however, this funding was not used to develop the tool presented in this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The main data from the study are available in the manuscript. The full dataset will be made available upon reasonable request.*

Supplemental material

Supplemental material for this article is available online.

References

Dangi

Jain

. Influenza Virus: A Brief Overview. Proc Natl Acad Sci India Sect B 2012; 82: 111–121. https://doi.org/10.1007/s40011-011-0009-6

Uyeki

Bernstein

Bradley

, et al. Clinical Practice Guidelines by the Infectious Diseases Society of America: 2018 Update on Diagnosis, Treatment, Chemoprophylaxis, and Institutional Outbreak Management of Seasonal Influenza. Clin Infect Dis Off Publ Infect Dis Soc Am 2019; 68: e1–e47. https://doi.org/10.1093/cid/ciy866

Punpanich

Chotpitayasunondh

. A review on the clinical spectrum and natural history of human influenza. Int J Infect Dis 2012; 16: e714–e723. https://doi.org/10.1016/j.ijid.2012.05.1025

Afilalo

Stern

Oughton

. Evaluation and Management of Seasonal Influenza in the Emergency Department. Emerg Med Clin North Am 2012; 30: 271–305. https://doi.org/10.1016/j.emc.2011.10.011

Mertz

Kim

Johnstone

, et al. Populations at risk for severe or complicated influenza illness: systematic review and meta-analysis. BMJ 2013; 347: f5061. https://doi.org/10.1136/bmj.f5061

Sockrider

Reinke

Lareau

. Influenza - “The Flu”: Prevention and Treatment of Influenza Infection (2019–2020 Season Update). American Thoracic Society. https://www.thoracic.org/patients (2019, accessed 12 March 2025).

Giwa

Ogedegbe

Murphy

. Influenza: diagnosis and management in the emergency department. Emerg Med Pract 2018; 20: 1–20.

Putri

WCWS

Muscatello

Stockwell

, et al. Economic burden of seasonal influenza in the United States. Vaccine 2018; 36: 3960–3966. https://doi.org/10.1016/j.vaccine.2018.05.057

Glezen

. Prevention and Treatment of Seasonal Influenza. N Engl J Med 2008; 359: 2579–2585. https://doi.org/10.1056/NEJMcp0807498

10.

Institut canadien d’information sur la santé . Métadonnées du Système national d’information sur les soins ambulatoires (SNISA). https://www.cihi.ca/en/national-ambulatory-care-reporting-system-nacrs-metadata (accessed 12 March 2025).

11.

Influenza (Seasonal and Pandemic). IPAC-Canada. https://ipac-canada.org/resource-centre/infectious-disease-resources/influenza-seasonal-and-pandemic/ (accessed 20 September 2025).

12.

The American College of Obstetricians and Gynecologists . Assessment and Treatment of Pregnant Women With Suspected or Confirmed Influenza. Obstet Gynecol 2018; 132: e169–e173.

13.

Government of Canada . Flu (influenza): Symptoms and treatment. https://www.canada.ca/en/public-health/services/diseases/flu-influenza.html (2025, accessed 12 March 2025).

14.

Chen

K-F

Hsieh

Y-H

Gaydos

, et al. Derivation of a clinical prediction rule to predict hospitalization for influenza in EDs. Am J Emerg Med 2013; 31: 529–534. https://doi.org/10.1016/j.ajem.2012.10.010

15.

Trent Rosenbloom

Daniels

Talbot

, et al. Triaging patients at risk of influenza using a patient portal. J Am Med Inform Assoc JAMIA 2012; 19: 549–554. https://doi.org/10.1136/amiajnl-2011-000382

16.

Burkett

Marwick

Thom

, et al. A comparative analysis of risk stratification tools for emergency department patients with chest pain. Int J Emerg Med 2014; 7: 10. https://doi.org/10.1186/1865-1380-7-10

17.

Ziebart

Kfrerer

Stanley

, et al. A Digital-First Health Care Approach to Managing Pandemics: Scoping Review of Pandemic Self-triage Tools. J Med Internet Res 2023; 25: e40983. https://doi.org/10.2196/40983

18.

Marx

Khelifi

, et al. A systematic review of tools for predicting complications in patients with influenza-like illness. Heliyon 2023; 10: e23227. https://doi.org/10.1016/j.heliyon.2023.e23227

19.

Virzi

. Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? Hum Factors J Hum Factors Ergon Soc 1992; 34: 457–468. https://doi.org/10.1177/001872089203400407

20.

Létourneau

Berthelot

Labrecque

, et al. P081: Adaptation of DECISION+, a training program in shared decision making on the use of antibiotics for acute respiratory infections in primary care, to the context of emergency department: a mixed methods study. Can J Emerg Med 2016; 18: S105–S106.

21.

Harris

Taylor

Thielke

, et al. Research electronic data capture (REDCap), A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–381. https://doi.org/10.1016/j.jbi.2008.08.010

22.

Harris

Taylor

Minor

, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019; 95: 103208. https://doi.org/10.1016/j.jbi.2019.103208

23.

Fitch

Bernstein

Aguilar

, et al. The RAND/UCLA Appropriateness Method User’s Manual. RAND Corporation. https://www.rand.org/pubs/monograph_reports/MR1269.html (2001, accessed 29 April 2025).

24.

Gouvernement du Québec . Médicaments antiviraux contre la grippe (influenza) pour les personnes à risque de complications. https://www.quebec.ca/sante/medicaments/medicaments-antiviraux-contre-la-grippe-influenza-pour-les-personnes-a-risque-de-complications (2024, accessed 6 May 2025).

25.

Gouvernement du Québec – Ministère de la Santé et des Services sociaux . Guide autosoins, grippe A(H1N1) ce que vous devez savoir, ce que vous devez faire pour protéger votre santé et la santé des autres. 1ère ed. Gouvernement du Québec. https://publications.msss.gouv.qc.ca/msss/fichiers/2009/09-235-11F.pdf (2009, accessed 26 March 2025).

26.

Red Hat . OpenShift: Container Platform. https://www.redhat.com/en/technologies/cloud-computing/openshift (2025, accessed 29 April 2025).

27.

Veterans Health Administration . Outil d’auto-évaluation de la grippe des Anciens Combattants. https://www.publichealth.va.gov/docs/flu_selfassess_flowchart.pdf (accessed 12 April 2025).

28.

Kellermann

Isakov

Parker

, et al. Web-Based Self-Triage of Influenza-Like Illness During the 2009 H1N1 Influenza Pandemic. Ann Emerg Med 2010; 56: 288–294.e6. https://doi.org/10.1016/j.annemergmed.2010.04.005

29.

Challen

Bright

Bentley

, et al. Physiological-social score (PMEWS) vs. CURB-65 to triage pandemic influenza: a comparative validation study using community-acquired pneumonia as a proxy. BMC Health Serv Res 2007; 7: 1–8. https://doi.org/10.1186/1472-6963-7-33

30.

Stein

Louie

Flanders

, et al. Performance characteristics of clinical diagnosis, a clinical decision rule, and a rapid influenza test in the detection of influenza infection in a community sample of adults. Ann Emerg Med 2005; 46: 412–419. https://doi.org/10.1016/j.annemergmed.2005.05.020

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.74 MB