Sage Journals: Discover world-class research

Abstract

Objective

The objective of this study was to deliver an intervention to improve health awareness for infection prevention, hygiene, and sanitation to assess its impact. Furthermore, it aimed to identify the risk of multimorbidity in women of reproductive years from low socio-economic background.

Methods

A randomized control trial was conducted in Pakistan among women aged 15–45 years. Overall, 820 women participated in the baseline survey; 388 women were part of the control group and 360 of the intervention group. A digital health literacy intervention was delivered by 91 trained community health workers. Data were analyzed using descriptive statistics and multivariate logistic regression.

Results

About 35.9% of women suffered from multimorbidity. The intervention group showed higher odds of confidence in managing health with respect to skill and technique acquisition (AOR = 2.21; 95% CI 1.01–4.84), self-monitoring and insight (AOR = 2.97; 95% CI 1.29–6.80) as well as sanitation and hygiene (AOR = 1.42; 95% CI 1.07–1.93). Two primary outcomes related to hand hygiene and protective behavior against infection did not show any significant improvement. The secondary outcomes of the study related to impact on overall health-related quality of life, social integration and support, and emotional well-being also did not show any significant improvement.

Conclusions

Digital health literacy interventions and multimorbidity management for women of reproductive years at the primary level are a way forward to alleviate communicable and non-communicable disease burden in developing countries like Pakistan. These efforts are also critical to improve maternal and child health in developing regions.

Keywords

Health literacy digitalization RCT primary healthcare Pakistan infection control hygiene sanitation

Introduction

Women of reproductive age in South Asia are at risk to face a lack of satisfactory services for primary healthcare.^1,2 The majority of women in this region are from low socio-economic backgrounds and have low health literacy. However, only limited efforts to improve health awareness for improving self-management skills are available.^3,4 Some local health education programs investigated via randomized-controlled trials (RCTs) have targeted services for women of reproductive age at community-level, but have concentrated specifically on awareness about preventing malaria,⁵ early breast cancer detection,⁶ and promotion of maternal health.^7,8 There have been no broad interventions to improve health literacy in women for the management of non-communicable diseases, communicable diseases, and hygiene and sanitation. The importance of latter has considerable importance, because the management of the coronavirus pandemic is highly dependent on women's leadership in the homes for infection prevention and sanitation, for herself, her children, and family members.

The negligence of women's health at primary level and low health literacy also mean that there is limited evidence about the prevalence of multimorbidity in women of reproductive age in Pakistan. However, some research has highlighted that multimorbidity is a grave concern not just for aging women, but also for women in the reproductive phase.^9,10 In fact, some research highlights that multimorbidity occurs a few decades earlier in disadvantaged communities¹¹ and is more prevalent in women.¹² Multimorbidity prevalence is best detected and managed at the primary level of healthcare.¹³ It is also agreed that the best ways to manage multimorbidity and infection control is by educating and improving awareness to prevent deterioration of physical health, support mental health, and improve health policy services from primary to tertiary level.¹² Though local research has highlighted that chronic disease burden amongst poor women of reproductive age at community-level is a significant problem,¹⁴ there has been no research with respect to multimorbidity prevalence in women across Pakistan.

In lieu of all this, the aims of this study were (a) to identify the risk of multimorbidity in women of reproductive age from low socio-economic background in Pakistan and (b) to deliver a digital health literacy intervention to women of reproductive age in primary communities to improve infection prevention and hygiene and sanitation, in order to measure the intervention's impact. The importance of introducing digital literacy interventions cannot be emphasized enough in developing nations like Pakistan, which face challenges of overpopulation and health sector service shortfalls.¹⁵ Digital health service options can also provide improved communication and access to remote and rural populations. However, it is important that training for e-health services is provided to disadvantaged populations unfamiliar with technology.¹⁶ This study also aims to deliver an intervention through community health workers (CHWs) as additional primary healthcare staff, thus lending impetus to strengthening the existing Lady Health Worker (LHW) program, which suffers from understaffing and overburden.^17,18

Methods

This RCT received ethics clearance from Forman Christian College University, Internal Review Board (Reference number: IRB-252/06-2020), and the study was registered at clinicaltrials.gov (Trial registration number NCT04603092). Detailed information about the design of the study can be found elsewhere.¹⁹

Settings and participants

The selection criteria for sampling were: (a) women of reproductive years (15 to 45 years), (b) women who were not receiving primary healthcare services from government providers due to lack of geographical access and the existing LHW program, and (c) women from poor socio-economic backgrounds. Data was collected based on permission and approval from the provincial health ministries through the Rural Support Programmes Network. The locations for data collection are summarized in Figure 1 and allocation of number of CHWs and participants per areas are summarized in Table 1. We were able to collect data from two provinces, Punjab and Khyber Pakhtunkhwa (KPK). Overall, six districts were sampled: (a) Punjab districts: Bhawalnagar, Jhang, Toba Tek Singh and Rahimyar Khan and (b) KPK districts: Mardan and Swabi. More information about the 89 Union Council areas where data collection took place can be found in Supplementary File 1.

Figure 1.

Data collection map.

Table 1.

Study sample and data collectors.

Province	District	Union Council areas	CHWs	Baseline respondents	Control group respondents	Intervention group respondents
Punjab	Rahim Yar Khan	6	6	50	25	23
	Bahawalpur	31	31	288	138	128
	Jhan	17	19	144	66	60
	Toba Tek Singh	17	17	166	79	77
KPK	Swabi	9	9	82	35	31
KPK	Mardan	9	9	90	45	41
Total		89	91	820	388	360

Intervention delivery and community health worker training

A total of 91 CHWs were recruited to deliver the intervention. The recruitment criteria for CHWs were: (a) married women from the community where data collection took place, (b) minimum education qualification of secondary schooling (grade 12 of study), and (c) minimum one-year experience working as health worker in the community. We assigned three CHW Team Leaders for each of the six districts that were sampled and one Field Coordinator for each district. In this way, a total of twenty-four District Team Leaders were responsible for ensuring that each CHW submitted their respective survey data on time and delivered their complete intervention according to the weekly schedule.

The CHWs were trained over a period of 4 weeks through a trickledown process. First the principal investigators (PIs) trained the District Team Leaders and Field Coordinators, and in turn the District Team Leaders and Field Coordinators trained the CHWs. CHWs were trained to understand (a) the objectives of each component of the project, (b) ethics of data collection and research, (c) use of digital tools, (d) communication skills and support of participants, and (e) communication protocols with health consultants and physicians. All CHWs were provided weekly supply of masks and hand sanitizers for themselves and the participants.

Apart from the initial training, written material and guidelines for intervention delivery were provided to the CHWs to help them during the intervention. This material has been attached as supplementary material to the study protocol.¹⁹ The content was translated according to CHW region and differed between one-on-one sessions and group sessions. In addition, the audio and video tutorials were recorded by the PIs so that CHWs could access guidelines and support material on their phone apps. Whatsapp groups and video calls were also used to communicate daily with CHWs. This helped in surveillance of the intervention and also in trouble-shooting daily problems.

The District Team Leaders and Field Coordinators were supervised by the PIs through weekly video meetings and the audit of an entry sheet. The data entry and intervention tracksheet (Appendix A) was maintained online as a shared document between the PIs and the District Team Leaders. This enabled immediate assessment and development of plans to counter problems. Some of the challenges that the PIs and District Team Leaders overcame through regular communication and simultaneous audit of data entry and intervention supervision included: (a) preventing large drop-out rates and improving attendance of participants, (b) ensuring no participant missed an intervention module by requesting CHW for repeat visits, and (c) increasing supply of masks and hand sanitizers for CHWs and participants.

All CHWs were required to submit photographic evidence of their location and field visits, which they transferred to their respective District Team Leaders. The District Team Leaders stored the pictorial proof in the shared online folder. This was considered important as PIs could not travel to remote locations for field audit of the intervention due to time and funding constraints. The CHWs were given a stipend of PKR 1500 (USD 9.07) per month for the project.

Randomized trial

Baseline data and randomization to control and experiment group

Participants completed an initial baseline survey, including questions about socio-demographics, health status, and multimorbidity. They were then invited to participate in a three-month intervention, which would be delivered by CHWs at participant's homes during the one-on-one sessions and at the home of a CHW or a volunteer participant within the community during group sessions. All participants gave written informed consent. Initials or thumb impression was taken from illiterate participants, and in some cases additional verbal consent had to be taken from family or husband for women to participate in the study. Patients were randomly allocated to the intervention and control group. As both the CHWs and participants knew which group they were allocated to, control or experiment, blind randomization was not possible. A pre and post-test survey was administered over a three-month period.

Health literacy booklet for both control and experiment group

Both the control and intervention group received a health literacy booklet which had information modules covering the study domains. It was important for the control group to be provided this health literacy booklet so that we could better understand the impact of the digital literacy intervention on the experiment group. The health literacy booklet was translated in the provincial languages and the participants were informed about the importance of keeping this booklet for the next three months to: (a) use the information for revision regularly and (b) to discuss the content with family members. The booklet material was designed in line with international health communication standards to support communication with illiterate women through the use of colors, symbols, and pictures (Appendix B). The initial steps of sharing the health literacy booklet were helpful in trust and rapport building with participants.

Intervention delivered to the experiment group

The intervention group received the following additional services over a 3-month period. None of these services were provided to the control group, which only received the health literacy booklet as mentioned above:

Health literacy video: this was communicated at least 5–6 times to each participant through the intervention for reinforcement (Appendix C).

Live video consultancy with specialists: each participant gained from at least 2–3 live video consultancies through the intervention.

One-on-one training: each participant gained from 12 weekly one-on-one sessions, which on average lasted 1–1.5 h.

Group training: each participant gained from 3 monthly group sessions. The sessions were held at CHW home or volunteer participant garden area, for an average of 1–2 h.

Self-management chart: one booklet provided to each participant, with training and discussion about chart maintenance held 6 times in one-on-one and group sessions. The self-management chart has been published in the appendix of the study protocol.¹⁹

Though the intervention details are provided elsewhere,¹⁹ it is important to summarize again that the intervention content to improve health literacy included: (a) Coronavirus awareness and prevention; (b) Hygiene and sanitation, (c) Self-management of health and chronic disease(s); and (d) Creating awareness for group and peer support for sustaining health. Table 2 summarizes the specific content for each literacy area and also provides information about which tools were used in order to measure the study outcomes.

Table 2.

Summary of intervention literacy areas, specific content for each literacy area and use of tools for delivery.

		Tools of delivery
Broad literacy area	Specific content	Health literacy booklet	Health literacy video	Live video consultancy with specialists	One-on-one training	Group training	Self-management chart	Outcome being measured (study domain)
		Reminders given to read in all sessions	Shown 5–6 times	2–3 live video consultancies	12 weekly sessions	3 monthly sessions	Discussed 6 times
Coronavirus awareness and prevention	- Basic information - Hand hygiene & mask-wearing - Social distance & public spaces - Symptom identification - Home management of infected	√	√	√	√	√	√	HQOL, STA, HDB, SMI, HSN, SH, HH, PBC
Hygiene and sanitation	- Disinfecting house, kitchen & eating/ drinking utensils - Boiling drinking water - Personal hygiene - Hygiene of children & family	√	√		√	√	√	HQOL, STA, HDB, SMI, SH
Self-management of health and chronic disease(s)	- Nutrition and immunity building - Physical exercise - Chronic ailment management: dosage; equipment management; follow-up visits - Monitoring health & regular consultation for early detection		√	√	√	√	√	HQOL, STA, HDB, SMI, HSN
Group and peer support for health	- Peer reinforcement for health monitoring and management - Developing social support and emotional strength - Importance of maintaining peer support post-intervention				√	√	√	HQOL, STA, HDB, SIS, EW

Note. EW, Emotional wellbeing; HDB, Health-directed behavior; HH, Hand hygiene; HQOL, Health-related quality of life; HSN, Health services navigation; PBC, Protective behavior against coronavirus infection; SH, Sanitation & hygiene; SIS, Social integration and support; SMI, Self-monitoring and insight; STA, Skill and technique acquisition.

Instruments

The “Multimorbidity Assessment Questionnaire for Primary Care” ²⁰ was used to collect baseline data, including information about sociodemographic characteristics, healthcare utilization, chronic diseases, mental health, disease severity, and health-related quality of life.

The pre- and post-test survey included items from three instruments:

The “Health Education Impact Questionnaire”²¹ was used to evaluate seven domains: (a) Health-related quality of life; (b) Health-directed behavior; (c) Skill and technique acquisition; (d) Self-monitoring and insight; (e) Health services navigation; (f) Social integration and support; and (g) Emotional wellbeing.

The “National Sanitation and Hygiene Knowledge, Attitudes, and Practices Survey”²² evaluated two domains: (i) Sanitation and hygiene and (ii) Hand hygiene.

The “Community-Based Assessment of Knowledge, Attitude, Practices and Risk Factors Regarding COVID-19 Survey”²³ was used to measure one domain of: Protective behavior against infection.

Both the baseline and pre and post-test surveys measured the perceptions of respondents and recorded answers on a 5-point Likert scale. Both instruments have been published as appendices elsewhere.¹⁹ Reliability tests were performed on the pre and post-test survey. All scales revealed satisfactory reliability (Table 3).

Table 3.

Reliability of study scales.

	Cronbach's Alpha	Number of items
Health-related quality of life	0.769	5
Health-directed behavior	0.824	4
Skill and technique acquisition	0.676	4
Self-monitoring and insight	0.786	6
Health services navigation	0.695	5
Social integration and support	0.739	5
Emotional wellbeing	0.844	5
Sanitation and hygiene	0.827	9
Hand hygiene	0.848	9
Protective behavior against infection	0.855	10

The primary outcomes for this study were positive and active changes in health-directed behavior, skill and technique acquisition, self-monitoring and insight, health services navigation, sanitation and hygiene, hand hygiene, and protective behavior against infection. The secondary outcomes for this study were improvements in health-related quality of life, social integration and support, and emotional wellbeing.

The specific study hypotheses are:

H1: Multimorbidity risk is higher in women of reproductive years (i) from lower socio-economic background and (ii) who have low health awareness and inadequate health services.

H2: There is a greater positive impact of the digital health literacy intervention on the intervention group compared to the control group with regard to improvements in primary outcomes, which are: (i) Health-directed behavior, (ii) Skill and technique acquisition, (iii) Self-monitoring and insight, (iv) Health services navigation, (v) Sanitation and hygiene, (vi) Hand hygiene, and (vii) Protective behavior against infection.

H3: There is a greater positive impact of the digital health literacy intervention on the intervention group compared to the control group with regard to improvements in secondary outcomes, which are: (i) Health-related quality of life, (ii) Social integration and support, and (iii) Emotional wellbeing.

Data analysis

Both the baseline and pre and post-test data were analyzed using descriptive statistics and bivariate odds ratios (OR). At the first step, univariate logistic regression was used to determine the predictors of multimorbidity among participants. The independent variable was multimorbidity prevalence, defined as more than two chronic diseases in participants. The dependent variables included socio-demographic characteristics, household characteristics, and health-related questions. At the second step, multivariate logistic regression was applied, holding participant's age, education, and household income constant.

With regard to the pre and post-test data, we compounded the ten study domains for the analysis and used “improved confidence in managing health” as the independent variable. Based on the Likert scale, responses of “Strongly agree” and “Agree” or “Always” and “Very Often”, depending on the scale, were considered to be improvements in confidence. Improved confidence was coded “1”, whereas no improvement (including categories of “Neutral” and “Sometimes”, depending on the scale) was coded “0”. Multivariate logistic regression was run in order to identify higher confidence in managing health following the intervention in the intervention versus control group. IBM SPSS Statistics 25.0 was used for all analyses. The level of significance was assigned at p < 0.05. Confidence intervals (CI) for OR have also been reported. There was no missing data, as the responses were filled with assistance of CHWs. An intention-to-treat analysis was undertaken.

Results

The flowchart of participant recruitment and retention is shown in Figure 2. The study started in February 2021 and ended by May 2021. A total of 1010 women were approached for participation in the study, with 820 agreeing to participate. Though all 820 completed the baseline data, only 793 agreed to participate in the intervention. The control and intervention participants at pre-test stood at 405 and 388 respectively, whereas after three months 388 and 360 participants completed the post-test survey, respectively. The dropout of 17 members from the control group was due to inability to contact women at their homes despite three visits, with reasons by family members reporting that the women were visiting relatives. The dropout of 28 participants from the intervention group was due to: (i) unwillingness to continue with the intervention due to time needed for home and child care, (ii) traveling to visit relatives and family, and (iii) not being able to continue due to revocation of permission from family or husband.

Figure 2.

Participant flow for study.

Sociodemographic characteristics at baseline

The majority of the sample is between 16 and 39 years (64.4%), and illiterate (39.6%) or has only received primary schooling up to grade 5 (25.1%). Nearly all participants belong to poor households. Comparable to the population density within Pakistan, the majority of study participants are from Punjab (79.8%) and 20.1% are from KPK province. The majority is unemployed (65.5%) and living in rented houses (72.9%). A total of 35.1% have a mobile phone with internet.

With regard to health characteristics, nearly all sampled women had no health insurance (92.1%) or savings (95.1%). A large majority has taken a loan in the last 5 years (89.0%) and 35.9% suffer from multimorbidity. Nearly half of the sample (41.1%) suffered from a mental health challenges in the previous 12 months and 72.3% perceived problems with physical mobility due to ill-health. Almost 80% suffered from some health challenge in the previous 12 months and almost 70% had undertaken some medical test for health issues. A large proportion has not received any information or only irregular information about health (52.0%) and nutrition and food safety (30%). More than half of participants was not taking the prescribed medicines or multivitamins advised to them by consultants (59.5%). The characteristics at baseline and pre-test are shown in Supplementary File 2.

Predictors of multimorbidity

Multivariate regression analysis of baseline data (Table 4) revealed that women from KPK are at higher risk of multimorbidity (AOR = 1.71; 95% CI: 1.19–2.45). Women who are unemployed (AOR = 1.46; 95% CI: 1.02–2.90) were also at higher risk. Furthermore, women who had taken a loan in the previous five years (AOR = 2.21; 95% CI: 1.40–3.48) or who faced unplanned hospitalization (AOR = 2.39; 95% CI: 1.62–3.52) are also at higher risk of suffering from multimorbidity. Suffered from mental health symptoms in the last 12 months (AOR = 3.18; 95% CI: 2.29–4.42) as well as lack of information and awareness about medical issues (AOR 2.23; 95% CI 1.32–3.79) and nutrition and food safety (AOR 1.84; 95% CI 1.31–2.58) are also predictors for multimorbidity.

Table 4.

Simple bivariate logistic regression for predictors of multimorbidity amongst women of reproductive years (n = 820).

Variable	Higher odds for multimorbidity OR (95% CI)	p-value*	Higher odds for multimorbidity AOR (95% CI)	p-value
Age
50 years and above	1.39 (0.94–2.07)	<0.001
40–49 years	2.49 (1.65–3.75)	<0.001
30–39 years	4.09 (2.33–7.16)	<0.001
16–29 years	1
Highest education
Illiterate	1.29 (0.73–2.28)	0.379
Primary	0.85 (0.46–1.55)	0.598
High school/Secondary	0.68 (0.37–1.24)	0.215
Above	1
Monthly household income
PKR 5000–20,000	1.50 (0.58–3.91)	0.398
PKR 21,000–40,000	1.58 (0.54–4.63)	0.401
PKR 41,000–60,000	1
Marital status
Currently married	1.36 (0.77–2.40)	0.287	0.92 (0.51–1.68)	0.801
Widowed	2.09 (0.81–5.39)	0.125	1.00 (0.36–2.72)	0.997
Divorced/Separated	2.93 (0.79–4.84)	0.107	1.85 (0.47–3.23)	0.374
Never married	1		1
Province
KPK	1.76 (1.24–2.49)	0.001	1.71 (1.19–2.45)	0.004
Punjab	1		1
Occupation
Unemployed	1.33 (0.96–1.84)	<0.001	1.46 (1.02–2.09)	0.038
Employed	1		1
Spouse's literacy
Illiterate	1.67 (0.84–3.33)	0.141	1.57 (0.71–3.46)	0.262
Primary	1.03 (0.50–2.12)	0.931	1.08 (0.50–2.32)	0.839
High school/Secondary	0.91 (0.44–1.87)	0.814	1.01 (0.48–2.12)	0.966
Other	1		1
Spouse's occupation
Unemployed	1.19 (0.88–1.61)	0.253	1.07 (0.77–1.49)	0.655
Employed	1		1
Number of children
6 or more children	2.45 (1.41–4.26)	0.001	1.28 (0.69–2.38)	<0.001
3–5 children	1.23 (0.76–1.99)	0.383	1.00 (0.51–1.42)	0.544
1–2 children	1.60 (0.87–2.97)	0.128	1.44 (0.69–2.38)	0.250
None	1		1
Housing type**
Kaccha	1.60 (0.42–1.84)	0.004	1.16 (0.75–1.75)	0.003
Semi-pucca	1.22 (1.64–5.32)	0.001	1.04 (0.30–1.84)	<0.001
Pucca	1		1
Housing ownership
Rented	1.60 (0.42–0.85)	0.004	1.55 (0.38–2.79)	0.001
Living with someone	3.22 (1.64–5.32)	0.001	2.99 (1.49–4.02)	0.002
Owned	1		1
Number of people living in the house
More than 7	2.08 (0.22–1.89)	0.515	2.04 (0.31–4.66)	0.362
6–7	1.84 (1.98–3.70)	0.591	1.48 (0.33–3.70)	0.371
3–5	1		1
Currently taking care of a disabled/dependent family member(s)
Yes	1.08 (0.66–1.27)	<0.001	1.07 (0.76–1.50)	<0.001
No	1		1
I have a mobile phone with internet
No	1.10 (0.81–1.42)	<0.001	1.07 (0.77–1.48)	<0.001
Yes	1		1
Household amenities***
No	1.52 (1.14–2.04)	0.004	1.28 (0.92–1.77)	0.003
Yes	1		1
Health insurance
No	1.05 (0.62–1.77)	0.851	1.06 (0.62–1.84)	0.813
Yes	1		1
Savings
No	1.84 (0.97–3.49)	<0.001	1.88 (0.97–3.61)	0.005
Yes	1		1
Taken a loan in the last five years
Yes	2.24 (1.44–3.49)	<0.001	2.21 (1.40–3.48)	0.001
No	1		1
Unplanned hospitalization in the last 5 years
Yes	2.21 (1.52–3.22)	<0.001	2.39 (1.62–3.52)	<0.001
No	1		1
Mental health symptoms (in the last 12 months)
Yes	3.29 (2.40–4.53)	<0.001	3.18 (2.29–4.42)	<0.001
No	1		1
Received information or participated in awareness session for health
None/Irregular	1.40 (1.12–1.74)	0.002	2.23 (1.32–3.79)	0.003
Regular	1		1
Use of prescribed medication and multivitamin supplements
None/Irregular	1.89 (1.29–2.79)	0.001	1.13 (0.86–1.89)	0.001
Regular	1		1
Received information or participated in awareness session for nutrition and food safety
None/Irregular	1.90 (1.36–2.64)	<0.001	1.84 (1.31–2.58)	<0.001
Regular	1		1

Note. Multivariate logistic regression analysis was carried out to obtain AOR after controlling for participant age, education, and household income. OR, odds ratio; CI, confidence interval; AOR, adjusted odds ratio.

*p-values are considered significant less than 0.05

**Housing type: Kaccha house-made from mud, thatch, or low quality materials; Semi-pucca house-made from partly low quality and partly high quality material; and Pucca house-made entirely from high quality materials including the floor, roof, and exterior walls.

***Household amenities is a compounded variable to denote: (i) having a washroom in the house, (ii) satisfactory gutter drainage, (iii) water pipes to house, and (iv) gas lines to house.

Pre- and post-test for control and intervention group

The pre and post-test results in terms of descriptive percentages of the respective study domains highlight that both the control and intervention group show improvements following the intervention (Table 5). Chi-square results after cross tabulating with the independent variable of “improved confidence in managing health” showed significant associations for all ten study domains.

Table 5.

Participant improvement in measured domains at post-test (n = 748)

	Control group %, n = 388				Intervention group %, n = 360
	Pre-test	Post-test	Diff	p-value	Pre-test	Post-test	Diff	p-value
Health-related quality of life	17.4	41.5	24.1	0.002	17.1	23.2	6.1	<0.001
Health-directed behavior	40.6	55.5	14.9	<0.001	35.9	43.9	8.0	<0.001
Skill and technique acquisition	23.2	41.8	18.6	<0.001	21.7	46.7	25.0	<0.001
Self-monitoring and insight	11.6	36.0	24.4	<0.001	12.7	16.8	4.1	<0.001
Health services navigation	16.9	38.7	21.8	0.003	15.1	40.0	24.9	<0.001
Social integration and support	14.6	39.9	25.3	0.037	15.1	18.8	3.7	0.006
Emotional wellbeing	39.2	50.6	11.4	<0.001	38.5	52.5	14.0	<0.001
Sanitation and hygiene	2.1	23.4	21.3	<0.001	3.7	13.9	10.2	<0.001
Hand hygiene	07.4	45.9	38.5	<0.001	06.8	22.0	15.2	<0.001
Protective behavior against coronavirus infection	06.3	33.6	27.3	0.078	07.8	26.3	18.5	<0.001

Multivariate regression results of the control versus intervention group are presented in Table 6. The control group showed significantly higher odds of confidence in managing health with respect to three domains: (i) Health-directed behavior (AOR = 1.73; 95% CI: 1.13–2.63); (ii) Health services navigation (AOR = 2.29; 95% CI: 1.34–3.92); and (iii) Sanitation and hygiene (AOR = 4.06; 95% CI: 2.94–7.44). Comparatively, the intervention group showed higher odds of confidence in managing health also in three—although different—domains: (i) Skill and technique acquisition (AOR = 2.21; 95% CI: 1.01–4.84); (ii) Self-monitoring and insight (AOR = 2.97; 95% CI: 1.29–6.80); and (iii) Sanitation and hygiene (AOR = 1.42; 95% CI: 1.07–1.93).

Table 6.

Likelihood of higher confidence in managing health following the intervention in both the control and intervention groups (n = 748)

	Control group AOR (95% CI)*	p-value**	Intervention group AOR (95% CI)	p-value
Health-related quality of life	2.25 (1.33–3.81)	0.300	1.52 (0.65–3.51)	0.326
Health-directed behavior	1.73 (1.13–2.63)	0.010	1.91 (0.94–3.85)	0.071
Skill and technique acquisition	1.00 (0.61–1.63)	0.981	2.21 (1.01–4.84)	0.047
Self-monitoring and insight	1.32 (0.71–2.50)	0.381	2.97 (1.29–6.80)	0.010
Health services navigation	2.29 (1.34–3.92)	0.002	1.18 (0.45–3.09)	0.735
Social integration and support	1.09 (0.61–1.94)	0.769	1.99 (0.83–4.77)	0.121
Emotional wellbeing	1.15 (0.71–1.75)	0.517	1.60 (0.75–3.42)	0.221
Sanitation and hygiene	4.06 (2.94–7.44)	0.049	1.42 (1.07–1.93)	0.015
Hand hygiene	1.14 (0.59–2.59)	0.739	1.19 (0.26–5.46)	0.816
Protective behavior against coronavirus infection	2.37 (0.85–4.57)	0.096	1.43 (0.31–6.44)	0.639

Note. Multivariate regression has been calculated while controlling for variables of age, education, income, region, occupation, and number of children. AOR, adjusted odds ratio; CI, confidence interval.

*Adjusted Odds Ratios (OR) and 95% confidence intervals (CI).

**p-values are considered significant below vales of 0.05.

Discussion

We found that more than one-third of women of reproductive years suffer from multimorbidity. Other studies have highlighted that unless developing countries prioritize multimorbidity detection and management at primary level, health costs and disease burden would increase.²⁴ In addition, multimorbidity in women of reproductive years may be a reason for unfavorable maternal and child health outcomes,²⁵ which is already a major health predicament in the region.²⁶

After adjusting for age, education, and household income, it was confirmed that predictors for multimorbidity include unemployment and unfavorable household characteristics such as not having amenities like water supply, gas, and gutter drainage. In addition, living in rented houses and living in houses made of non-permanent material are also predictors of multimorbidity. Other research confirms that joblessness and low pay²⁷ as well as unfavorable household characteristics²⁸ contribute to multimorbidity. Women who have more than six children and are taking care of one or more physically dependent family members are also at higher risk. Other research suggests that a combination of physical, psychological and social morbidities in women, post childbirth, prevents them from taking care of their health, thus contributing to greater risk of multimorbidity.²⁹

When people do not have access to health information through smartphones, internet, television and radio, it contributes to higher risk of multimorbidity and prevents effective management of self-health.³⁰ Lack of access to health centers and barriers to geographical mobility can also extend health conditions or worsen multimorbidity.³¹ Women from conservative and poorer regions, particularly South Asia, have higher caregiver burdens which prevents them from having the time and energy needed for managing health.³²

Our study indicates that women without savings, who had taken a loan in the last 5 years, and who are not taking their prescribed medicines or multivitamins are also at higher risk of multimorbidity. Previous research has shown that when women have more finances and savings, they are better able to manage health problems and prevent development of multimorbidity or chronic disease.³³ We also found that women who suffer from mental health problems or had experienced unplanned hospitalization in the last five years are at higher risk of multimorbidity. Previous research confirms the relationship between multimorbidity and higher risk of unplanned hospitalization.³⁴ Similarly, there is a complex relationship between multimorbidity and mental health, especially with respect to underprivileged patients belonging to regions where mental health is not prioritized.³⁵

The second and third hypotheses of this study are partially proven correct. Both the control and intervention groups experienced significant improvements after the intervention in the area of sanitation and hygiene. The results imply that the health literacy booklet is a valuable literacy intervention for disadvantaged and semi-literate or illiterate populations. Women from low socio-economic backgrounds are known to benefit from written and printed material to improve health education and awareness as it helps in retention of information.³⁶ In addition, health literacy booklets are useful in sharing information and communicating with household and family members. The intervention group, compared to the control group, experienced significant improvement after the intervention in the domain areas of skill and technique acquisition as well as self-monitoring and insight. Other research reveals that interventions to improve self-management and monitoring of health in women empower them to believe that their health and wellbeing is in their control.³⁷ Such interventions can have long-run benefits of improving early detection and even providing support to family and children for health management and monitoring.

The control group, compared to the intervention group, experienced significant improvements after the intervention in the domain areas of health-directed behavior and health services navigation. There can be numerous reasons for the improvement in the control group, the main being the impact of the health literacy booklet as discussed above. Also, participants in the control group volunteered to participate in the study and thus were motivated to show commitment to retaining information from the booklet and show improvement in health. Given that the CHWs were from the same community as the participants, there may also be a case of social desirability bias, where participants felt compelled to respond positively.³⁸ We were unable to separate the intervention and control group who lived in the same community and the CHWs were delivering services to both intervention and control groups. Thus, contamination may also be an explanation for improved results in the control group.

There is concern that the intervention group did not show any significant improvement in the two domains of hand hygiene and protective behavior against coronavirus infection. Results imply that women are not confident about the management of infection and that they may still be uncertain of infection control practices due to mixed messages from different sources, such as religious and community leaders and government sources.³⁹ Also, this intervention was delivered during the second peak of the coronavirus in Pakistan, and it may be that a lot of people still face confusion and uncertainty about the disease which has impacted results. Women of the region are heavily influenced by social norms and cultural attitudes with respect to management of infection and thus may be less confident about international guidelines for infection control practices.⁴⁰ In lieu of this, we believe there is a critical need for collective mobilization in supporting education for infection prevention and to target whole families, especially the heads of households.⁴¹

Furthermore, the intervention group did not show any significant improvement in health-directed behavior and health services navigation. Research suggests that improvements in women's health in South Asia are highly dependent on multiple external factors, such as type of family, in-laws, and husband,⁴² poverty and financial situation, geographical mobility and transport access,⁴³ and public sector health services and health insurance.⁴⁴ Similarly, health service navigation is dependent on mobility and service outreach by the public sector at community-level. Sampled women in our study belong to conservative regions with distant access to public sector services. Deriving from this, we must recognize that though digital health literacy interventions are critically needed, they must be partnered with an adequate social protection floor in order to help women translate health literacy to satisfactory health behaviors and management.⁴⁵

Secondary outcomes of the study—which have been health-related quality of life, social integration and support, and emotional wellbeing—show no significant improvement. The results confirm that these areas need more attention for intervention delivery and also collective stakeholder support. Overall, only three out of ten domains showed improvements in the intervention group. Results highlight the difficulties in developing successful interventions for women of reproductive years from disadvantaged backgrounds.⁴⁶ However, our digital literacy intervention has the benefit of assistance of CHWs from within the community, which encouraged participation and response of women from conservative and semi-literate backgrounds.

Limitations

Limitations of the study include selection bias. We were unable to sample all provinces of Pakistan due to a lack of permission and safety issues. We were unable to sample more women due to lack of funding and time. Also, our intervention was planned for three months and the post-test was taken immediately after the intervention ended. Future studies may benefit from longer intervention periods which may show the long-term impact. As participants could not be blinded, some results may be due to desirability bias. Furthermore, this study was limited to participant's perceived responses. For future studies, we recommend baseline clinical assessment and the measurement of clinical outcomes post the intervention.

Two main findings communicated by CHWs with regard to participant's experience was that: (i) women did not want to share their medical issues with family members, as it was associated with cultural shame; (ii) they did not want to ask male family members to take them for treatment and checkup due to financial burden and time issues; and (iii) a few participants initially did not want to see the video as they believed the screen will strain their eyes and give them a headache. For future studies, we recommend holding interventions for awareness with husbands and family members to inculcate importance for women's health and seeking referral access from specialists outside the home. We also recommend free health services at tertiary level for disadvantaged women and transport stipends for health visitation to strengthen the link between primary to tertiary level health access.

Some of the main challenges that the CHWs faced included: (i) need for higher stipend for their efforts and work, (ii) internet connectivity issues, (iii) need for additional training in speech confidence and some basic level of counseling support, and (iv) inability to manage or deny requests from other women in the community to attend the group sessions. With regard to the latter, these women were from either the control group or non-participants of the study, thus leading to problems of contamination and also extra work hours for CHWs, as larger group sessions ended up lasting three to four hours. We recommend additional support for training of CHWs and partnership with third party training centers to deliver digital training to health workers for communication and informal counseling. Lastly, Pakistan needs good internet access across the nation for digital health literacy interventions to be effective.

Conclusion

The results of this study reveal the critical need to integrate services for detection, prevention and management of multimorbidity at primary level for women of reproductive years in Pakistan. The study also confirms the value and benefits of digital health literacy interventions. It is recommended that government and private bodies collaborate to fund and strengthen primary care services, specifically for health literacy and multimorbidity management across the country. Collaborative efforts would support comprehensive outreach, financial sustainability, cross monitoring, continued assessment of impact, and policy development over the long-run. Finally, services must be delivered by recruitment and training of supplementary community health workforce at primary level and integrated with the existing LHW services for greater impact.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076221144506 - Supplemental material for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial

Supplemental material, sj-docx-1-dhj-10.1177_20552076221144506 for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial by Sara Rizvi Jafree, Anam Muzammil, Syeda Khadija Burhan, Nadia Bukhari and Florian Fischer in Digital Health

Supplemental Material

sj-docx-2-dhj-10.1177_20552076221144506 - Supplemental material for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial

Supplemental material, sj-docx-2-dhj-10.1177_20552076221144506 for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial by Sara Rizvi Jafree, Anam Muzammil, Syeda Khadija Burhan, Nadia Bukhari and Florian Fischer in Digital Health

Supplemental Material

sj-docx-3-dhj-10.1177_20552076221144506 - Supplemental material for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial

Supplemental material, sj-docx-3-dhj-10.1177_20552076221144506 for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial by Sara Rizvi Jafree, Anam Muzammil, Syeda Khadija Burhan, Nadia Bukhari and Florian Fischer in Digital Health

Supplemental Material

sj-docx-4-dhj-10.1177_20552076221144506 - Supplemental material for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial

Supplemental material, sj-docx-4-dhj-10.1177_20552076221144506 for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial by Sara Rizvi Jafree, Anam Muzammil, Syeda Khadija Burhan, Nadia Bukhari and Florian Fischer in Digital Health

Supplemental Material

sj-docx-5-dhj-10.1177_20552076221144506 - Supplemental material for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial

Supplemental material, sj-docx-5-dhj-10.1177_20552076221144506 for Impact of a digital health literacy intervention and risk predictors for multimorbidity among poor women of reproductive years: Results of a randomized-controlled trial by Sara Rizvi Jafree, Anam Muzammil, Syeda Khadija Burhan, Nadia Bukhari and Florian Fischer in Digital Health

Footnotes

Acknowledgments

The authors gratefully acknowledge the contribution of all participants, their family members, counselors and community health workers who participated in this project. We would also like to thank the research assistants for support in data management including Kundan Raza, Amna Maryam, and Hania Afzal. The following people are acknowledged for assistance in data collection and gaining permissions: Sanam Jhatial, Tanzeela Naz, and Arzu Moosvi. We also appreciate the support of Mr Adeel Ahmad, from the University of the Punjab, Dr Sobia Javed and Dr Sabeen Fatima Haque. Finally, we are indebted to the (i) 18 District Team Leaders for this project: Samina Jafri, Zahida Zubaida, Noureen Irfan, Zareena Anjum, Saima Noureen, Irshad Bibi, Tahira Batool, Shamim Akhtar, Anila Shakeela, Nargis Parveen, Samina Tabasum, Nasreen Akhtar, Rubina Shaheen, Nabila Anjum, Shazia Maqbool, Shakeela Kousar, Sayeeda Rafia Batool, Shazia Kanwal; and the (ii) 6 Field Coordinators for each district: Sajid Iqbal, Khalid Bashir, Sidra Kashif, Sehrish Khan, Shazia Begum, and Arshad Mohammad. We also must thank community members who helped in providing confirmation about translations and in coordinating group sessions during the intervention: Anila Nazish, Kalsoom Anwar, Shahnaz Akthar, Sughra Bibi, Asma Naseer, and Safia Arbai.

Contributorship

SRJ drafted this article and was principal investigator for the project. AM, SKB, and NB supervised the data collection. FF provided supervision for the entire project and helped draft the manuscript. AM provided support for baseline data analysis and SKB provided support for the pre and post-test data analysis. All authors read and approved the final version of the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Ethics approval was obtained from the Research Ethics Board of Forman Christian College University, Lahore, Pakistan (2010–044).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by doctHERS, in partnership with University of Health Sciences (UHS), Punjab Population Innovation Fund (PPIF), Women Chamber of Commerce and Industry (WCCI) and Rural Support Programmes Network (RSPN), under their joint agreement to provide maternal, child, health and family planning services via telemedicine to underprivileged women of Pakistan. The contact point for doctHERS is asher@docthers.com. The funds have been used for the following expenses: digital assets (smartphone, Wi-Fi and video development), stipend for community healthcare workers for field visits and intervention delivery. The principal and co-investigators did not receive a salary for this project. The funding body was not involved in study design, data management, or interpretation of the data.

Guarantor

Trial registration

NCT04603092

ORCID iD

Florian Fischer

Supplemental material

Supplemental material for this article is available online.

Appendix A

Appendix B

Health litearcy booklet provided to control and intervention group

Appendix C

References

Wazir

Shaikh

Ahmed

. National program for family planning and primary health care Pakistan: a SWOT analysis. Reprod Health 2013; 10: 60.

Van Weel

Kassai

Qidwai

, et al. Primary healthcare policy implementation in South Asia. BMJ Glob Health 2016; 1: e000057.

Abdullah

Zakar

. Health literacy in south Asia: clarifying the connections between health literacy and wellbeing in Pakistan. South Asian Studies 2019; 34: 2.

Jafree

. The Sociology of South Asian Women’s Health. Cham: Springer, 2020.

Kumar

Farzeen

Hafeez

, et al. Effectiveness of a health education intervention on the use of long-lasting insecticidal nets for the prevention of malaria in pregnant women of Pakistan: a quasi-experimental study. Malar J 2020; 19: 232.

Shallwani

Ramji

Ali

, et al. Self-examination for breast and testicular cancers: a community-based intervention study. Asian Pac J Cancer Prev 2010; 11: 145–148.

Jokhio

Winter

Cheng

. An intervention involving traditional birth attendants and perinatal and maternal mortality in Pakistan. N Engl J Med 2005; 352: 2091–2099.

Omer

Mhatre

Ansari

, et al. Evidence-based training of frontline health workers for door-to-door health promotion: a pilot randomized controlled cluster trial with Lady Health Workers in Sindh Province, Pakistan. Patient Educ Couns 2008; 72: 178–185.

Feng

Jehan

De Silva

, et al. Prevalence and correlates of cardiometabolic multimorbidity among hypertensive individuals: a cross-sectional study in rural South Asia—Bangladesh, Pakistan and Sri Lanka. BMJ Open 2019; 9: e030584.

10.

Singh

Patel

Biswas

, et al. Multimorbidity in South Asian adults: prevalence, risk factors and mortality. J Public Health 2019; 41: 80–89.

11.

Barnett

Mercer

Norbury

, et al. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet 2012; 380: 37–43.

12.

Violan

Foguet-Boreu

Flores-Mateo

, et al. Prevalence, determinants and patterns of multimorbidity in primary care: a systematic review of observational studies. PloS ONE 2014; 9: e102149.

13.

Contant

Loignon

Bouhali

, et al. A multidisciplinary self-management intervention among patients with multimorbidity and the impact of socioeconomic factors on results. BMC Fam Pract 2019; 20: 53.

14.

Jafree

. Determinants of depression in women with chronic disease: evidence from a sample of poor loan takers from Pakistan. J Community Psychol 2020; 48: 2238–2251.

15.

Sabih

Bile

Buehler

, et al.

Implementing the district health system in the framework of primary health care in Pakistan: can the evolving reforms enhance the pace towards the Millennium Development Goals?

East Mediterr Health J 2010; 16: S132–S144.

16.

Banbury

Roots

Nancarrow

. Rapid review of applications of e-health and remote monitoring for rural residents. Aust J Rural Health 2014; 22: 211–222.

17.

Hafeez

Mohamud

Shiekh

, et al. Lady health workers programme in Pakistan: challenges, achievements and the way forward. J Pak Med Assoc 2011; 61: 210–215.

18.

Bhutta

Hafeez

Rizvi

, et al. Reproductive, maternal, newborn, and child health in Pakistan: challenges and opportunities. Lancet 2013; 381: 2207–2218.

19.

Jafree

Bukhari

Muzamill

, et al. Digital health literacy intervention to support maternal, child and family health in primary healthcare settings of Pakistan during the age of coronavirus: study protocol for a randomised controlled trial. BMJ Open 2021; 11: e045163.

20.

Pati

Hussain

Swain

, et al. Development and validation of a questionnaire to assess multimorbidity in primary care: an Indian experience. BioMed Res Int 2016; 2016: 6582487.

21.

Osborne

Elsworth

Whitfield

. The health education impact Questionnaire (heiQ): an outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Educ Couns 2007; 66: 192–201.

22.

Vivas

Gelaye

Aboset

, et al. Knowledge, attitudes, and practices (KAP) of hygiene among school children in Angolela, Ethiopia. J Prev Med Hyg 2010; 51: 73–79.

23.

Afzal

Khan

Qureshi

, et al. Community-based assessment of knowledge, attitude, practices and risk factors regarding COVID-19 among Pakistanis residents during a recent outbreak: a cross-sectional survey. J Community Health 2021; 46: 476–486.

24.

Anindya

Atun

, et al. Effect of multimorbidity on utilisation and out-of-pocket expenditure in Indonesia: quantile regression analysis. BMC Health Serv Res 2021; 21: 427.

25.

Puri

Kothavale

Singh

, et al. Burden and determinants of multimorbidity among women in reproductive age group: a cross-sectional study based in India. Wellcome Open Res 2020; 5: 275.

26.

Sarwer

Javed

Soto

, et al. Impact of the COVID-19 pandemic on maternal health services in Pakistan. Int J Health Plann Manage 2020; 35: 1306–1310.

27.

O'Callaghan

Yadav

Natarajan

, et al. Prevalence and predictors of multimorbidity among immigrant Asian Indian women residing in Sydney Australia: a cross-sectional study. F1000 Research 2021; 10: 634.

28.

Hossain

Govil

MIK

. Persistence of multimorbidity among women aged 15–49 years in India: an analysis of prevalence, patterns and correlation. Int J Public Health 2021; 66: 601591.

29.

McCauley

Zafar

van den Broek

. Maternal multimorbidity during pregnancy and after childbirth in women in low-and middle-income countries: a systematic literature review. BMC Pregnancy Childbirth 2020; 20: 637.

30.

Barbabella

Melchiorre

Quattrini

, et al. How can eHealth improve care for people with multimorbidity in Europe? Copenhagen: World Health Organization, Regional Office for Europe Copenhagen, 2017.

31.

Pati

Swain

Knottnerus

, et al. Magnitude and determinants of multimorbidity and health care utilization among patients attending public versus private primary care: a cross-sectional study from Odisha, India. Int J Equity Health 2020; 19: 57.

32.

Fikree

Pasha

. Role of gender in health disparity: the South Asian context. Br Med J 2004; 328: 823–826.

33.

Low

Kwan

MSM

, et al. Epidemiologic characteristics of multimorbidity and sociodemographic factors associated with multimorbidity in a rapidly aging Asian country. JAMA Netw Open 2019; 2: e1915245.

34.

Akugizibwe

Calderón-Larrañaga

Roso-Llorach

, et al. Multimorbidity patterns and unplanned hospitalisation in a cohort of older adults. J Clin Med 2020; 9: 4001.

35.

Langan

Mercer

Smith

. Multimorbidity and mental health: can psychiatry rise to the challenge? Br J Psychiatry 2013; 202: 391–393.

36.

Houts

Doak

, et al. The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence. Patient Educ Counsel 2006; 61: 173–190.

37.

Winters

Cudney

Sullivan

, et al. The rural context and women's self-management of chronic health conditions. Chronic Illn 2006; 2: 273–289.

38.

Chung

Monroe

. Exploring social desirability bias. J Bus Ethics 2003; 44: 291–302.

39.

Azlan

Hamzah

Sern

, et al. Public knowledge, attitudes and practices towards COVID-19: a cross-sectional study in Malaysia. PLoS One 2020; 15: e0233668.

40.

Van Lissa

Stroebe

Leander

, et al. Early Indicators of COVID-19 Infection Prevention Behaviors: Machine Learning Identifies Personal and Country-Level Factors. 2020. doi:10.31234/osf.io/whjsb

41.

Al-Hanawi

Angawi

Alshareef

, et al. Knowledge, attitude and practice toward COVID-19 among the public in the Kingdom of Saudi Arabia: a cross-sectional study. Front Public Health 2020; 8: 217.

42.

Fikree

Khan

Kadir

, et al.

What influences contraceptive use among young women in urban squatter settlements of Karachi, Pakistan?

. Int Fam Plan Perspect 2001; 27: 130–136.

43.

Mumtaz

Salway

Bhatti

, et al. Improving maternal health in Pakistan: toward a deeper understanding of the social determinants of poor women’s access to maternal health services. Am J Public Health 2014; 104: S17–S24.

44.

Shaikh

Hatcher

. Health seeking behaviour and health service utilization in Pakistan: challenging the policy makers. J Public Health 2005; 27: 49–54.

45.

Jafree

Anwar

. Social protection for socioeconomically vulnerable women of Pakistan-during COVID-19 and beyond. Pakistan Journal of Women’s Studies 2020; 27: 2.

46.

Abdulai

A-F

Tiffere

A-H

Adam

, et al. COVID-19 information-related digital literacy among online health consumers in a low-income country. Int J Med Inform 2021; 145: 104322.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.02 MB

1.81 MB

0.01 MB