Sage Journals: Discover world-class research

Abstract

Background

Generating high-quality data at health facilities is fundamental for effective decision making, which helps to strengthen health system performance and promote better health outcomes. Previous studies have revealed inconsistent findings and design-related drawbacks regarding the quality of routine health data. This study, therefore, aimed to assess the level of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa zone, Southern Ethiopia.

Methods

An institution-based quantitative cross-sectional study was mixed with a phenomenological qualitative study. Data collection was executed from April 1 to 30, 2023. Samples of 304 health workers were randomly selected. A total of six in-depth interviews and three focus group discussions were also done. Multilevel linear regression and thematic analysis were applied.

Results

The perceived level of good data quality was 59.5% (95% CI = 53.8, 65.1). The report timeliness, data accuracy, and the data completeness were 53.3%, 89.4%, and 93.5%, respectively. Factors that were significantly associated with data quality include culture of information utilization (β = 0.23, 95% CI = 0.20, 0.40), perceived skill of data management (β = 0.10, 95% CI = 0.03, 0.18), and electric power access (β = 0.11, 95% CI = 0.01, 0.21). The data quality was decreased in health posts by −0.20 (95% CI = −0.371, −0.030).

Conclusion

The overall quality of routine data among health workers was lower. Improving the skills of data management, cultivating a habit of data use, and enhancing the capacity of health workers are the potential interventions for ensuring data quality in health institutions.

Keywords

Data quality routine health information system public health institutions health workers southern Ethiopia

Introduction

A health information system (HIS) is defined as an integrated effort to collect, process, report, and use health information and knowledge to influence policymaking, program action, and research, which are essential to the effective functioning of health systems.¹ A routine health information system (RHIS) comprises data that are generated at regular intervals from public, private, and community-level health facilities.² A RHIS has two interlinked functions: the production of quality routine health data and the effective utilization of the data for decision making.³ Data quality is a state of data accuracy, completeness, and timeliness that make data appropriate for use in identifying gaps, setting targets, making various decisions, and providing evidence-based feedback.⁴ Producing high-quality data at health facilities is fundamental for effective decision making, which helps to strengthen health system performance, improve service delivery, and promote better health outcomes.^5,6

In both developing and developed nations, many resources have been invested to enhance national RHIS, but the system remains fragmented, poorly coordinated, and burdened with parallel reports, which might impede the country's ability to manage the quality of routine health data.^7,8 In these nations, particularly in developing countries, the use of RHIS data has been questioned due to missing values, biases, data classification issues, and entry errors.⁹

The key problem of RHIS in African countries like Lesotho, Mauritius, Sudan, and Tanzania was the disintegrated management of data due to program-specific demands by different vertical partners.¹⁰ In these countries, the quality of routine health data, measured through metrics like the accuracy, timeliness, and completeness of data, ranged from 34% to 79%.^3,11 The magnitude is less than the World Health Organization (WHO) standard of 90%.^12,13 A survey in Côte d’Ivoire reported that the data verification system and data accuracy at the district level were both 56%, while completeness at the health facility level was 51%.¹⁴ In Malawi and Tanzania, only 40% and 72% of public health facilities and district health offices performed internal data quality checks, respectively.^12,13 In Kenya, data accuracy for fully immunized indicators was 49%.¹⁵

In Ethiopia, only 30% of the Antenatal Care (ANC) data reported was consistent with the source documents in health facilities. The study also revealed that 16% of malaria and 15% of pentavalent three-dose data were overreported at the district level.¹⁶ In the Jimma Zone of Ethiopia, the completeness of data ranged from 34% to 75% and timeliness from 32% to 70% in 8 of the indicators assessed.¹⁷ The completeness of registers and reports was 54% and 56%, respectively, whereas the consistency was only 39% according to the findings in Tigray of Ethiopia. The study also revealed that the lot quality assurance sampling (LQAS) system of routine report was 61%.¹⁸ An assessment in Southern Ethiopia reported that the average data accuracy level of health centers was 76%.¹¹ The findings were lower than the 90% standard of the Ministry of Health in Ethiopia.¹⁹

Ethiopia has undertaken a comprehensive reform of the RHIS through the implementation of the reformed health management information system (HMIS) since 2008, with a primary emphasis on producing high-quality data and ensuring its effective utilization.²⁰ The reform aimed to standardize, integrate, and simplify the data management system.²¹ The information revolution, an initiative implemented since 2016, is one of Ethiopia's five key agendas of the Health Sector Transformation Plan II (HSTP II) (2021–2025).²² The initiative was designed to improve the data management process and influence decision making through the adoption of new digital innovations in RHIS, like district health information system 2 (DHIS2).²³

Currently, the quality of health data in Ethiopia is poor, especially at the health facilities, which bear primary responsibility for the production of routine data. The basic reasons underlying poor quality of data in RHIS were not adequately addressed in Ethiopia. In the previous studies, the level of data quality in RHIS had inconsistent findings. The studies also have design-related drawbacks, such as a lack of accounting for institutional-level variation in data quality. Failing to account for institutional-level variation in data quality can lead to inaccurate assessments and misleading conclusions. There is very limited existing evidence that considers the qualitative aspects, as data quality is a complex construct influenced by personal perspectives and beliefs. Although there are studies conducted at the district, health center, and hospital level, there has been limited evidence regarding the quality of data at the health post level.

For the description of variables, this study applied the Performance of Routine Information System Management (PRISM) framework, which encompasses behavioral, technical, and organizational determinants of data quality.²⁴ The behavioral determinant included was data management skills; technical determinants encompassed data collection and reporting forms and designs; and organizational determinants included the information use culture, performance monitoring meetings, training, supervision, feedback, and infrastructure resources such as computer availability, electricity supply, and internet access. These variables were selected because they provide a holistic view of aspects of data quality that have been inadequately understood in previous studies. Data quality is a complex construct that may not be easily understood through quantitative measurement alone; instead, it requires exploring health workers’ experiences, beliefs, and perspectives.

Therefore, the hypotheses of this study are:

H₀₁:

The level of data quality in RHIS is not lower than the Ethiopian national standard.

H₀₂:

There is no significant association between the factors and data quality practice in RHIS.

H₀₃:

The level of data quality does not significantly vary among different health institutions.

Overall, this study aimed to assess the level of data quality and associated factors in the RHIS among health workers in public health institutions of Southern Ethiopia.

Methods

Study setting

This study was implemented in the public health institutions of Gofa Zone, located 526 km south of Addis Ababa, the capital of Ethiopia. It also has 11 districts (7 urban and 4 rural) and 196 kebele (the smallest administrative units). Currently, the zone has 1510 health workers serving in 179 health posts, 26 health centers, 2 governmental hospitals, and 11 district health offices.

Study design

A mixed methods design was adapted. An institution-based cross-sectional quantitative study was combined with a qualitative study design to integrate the two approaches and develop a comprehensive understanding of the aspects of data quality. One of the qualitative research designs, phenomenology, was applied to assess the lived experiences and knowledge of health workers regarding data quality. Combining the two sources of data enables us to benefit from the detailed, contextualized insights of qualitative interview and the generalizable, externally valid results of quantitative data.²⁵ In this way, the strengths of one type of data mitigate the weaknesses of the other.²⁶ We used a mixed-methods approach because the data quality is a complex phenomenon that encompasses both subjective and objective viewpoints, which need to be addressed in an integrated manner. One of the mixed-methods design typologies, concurrent triangulation design, was adopted to collect both sets of data simultaneously and analyze them separately.²⁷ Then, the two data sources were given equal priority and integrated through triangulation during the presentation of findings, interpretation, and discussion.²⁸ The data collection was executed from April 1 to 30, 2023.

Study participants

All health workers and public health institutions in the study area constitute the source population. Randomly selected districts and respective health institutions were considered as the study population. Similarly, heads of health institutions and departments, HMIS unit leaders, and randomly sampled health workers who are directly involved in the recording and reporting process were considered for inclusion (quantitative). In the qualitative interview, a purposive sample of health workers were included, comprising heads of the institutions, department leaders, health service providers from the outpatient and maternal and child departments, and health extension workers. Newly employed health workers with less than 6 months of experience and privately owned health institutions were excluded.

Sample size determination

In determining the sample size, the study applied the assumptions of confidence level of 95%, power of 90%, nonresponse rate of 10%, and marginal error of 5%. The intracluster correlation coefficient (ICC) of a previous related report was 0.35, with an average cluster size of 4.3 and a design effect of 2.16.²⁹ The percent of unexposed with an outcome was 33%, and the odds ratio was 3.42.³ By applying all the assumptions, a total of 309 health workers were planned to be sampled. Regarding clusters, 6 from 11 districts of the zone were randomly selected, constituting a total of 72 public health institutions (6 districts, 18 health centers, 2 hospitals, and 46 health posts).

In the qualitative interview, both purposive sampling and maximum variation sampling techniques were applied. We selected health workers whom we believe are appropriate and capable of providing accurate information for the study. Additionally, we applied a judgmental approach to select participants, considering a range of variations in their backgrounds and experiences related to data quality. Accordingly, a purposive sample consisting of 3 focus group discussions (FGDs) and 6 in-depth interviews (IDIs) was conducted, involving a total of 18 informants. Data collection was determined to end once we felt that no new information was being obtained across all expected themes.

Data collection procedures and tools

A total of 8 data collectors and 3 supervisors were deployed for data collection after receiving 3-day intensive training on data collection protocol. Standardized questionnaires, adopted from the PRISM series, were utilized.^24,30 The PRISM tool is designed by the WHO and MEASURE Evaluation. This standard tool is the most common, widely accepted, and frequently utilized standard data collection instrument in the existing literature for HIS in general and RHIS in particular. The questionnaire was developed in an English version and translated to the Amharic language for better understanding of respondents.

For the qualitative interview, semistructured, open-ended, and nondirective Amharic version interview guides were designed. Individual (IDIs) and group-based (FGD) face-to-face approaches were organized and facilitated by trained and experienced data collectors. An audio recording was done, and field notes were taken. The research team did not have any prior contact or experience with the participants before the beginning of the data collection process. On average, each FGD took about 85 minutes, while each IDI lasted about 40 minutes. Precautions were taken to ensure that there were no distractions during the conversation, with all IDIs and FGDs conducted in private rooms at the participants’ respective working institutions.

Variable measurement

Since data quality is a multidimensional construct, it is a function of different dimensions commonly including data accuracy, completeness, and timeliness.⁴

Data accuracy rate was determined by comparing the data recorded in the HMIS report with the data recorded in the registers for the third quarter of the Ethiopian reporting period of 2023 (January, February, and March).³¹ A sample of 13 data elements, including ANC first (ANC1), ANC4, protected at birth, prevention of mother-to-child transmission, pentavalent first dose, pentavalent third dose, measles first dose, skilled delivery attendance, postnatal care, contraceptive acceptance rate, the number of tuberculosis cases, and the number of malaria cases diagnosed, were used for assessment. The accuracy of these individual data elements was computed to get the common average value. The computed result was considered acceptable at 5% (95% to 105%), fair at 10% (90% to 110%), and unacceptable at greater than 10% (below 90% or above 110%).³²

Report completeness was measured by the number of institutions reported against the total institutions expected for the 3-month reporting period.⁵ Data completeness of the indicators was computed by considering the percentage of monthly reports of health institutions that have non-zero or missing data for all 13 data elements.³³ Then, the overall mean of the data elements was computed to determine data completeness.

Timeliness of reporting was computed by analyzing health institution summary forms that were remitted to the next level within a predetermined reporting period based on the Ethiopian national reporting schedule.¹⁹

Perception of respondents on data quality: All the stated dimensions of data quality were computed only at the institutional level. In order to identify the level of data quality at the individual level, the perception of health workers on data quality practice was assessed on 11 items. These 11 items, intended to measure data quality practices, include the sufficiency of routine health data, good documentation practices, accurate data compilation, effective data communication, a good data recording and reporting system, appropriateness of data content, quality information and decision-making systems, report timeliness, an assurance system for data completeness, data quality review practices, and encouraging integrity in data management. The responses were designed in a Likert scale rating where “1” indicates “strongly disagree,” “2” “disagree,” “3” “neutral,” “4” “agree,” and “5” “strongly disagree.” The average responses of all individuals were computed and then aggregated to find the grand mean, resulting in a single numerical representation of the overall outcome across all respondents. Respondents were categorized as having a “good” perception of data quality if their score was above the mean and as a “poor” perception if it was below.⁵

Other predictor variables, such as perception of data management skills and attitudes toward information utilization culture, were also measured using a Likert scale structure computed in a similar procedure.

Data analysis

Data were collected by electronic data collection software, with the Kobo Collect application and exported to SPSS Version 26 for analysis after being checked for completeness and consistency. Descriptive statistics like frequencies, proportions, mean, and standard deviation were computed.

Principal component analysis with orthogonal varimax rotation was conducted to assess the underlying structure of the 11 variables related to data quality practices. The assumption of absence of multicollinearity was met, with a determinant score of 0.035 (greater than 0.00001). The measure of sampling adequacy was satisfied with a Kaiser-Meyer-Olkin (KMO) test value of 0.64 (greater than the recommended value of 0.6). Additionally, Bartlett's Test of Sphericity was significant, p < 0.001 (χ² = 999.29, df = 55, n = 304). An eigenvalue criterion of component extraction was utilized, resulting in a four-factor solution with a cumulative variance of 70.73%. No items were removed; therefore, all 11 items were deemed eligible for the final score calculation. Three variables, such as good documentation practices, sufficiency of routine health data, and accurate data compilation, are loaded onto the factor pertaining to the first construct. On the other hand, data quality review, encouraging integrity in data management, and assurance of data completeness are loaded onto the second factor. A good data recording and reporting system, effective data communication, and appropriateness of data content are loaded onto the third factor. Similarly, quality information and decision-making systems, along with report timeliness, are loaded onto the fourth factor.

Multilevel linear regression analysis was adopted. In conventional regression analysis, individual data records are considered independent, or the data are assumed to have no correlation with each other.³⁴ However, this assumption may not be valid in analyses of multi-institutional studies for constructs like data quality, because data from the same clusters or institutions are often correlated. There is existing evidence that data quality in RHIS varies among health institutions.³ Therefore, multilevel analysis is an appropriate and recommended approach for accounting for the clustering effects for this outcome.³⁵

Before employing the main analysis, the ICC of the data was calculated. The results of the no-predictors model indicate that the building of a multilevel model was warranted. This is because (1) the intercepts varied significantly across health institutions (Wald Z = 3.69, p < 0.001) and (2) the ICC suggested that about 36.7% of the total variation in data quality occured among public health institutions. The result is much higher than the expected satisfactory ICC of 5% or more.³⁶

Variables with a p-value of less than 0.20 in the multilevel simple linear regression analysis were entered into the multilevel multivariable linear regression analysis. Standardized beta coefficient values (β), along with 95% confidence intervals (CIs) and p-values, were reported.

The audio-recorded interviews and the field notes were transcribed in Amharic and translated into English. Coding and categorizations of themes and subthemes were done based on the similarity of responses. To ensure a fair interpretation of the data, the coding was done by two independent authors (BKD and DHK), who compared their results and reached an agreement on each theme. Thematic content analysis was employed using ATLAS ti 7.5 software, and themes were developed among the codes after the codebook was prepared. The themes were primarily derived from the data in an inductive manner.

Data quality control

Three-day training was provided to the survey team on the data collection protocol for both qualitative and quantitative approaches. The pretesting of the questionnaire was conducted on 33 health workers sampled from health institutions in the adjacent district of Uba Debretsehay. The pretest data was analyzed to obtain Cronbach's alpha, with a reliability coefficient of 0.70 or higher considered reliable for multivariate variables.³⁷ Standardized quantitative questionnaires were applied.

In the qualitative interview, the data collector conducted one in-depth interview during the pretest process to test the interview protocol, identify issues, and incorporate additional feedback. The audio-recorded files were translated and transcribed into the English language carefully by a trained investigator. The transcripts were then reviewed and validated by the other co-authors. An experienced qualitative expert from Hawassa University also reviewed the transcript and provided feedback on its quality.

Results

Characteristics of the study participants

A total of 72 public health institutions were assessed. Among these institutions, 304 health workers participated in the study, with a response rate of 98.4%. Among the respondents, about 194 (63.8%) were males, and 239 (78.6%) participated from rural health institutions. Of the 304 health workers, 198 (65.1%) were diploma holders, and 177 (58.2%) were selected from health centers (Table 1).

Table 1.

Distribution of the characteristics of the health workers in Gofa Zone, Southern Ethiopia, 2023 (n = 304).

Variable		Frequency	Percent
Sex
	Male	194	63.8
	Female	110	36.2
Location
	Rural	239	78.6
	Urban	65	21.4
Educational level
	Certificate and lower	3	1.0
	Diploma	198	65.1
	Degree	97	31.9
	Masters and above	6	2.0
Profession
	HIT/IT	20	6.6
	Public health officer	56	18.4
	Nurse (all type)	111	36.5
	Pharmacy	22	7.2
	Medical laboratory	17	5.6
	General practitioner	1	.3
	Midwifery	31	10.2
	Health extension worker	46	15.1
Type of health institutions
Health post		46	15.1
Health center		177	58.2
Hospital		20	6.6
District health office		61	20.1
PMT meeting
Yes		223	73.4
No		81	26.6
Nature of reporting
Electronic only		19	6.3
Paper based		220	72.4
Mixed		65	21.4
Supervision on RHIS
Yes		42	13.8
No		262	86.2
Training on RHIS
Yes		74	24.3
No		230	75.7
Computer availability
Yes		158	52.0
No		146	48.0
Electric power supply
Yes		140	46.1
No		164	53.9
HMIS focal person
Yes		247	81.3
No		57	18.8
Internet access
Yes		83	27.3
No		221	72.7

HIT/IT: health information technician/information technician; HMIS: health management information system; HW: health workers; PMT: performance monitoring team; RHIS: routine health information system.

The average age of health workers was 30.07 years with a standard deviation (SD) of 3.77. The median monthly income and work experience of the participants were 6193.00 Ethiopian Birr and 6.69 years, respectively. The mean levels of information utilization culture and data management skill were 2.26 and 2.30, respectively.

In the qualitative interview, a total of 18 informants participated. The median age of participants was 30 years, with a standard deviation of 3.9. The median work experience was 9.5 years. A total of 13 participants (72.2%) were females. Regarding the institutions from which the participants were selected, 4 (22.2%) were from the district health office, 7 (38.9%) health centers, and 7 (38.9%) health posts. A total of 12 (66.7%) of the participants were rural residents, while the remaining 6 (33.3%) were from urban areas.

The level of data quality by dimensions

Completeness and timeliness

The average report completeness of the health institutions is found to be 96.2% and timeliness is 53.3% (Figure 1). On the other hand, the average data completeness of the health institutions is 93.5%, where the lowest is recorded in health posts (92.1%) (Figure 2). In the qualitative interview, the theme entitled “Challenges Related to Data Quality” highlighted problems associated with completeness and timeliness, particularly at the health post level. According to an informant in the IDI, the routine reports were not submitted within the expected timeframe due to a shortage of reporting and recording tools at the health post level, as stated below:

“In terms of completeness and timeliness, delays are particularly notable in health post reports. At the health post level, there is a shortage of standard documentation tools and training for health extension workers on the HMIS program.” (HMIS unit director)

Figure 1.

Report completeness and timeliness by the health institutions in Gofa Zone, Southern Ethiopia, 2023 (n = 24).

Figure 2.

Data accuracy and data completeness by the health institutions in Gofa Zone, Southern Ethiopia, 2023 (n = 72).

Data accuracy

The mean data accuracy, explained by the verification factor of the institutions, was 89.4%. A total of 15 health institutions (20.8%) overreported, while 3 institutions (4.2%) underreported the data (Figure 3). As mentioned in the theme “How to Maintain and Keep Documentation” from the qualitative interviews, there are numerous challenges in maintaining the accuracy of routine health data. For example, an IDI participant described the situation as follows:

“Sometimes, registries or tally sheets have missing or incomplete data, which makes the LQAS process difficult. Additionally, there may be instances of overreporting or underreporting.” (Head of district health office)

Figure 3.

Data verification among health institutions of Gofa Zone, Southern Ethiopia, 2023 (n = 72).

Perception of respondents on data quality

The mean perception of data quality was 2.32 (95% CI = 2.29, 2.35, SE = 0.02). A total of 181 (59.5%) (95% CI = 53.80, 65.10, p < 0.001) of the respondents were categorized as having “good” perception on data quality practice (Figure 4).

Figure 4.

Perception of respondents on data quality in Gofa Zone, Southern Ethiopia, 2023 (n = 304).

Factors and themes associated with data quality practice

In the bivariate multilevel analysis, type of health institutions, receiving feedback, providing feedback, computer availability, availability of electric supply, presence of HMIS focal person, information utilization culture, and perceived skill of data management are significantly associated with the outcome variable, perception of respondents on data quality. However, only information utilization culture, perceived skill of data management, type of institutions, and availability of electric power supply were significantly associated with the outcome in multilevel multivariate analysis (Table 2).

Table 2.

Multilevel multivariate analysis of predictors on the perception of data quality among public health institutions in Gofa Zone, southern Ethiopia, 2023 (n = 304).

Variable	Model 1		Mode 2		Mode1 3
Variable	β (95% CI)	p	β (95% CI)	p	β (95% CI)	p
Constant	1.32 (1.05, 1.59)	0.000	2.32 (2.09, 2.55)	0.000	1.54 (1.21, 1.87)	0.001
Information utilization culture	0.31 (0.21, 0.42)	0.000			0.23 (0.20, 0.40)	0.001
Data management skill	0.12 (0.04, 0.19)	0.003			0.10 (0.03, 0.18)	0.001
Receive feedback	0.05 (−0.01,0.10)	0.121
Type of health institution (DHO)
Health post			−0.11(−0.35, 0.12)	0.338	−0.20 (−0.37, −0.03)	0.022
Health center			−0.24 (−0.38, −0.11)	0.001	−0.17(−0.31, −0.30)	0.019
Hospital			−0.02 (−0.40, −0.08)	0.042	−0.17(−0.31, −0.03)	0.105
Power supply			0.09 (−0.02, 0.19)	0.100	0.11 (0.01, 0.21)	0.032
HMIS focal person			0.12 (0.01, 0.39)	0.042
Computer availability			−0.05 (−0.16, 0.05)	0.302	−0.08 (−0.16, 0.05)	0.155

CI: confidence interval; DHO: district health office; HMIS: health management information system; Model 1: individual level variables; Mode1 2: institutional level variables; Model 3: both individual and institutional level variables.

Type of health institution

As compared with district health office, the data quality decreases in health post by −0.20 (95% CI = −0.371, −0.030, p < 0.05). “Data quality and institutional practices” was a theme that emerged from the interviews, where differences in routine data quality among health institutions and related challenges were discussed. The quality assessment practice of data collected or reported at lower-level health institutions (health posts) is lower than that at higher-level health institutions (district health offices). This finding is supported by the IDI interview, as exemplified below:

“At the health post level, DQA is poor. However, at the health center level, it is mostly practiced, at about 90 to 95%. At the office level, we have a team of routine DQA, which often plans and conducts data assessment at the facility level.” (Head of district health office)

Information utilization culture

Every unit increase in information utilization culture increased data quality by 0.23 (95% CI = 0.195, 0.402, p < 0.001). In the theme “Habit of Data Use and Experience,” the qualitative informants explained their experiences and the rationale behind using routine health data for the decision-making process: For instance, one of the IDI informants explained the situation as follows:

“We use data for different purposes. We collect and analyze data, ensuring its quality before presenting it for various decisions.” (Head of district health office)

Data management skill

Having data management skill improves the data quality by 0.103 (95% CI = 0.026, 0.180, p < 0.001). In the qualitative interview, “Skill-Related Perception in the Data Management Process” was a theme related to the data management skill construct. In this theme, the interview informants emphasized the importance of collective skills in data management for enhancing the quality of routine health data, as represented by the explanation below:

“Currently, all other departments have become responsible for data management as we gradually improve our experience through practice. Therefore, the skills and initiative of all staff are important for achieving success on data quality.” (Head of MCH department)

Availability of electric power supply

When the health institution have an electric supply, data quality increases by 0.11 times (95% CI = 0.010, 0.211). In explaining the theme “Basic Infrastructure Used for HMIS Implementation,” the qualitative participants also highlighted technology infrastructure as a critical factor for streamlining health service delivery and improving the outcome. According to most participants, without access to electricity, it is impossible to install computers in health facilities, which means that automation cannot be achieved and, therefore, ensuring the quality of routine data is very difficult.

“Electric access is the basic infrastructure required for the implementation of HMIS services. If the facility has access to electricity, it has a higher possibility of owning an HMIS computer. The availability of a computer is an important tool for automating routine service data and reporting systems.” (Head of OPD department)

Discussion

This study aimed to assess the level of data quality and associated factors in the RHIS. Data quality in terms of data accuracy, report completeness, data completeness, and timeliness was 89.4%, 96.2%, 93.5%, and 53%, respectively. The perceived level of good data quality was found to be 59.5%.

The report timeliness was found to be 53.3%. An almost similar finding was reported from public health centers of Addis Ababa in Ethiopia (56%).³⁸ But the result is lower than other studies conducted in Côte d’Ivoire (85%),¹⁴ Harari region of Ethiopia (93.7%),³ and the national report timeliness standard of Ethiopia (90%).³⁹ The possible explanation for the difference could be the availability of the remotest health facilities in this study area. Shortage of electric power supply (53.9%) and poor internet access (27.3%) in health institutions of this study could be other possible reasons for delayed reports.⁴⁰

Report completeness was 96.2% and data completeness was 93.5%. A similar finding of data completeness was reported in Harare (93%)³ and Addis Ababa (93%) of Ethiopia.⁴¹ However, the finding of data completeness is higher than the reports observed in Côte d’Ivoire (85%)¹⁴ and Eastern Ethiopia (75.8%).⁴² On the other hand, the report completeness of the study was also higher than that of a survey conducted in Southern Ethiopia (86%).¹⁹ The possible reason might be that, in the current health system, the submission of reports is highly encouraged even after the due dates of the reporting schedule. In addition to this, recently, the monitoring and evaluation system of recording and reporting has been gradually improved.⁴³

The data accuracy of this study was 89.4%, which is consistent with the study conducted in Eastern Ethiopia (88.1%)⁴² and Rwanda (94.7%).⁴⁴ But it is below the verification factor standard interval (0.90–1.10).⁴⁵ That means overreporting is a typical feature of this study. About 20.8% of the health institutions have overreported. Possible explanation for factors associated with over-reporting were that data are not documented or kept, incomplete source documents, manipulation of figures for competition, reporting false achievements, and lack of tools such as tally sheets, registers, guidelines, and standards.^45,46

The perceived level of good data quality among health workers was 59.5%. An almost similar finding was reported in Addis Ababa of Ethiopia (57.9%).⁴⁷ However, the finding is lower than the study from Southern Ethiopia (83%).¹⁹ The possible reason might be, in this study, the majority of the health workers (76%) did not receive training on RHIS. A shortage of program-specific supportive supervision (13.8%) and feedback, as well as limited data management skills of the staff, might be another reason for lower data quality.⁴⁸

Improving the habit of information utilization increased data quality by 0.23. A similar qualitative finding was reported in Northwest Ethiopia, where valuing data is an important facilitator to improve the quality of routine health data.⁴⁹ The more positive experiences a decision maker has in using information to support a decision, the stronger will be the commitment to improving the quality of routine data and data collection systems.⁵⁰ This strengthens the theoretical understanding of how attitudes toward data utilization affect data quality practices.

As compared with the district health office, the data quality was decreased in the health post by −0.20. Similar evidence was reported from the Harare Region of Ethiopia, where health centers were 2.5 times more likely to have good quality data as compared to the health posts.³ A possible reason might be that most staff working in the lower levels of the health systems, involved in the HMIS process, have limited capacity in data quality management.⁵¹ This is due to limited training on RHIS⁵ and a shortage of program specific supportive supervision and assessment system for the RHIS in general and data quality in particular.⁵² Therefore, a multilevel analysis approach advances theoretical knowledge by highlighting the variability of data quality outcomes across different contexts and institutions, which leads to more reliable conclusions of the associated factors.

Perceived skill of data management is a significant predictor of data quality. The finding is consistent with a study carried out in the Oromia regional state of Ethiopia⁵ and in Northwest Ethiopia.³⁹ When the health workers have a required skill in the data management process, their confidence levels and motivation for reporting and recording of routine data improve, so that the performance of maintaining data quality increases.¹¹ This finding supports and extends existing evidence of how human factors, such as skills and confidence, influence data quality performance.

When a health institution has an electric power supply, data quality increases by 0.11 times. Similar evidence was reported from Dire Dawa City of Ethiopia.⁵³ Without the availability of electricity, there would be no chance of using DHIS2 software.²³ In a resource limited settings, expansion of electric power supply in health institutions enables reliable access to the routine data and timely dissemination of reports via offline or online DHIS2 reporting system that motivates the health workforce to maintain data quality.⁵⁴ This empirical evidence builds upon the knowledge of how infrastructural factors play a significant role in improving data quality performance.

Strength and limitation of the study

As a strength, the study employed a mixed methods approach. By applying complementary strengths and triangulation, a mixed-method approach can provide a more comprehensive understanding of the research question and improve the validity and reliability of the study findings. This study also applied a multilevel analysis approach, which allows for the modeling of variation at individual and health institution levels, capturing both within-group and between-group variability of data quality. In contrast, excluding privately owned health institutions limits the generalizability of the study findings only to the public health institutions, and the status of private facilities remains unknown. The primary limitation of cross-sectional studies is that determining the temporal relationship is difficult as the outcome and the exposure were measured at the same time. Small samples of health institutions were used to compute the proportion of data quality dimensions.

Conclusion

The level of data quality in RHIS in the public health institutions is found to be below the target of the national standard. Perceived skills of data management, culture of information utilization, availability of electric power supply, and types of health institutions were significantly associated with data quality. Improving the skill of data management for health workers, maintaining culture of data use in the health institutions, and expansion of the electric power supply in the health institutions are the potential interventions for assuring the quality of data in the RHIS. Enhancing the capacity of health extension workers at the health post level through the provision of training and maintaining data quality assessment practice were important actions for improving data quality at the health post level.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241297227 - Supplemental material for Assessment of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa Zone, Southern Ethiopia: A mixed methods study

Supplemental material, sj-docx-1-dhj-10.1177_20552076241297227 for Assessment of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa Zone, Southern Ethiopia: A mixed methods study by Bedilu Kucho Doka, Abebaw Gebeyehu Worku, Keneni Gutema Negeri and Dejene Hailu Kassa in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241297227 - Supplemental material for Assessment of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa Zone, Southern Ethiopia: A mixed methods study

Supplemental material, sj-docx-2-dhj-10.1177_20552076241297227 for Assessment of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa Zone, Southern Ethiopia: A mixed methods study by Bedilu Kucho Doka, Abebaw Gebeyehu Worku, Keneni Gutema Negeri and Dejene Hailu Kassa in DIGITAL HEALTH

Footnotes

Acknowledgements

We would like to thank Hawassa University, College of Health and Medical Sciences, School of Public Health; former Southern National Nationalities and Peoples Regional Health Bureau and the funding organization, Doris Duke Charitable Foundation for giving us the opportunity to conduct this study.

Availability of data

The data sets utilized in this study are available from the corresponding author and provided on reasonable request of authorized personnel.

Contributorship

BKD conceived the study, designed the work, coordinated the data collection process, conducted statistical analysis, interpreted the data and wrote the manuscript. AGW, KGN, and DHK contributed to the conception and design of the work, analysis and interpretation of the data, and revision of the manuscript. DHK was the main advisor for the manuscript. All the authors have revised and approved the final version of the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The funding source has no role in data collection, analysis, and publication.

Ethical Approval

Ethical approval of the protocol for this study was received from the institutional review board of the College of Medicine and Health Sciences, Hawassa University, with the Reference Number of IRB/183/14 and date 08/06/2022. Official letter permission was received from the former Southern National Nationalities and Peoples Regional Health Bureau. Permission letters were also obtained from the Gofa Zone Health Department, District Health Offices, and each of the respective health facilities.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The fieldwork of this study was supported by the Doris Duke Charitable Foundation granted through the Hawassa University project. The mission of the Doris Duke Charitable Foundation is to improve the quality of people’s lives through grants supporting the performing arts, environmental conservation, medical research and child well-being, and through preservation of the cultural and environmental legacy of Doris Duke’s properties.

Guarantor

BKD

Informed Consent

Informed written consent was obtained from each study participant. Information about the study purpose was provided to each participant and their anonymity and confidentiality of responses and right to withdraw were emphasized. All procedures were conducted based on their voluntary participation.

ORCID iD

Bedilu Kucho Doka

Supplemental material

Supplemental material for this article is available online.

References

Health Metrics Network, World Health Organization. Framework and standards for country health information systems. Geneva, Switzerland: WHO, 2008.

Dagnew

Woreta

Shiferaw

. Routine health information utilization and associated factors among health care professionals working at public health institution in North Gondar, Northwest Ethiopia. BMC Health Serv Res 2018; 18: 685.

Shama

Roba

Abaerei

, et al. Assessment of quality of routine health information system data and associated factors among departments in public health facilities of Harari region, Ethiopia. BMC Med Inform Decis Mak 2021; 21: 287.

Chen

Hailey

Wang

, et al. A review of data quality assessment methods for public health information systems. IJERPH 2014; 11: 5170–5207.

Kebede

Adeba

Chego

. Evaluation of quality and use of health management information system in primary health care units of east Wollega zone, Oromia regional state, Ethiopia. BMC Med Inform Decis Mak 2020; 20: 107.

Gonete

Yazachew

Endehabtu

. Improving data quality and information utilization at Metema Primary Hospital, Amhara national regional state, Northwest Ethiopia 2018: capstone project. Health Inform J 2021; 27: 146045822110431.

Nyella

. Challenges in health information systems integration: Zanzibar experience. In: 2009 International conference on information and communication technologies and development (ICTD). Doha, Qatar: IEEE, 2011, pp.243–251.

Bosch-Capblanch

Besada

Auer

, et al. Making the case for investing in Routine Health Information Systems (RHIS) to achieve the health‐related SDGs. Health Data Collaborative/World Health Organization. (final version). July 2023.

Wagenaar

Sherr

Fernandes

, et al. Using routine health information systems for well-designed health evaluations in low- and middle-income countries. Health Policy Plan 2016; 31: 7.

10.

Lippeveld

Sauerborn

Bodart

, et al. (eds). Design and implementation of health information systems. Geneva: World Health Organization, 2000.

11.

Solomon

Addise

Tassew

, et al. Data quality assessment and associated factors in the health management information system among health centers of Southern Ethiopia. PLoS ONE 2021; 16: e0255949.

12.

O’Hagan

Marx

Finnegan

, et al. National assessment of data quality and associated systems-level factors in Malawi. Glob Health Sci Pract 2017; 5: 367–381.

13.

Mboera

LEG

Rumisha

Mbata

, et al. Data utilisation and factors influencing the performance of the health management information system in Tanzania. BMC Health Serv Res 2021; 21: 498.

14.

Bosso

Ba-Gomis

Gnassou

, et al. Assessment of the performance of routine health information system management in Côte d’Ivoire. MEASURE Evaluation 2018: 2.

15.

Manya

Nielsen

. Reporting practices and data quality in health information systems in developing countries: an exploratory case study in Kenya. J Health Inform Dev Ctries 2016; 13.

16.

Institute EPH. Ethiopia health data quality review: system assessment and data verification for selected indicators. 2016. Report, http://repository.iifphc.org/handle/123456789/132 (accessed 3 December 2020).

17.

Ouedraogo

Kurji

Abebe

, et al. A quality assessment of health management information system (HMIS) data for maternal and child health in Jimma zone, Ethiopia. PLoS ONE 2019; 14: e0213600.

18.

Gebreslassie

Below

Ashebir

, et al. Enhancing health facility-based data quality and use for decision making at primary health care units to improve health service delivery of maternal newborn child and adolescent health, Tigray Ethiopia 2018. Arch Community Med Public Health 2020; 6: 031–035.

19.

Getachew

Erkalo

Garedew

. Data quality and associated factors in the health management information system at health centers in Shashogo district, Hadiya zone, southern Ethiopia, 2021. BMC Med Inform Decis Mak 2022; 22: 154.

20.

Bogale

. Implementation status of health management information system in hospitals of South West Shoa zone, Oromia, central Ethiopia. CEOR 2021; 13: 1–8.

21.

Belay

Azim

Kassahun

. Assessment of health management information system (HMIS) performance in SNNPR, Ethiopia. MEASURE Evaluation 2013: 29.

22.

Cook

. Improving health through information revolution in Ethiopia. Adv Partners Commun 2019. https://www.advancingpartners.org/improving-health-through-information-revolution-ethiopia (accessed 1 December 2020).

23.

Mekebo

Gobena

Hawulte

, et al. Level of implementation of district health information system 2 at public health facilities in eastern Ethiopia. Digit Health 2022; 8: 20552076221131151.

24.

MEASURE Evaluation. Performance of Routine Information System Management (PRISM) Toolkit: PRISM Tools, https://www.measureevaluation.org/prism). 2019.

25.

Fetters

Curry

Creswell

. Achieving integration in mixed methods designs—principles and practices. Health Serv Res 2013; 48: 2134–2156.

26.

Rodriguez

Smith

. Phenomenology as a healthcare research method. Evid Based Nurs 2018; 21: 96–98.

27.

Kroll

Neri

. Designs for mixed methods research. In: Andrew

Halcomb

(eds) Mixed methods research for nursing and the health sciences. USA: Wiley, 2009, pp.31–49.

28.

Creswell

Poth

. Qualitative inquiry & research design: choosing among five approaches. Fourth edition. Los Angeles: Sage, 2018.

29.

Chanyalew

Yitayal

Atnafu

, et al. Routine health information system utilization for evidence-based decision making in Amhara national regional state, northwest Ethiopia: a multi-level analysis. BMC Med Inform Decis Mak 2021; 21: 28.

30.

Hotchkiss

Aqil

Lippeveld

, et al. Evaluation of the performance of routine information system management (PRISM) framework: evidence from Uganda. BMC Health Serv Res 2010; 10: 188.

31.

Moukénet

de Cola

Ward

, et al. Health management information system (HMIS) data quality and associated factors in Massaguet district, Chad. BMC Med Inform Decis Mak 2021; 21: 326.

32.

Njuguna

Vandi

Mugagga

, et al. Institutionalized data quality assessments: a critical pathway to improving the accuracy of integrated disease surveillance data in Sierra Leone. BMC Health Serv Res 2020; 20: 724.

33.

Taye

Gezie

Atnafu

, et al. Data completeness and consistency in individual medical records of institutional births: retrospective cross-sectional study from Northwest Ethiopia. BMC Health Serv Res 2023; 23. Epub ahead of print 2 November 2023. DOI: https://doi.org/10.1186/s12913-023-10127-0.

34.

Yamana

. Introduction to multilevel analysis. Annals of Clinical Epidemiology, 2021

35.

Bickel

. Multilevel analysis for applied research: it’s just regression! New York: Guilford Press, 2007.

36.

Oshchepkov

Shirokanova

. Multilevel modeling for economists: why, when and how. SSRN J. 2020. Epub ahead of print 2020. DOI: https://doi.org/10.2139/ssrn.3637907.

37.

Bonett

Wright

. Cronbach’s alpha reliability: interval estimation, hypothesis testing, and sample size planning: CRONBACH’S ALPHA RELIABILITY. J Organiz Behav 2015; 36: 3–15.

38.

Ayele

Biruk

Habtamu

, et al. Data quality and it’s correlation with routine health information system structure and input at public health centers in Addis Ababa, Ethiopia. Ethiop J Health Dev 2021; 9.

39.

Chekol

Ketemaw

Endale

, et al. Data quality and associated factors of routine health information system among health centers of West Gojjam zone, northwest Ethiopia, 2021. Front Health Serv 2023; 3: 1059611.

40.

Khogali

Ahmed

Ibrahim

, et al. Building powerful health systems: the impacts of electrification on health outcomes in LMICs. Psychol Health Med 2022; 27: 124–137.

41.

Haftu

Taye

Ayele

, et al. A mixed-methods assessment of routine health information system (RHIS) data quality and factors affecting it, Addis Ababa City Administration, Ethiopia, 2020. Ethiopian J Health Dev 2021; 35. https://www.ajol.info/index.php/ejhd/article/view/210746 (accessed 18 January 2024).

42.

Gobena

Shore

Berhanie

, et al. Routine health information system data quality and associated factors in selected public health facilities of Jigjiga Woreda, Somali Regional State’s, Eastern Ethiopia. Ethiop J Health Dev 2022; 36.

43.

Mekuria

Adem

Ayele

, et al. Routine health information system utilization and associated factors among health professionals in public health facilities in Dire Dawa, eastern Ethiopia: a cross-sectional study. Digital Health 2023; 9: 20552076231203914.

44.

Nshimyiryo

Kirk

Sauer

, et al. Health management information system (HMIS) data verification: a case study in four districts in Rwanda. PLoS ONE 2020; 15: e0235823.

45.

Rumisha

Lyimo

Mremi

, et al. Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania. BMC Med Inform Decis Mak 2020; 20: 340.

46.

Endriyas

Alano

Mekonnen

, et al. Understanding performance data: health management information system data accuracy in Southern Nations Nationalities and People’s Region, Ethiopia. BMC Health Serv Res 2019; 19: 175. Epub ahead of print 2019. DOI: https://doi.org/10.1186/s12913-019-3991-7

47.

Ayele

Biruk

Habtamu

, et al. Implementation of human development model impact on data quality and information use in Addis Ababa, Ethiopia. Ethiop J Health Dev 2021; 9.

48.

Tolera

Firdisa

Roba

, et al. Barriers to healthcare data quality and recommendations in public health facilities in Dire Dawa City administration, eastern Ethiopia: a qualitative study. Front Digit Health 2024; 6: 1261031.

49.

Tilahun

Derseh

Atinafu

, et al. Level and contributing factors of health data quality and information use in two districts in northwest Ethiopia: social-ecological perspective. BMC Med Inform Decis Mak 2021; 21: 373.

50.

Anasel, M. G., Swai, I. L., & Masue, O. S. (2019). Creating a culture of data use in Tanzania: Assessing health providers' capacity to analyze and use family planning data. Chapel Hill, NC: MEASURE Evaluation, University of North Carolina.

51.

Ajami

. Training and its impact on Hospital Information System (HIS) success. J Inform Tech Soft Engg 2012; 02: 5. Epub ahead of print 2012. DOI:https://doi.org/10.4172/2165-7866.1000112.

52.

European Centre for Disease Prevention and Control. Data quality monitoring and surveillance system evaluation: a handbook of methods and applications. LU: Publications Office.Stockholm, 2014. Available at: https://data.europa.eu/doi/10.2900/35329 (accessed 20 January 2024).

53.

Tolera

Oljira

Dingeta

, et al. Electronic medical record use and associated factors among healthcare professionals at public health facilities in Dire Dawa, eastern Ethiopia: a mixed-method study. Front Digital Health 2022; 4. https://www.frontiersin.org/journals/digital-health/articles/10.3389/fdgth.2022.935945 (accessed 21 January 2024).

54.

Ross

Stevenson

Lau

, et al. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016; 11: 146.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.16 MB

Assessment of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa Zone,Southern Ethiopia: A mixed methods study

Abstract

Background

Methods

Results

Conclusion

Keywords

Introduction

Methods

Study setting

Study design

Study participants

Sample size determination

Data collection procedures and tools

Variable measurement

Data analysis

Data quality control

Results

Characteristics of the study participants

The level of data quality by dimensions

Completeness and timeliness

Data accuracy

Perception of respondents on data quality

Factors and themes associated with data quality practice

Type of health institution

Information utilization culture

Data management skill

Availability of electric power supply

Discussion

Strength and limitation of the study

Conclusion

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241297227 - Supplemental material for Assessment of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa Zone, Southern Ethiopia: A mixed methods study

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241297227 - Supplemental material for Assessment of data quality and associated factors in the routine health information system among health workers in public health institutions of Gofa Zone, Southern Ethiopia: A mixed methods study

Footnotes

Acknowledgements

Availability of data

Contributorship

Declaration of conflicting interests

Ethical Approval

Funding

Guarantor

Informed Consent

ORCID iD

Supplemental material

References

Supplementary Material