Abstract
In this article, we have investigated how researchers use the data provided by the Brazilian Information System of Primary Care . We also searched, for the first time, studies that evaluated the quality and reliability of the information provided by the Primary Care Information System. An integrative review of the literature was performed using the keywords ‘information systems, primary care and SIAB’ on search databases, and 53 of 174 articles were selected. These publications were classified into two large subgroups: those using the Primary Care Information System as ‘data source’ and those that took it as the ‘object of study’. The first group included 35 studies, 18 of which used demographic and social health data records, and nine described data about diseases, specifically hypertension and diabetes. These data were used by researchers for association with health indicators (20%) or comparison with other information systems (17%), sample or population calculus (9%), estimation of prevalence and characterization of the epidemiological profile of a population (26%) or, more generally, to carry out the assessment of health status (29%). The Primary Care Information System as the ‘object of study’ group included 18 works, describing the knowledge and practices of professionals in relation to the information system. These researchers pointed out issues in the process of production and information consolidation, mainly due to the lack of training and supervision of community health workers and bureaucratization of their work process. Although some issues in the quality of data provided by the Primary Care Information System were reported by researchers, these findings were not corroborated by two studies that assessed the reliability of information disclosed by this system. Despite changes in the Brazilian health policies, the issue of data quality in health information systems continues to be a challenge preventing data from being used for decision-making and knowledge production.
Background
In Brazil, the Unified Health System (SUS) was created by law in 1990; since then, many information systems have been developed by the Department of Public Health Care System (DATASUS), providing epidemiological data such as Information System on Live Births (SINASC), Information System on Deaths (SIM) and Information System on Compulsory Notifiable Diseases (SINAN), among others. In 1998, information about the health status of populations, collected by community health workers (CHWs), was provided by the Primary Care Information System (SIAB) for the first time. In 2014, this system was replaced by a new platform, the e-SUS of primary care (e-SUS AB). 1 In both the systems, the data are collected by CHWs at the local level and published in the DATASUS database, allowing the analysis and comparison of population indicators at the local, regional and national levels.1,2
In the SIAB, each CHW collects monthly information about 150 families on average in their homes, filling in different data sheets such as a family registration and survey of social and health data (Form A); monitoring of risk groups and priority health problems (Form B); child’s follow-up (Form C); and the activities of registration, procedures and notifications (Form D). The CHWs also enter information into SIAB regarding certain diseases, for example, alcoholism, Chagas disease, disability, diabetes, epilepsy, leprosy, hypertension, malaria and tuberculosis. This information is self-reported, because the CHWs do not request any clinical reports. 3
To date, three integrative reviews of the use of data provided by SIAB have been published. In 2010, Radigonda et al. 4 analysed 18 studies to investigate how this system is used by professionals in primary care. Nogueira et al. 5 examined seven publications to investigate the use of SIAB in scientific research, creating three main categories classifying its use: as a tool in the planning of health actions in primary care, by professionals in primary care and in studies stating the difficulty of completing the SIAB questionnaires. The third integrative review analysed 12 publications to explore the use of SIAB data in scientific research. 6 The main issues of the SIAB were an excessive number of questionnaires, professionals with little training in collecting information and misuse of SIAB data for local planning. None of the above studies analysed the quality and reliability of the information provided by the SIAB.
The Brazilian government has spent a huge amount of resources to produce health indicators to be used for planning and evaluating primary care services. For 16 years, CHWs have gathered health information about the Brazilian population using the SIAB. In this integrative review, we investigated how these data, provided by SIAB, were used by researchers. We also searched, for the first time, for studies that evaluated the quality and reliability of the information provided by SIAB.
Materials and methods
This is an integrative review of the literature.7,8 The search occurred during 1–20 November 2015, considering publications from 1998 to 2014. The keywords ‘primary care, information systems and SIAB’ were used to search and to conduct a review of the Virtual Health Library (BVS-Bireme), with consultation of the following databases: Literature Latin-American and Caribbean Health Sciences (Lilacs), Medical Literature Analysis and Retrieval System Online (MEDLINE), and the portal of the Scientific Electronic Library Online (Scielo).
As inclusion criteria, we searched for studies in the scientific article format with free access available in the databases in English, Spanish and Portuguese. We excluded reviews, theses, dissertations and technical productions, books and monographs from the Ministry of Health, and articles that were not fully available for reading. In total, we identified and read 174 titles and abstracts on the 3 databases searched, of which 2 were conference abstracts, 17 were repeat publications, 30 were theses and/or dissertations, 16 were technical productions and 56 were articles that did not meet the objectives of the study, as well as 3 literature reviews. After application of the inclusion and exclusion criteria, 53 scientific papers remained, which were read in full for analysis performed by two researchers (Figure 1).
Flow chart of the selection of articles in databases.
The publications were first read, and the groupings of categories were created ‘a posteriori’ using the content analysis method. 9 The articles were classified into two large groups: those using the SIAB as a data source (A) and those that took it as the object of study (B) (Figure 1). In group A, the researchers used data from the chips or the reports, directly or indirectly, as primary or secondary data to produce knowledge. In Group B, the profile of professionals, the work process and data collection or the system itself was the focus of the research. After that, in each group, we classified the characteristics of the studies into categories, such as type and approach of studies, scope and geographic region and so on. Data analysis was performed using qualitative and quantitative approaches to measure the frequency of each category and to describe its content. The categories are described in Figure 2.
Description of the categories used for the classification of studies reviewed in this systematic review (SIAB – Information System of Primary Care).
Results
The SIAB was launched in 1998. Over a period of 14 years, from 2000 to 2014, 53 scientific studies used the primary or secondary data provided by SIAB or took the SIAB itself as an object of study (Table 1). On average, approximately 2 articles per year were published; however, in 2010 and 2012, 23 papers were published, which represents 43 per cent of the total.
Studies selected and analysed in a systematic review, according to the database consulted, author(s), year of publication, origin of the populations studied, and use of SIAB.
A: ‘the SIAB as data source’; B: ‘the SIAB as a study object’ and type of approach; QT: quantitative, QL: qualitative; QLT: quali-quantitative.
Regarding the scope of these studies, 2 were national, 13 regional, and 39 local; 73 per cent of the studies had a local interest and used data about specific municipalities. There was a preponderance of studies in the southeast (47%) compared with the northeast and south regions (21% and 23%, respectively); the midwest region produced two publications, and only one study was conducted in the north. Table 1 shows the evaluated studies and their methods.
The SIAB as data source
Among the 35 studies using the SIAB as a source of data, most of them (60%) were published in the period from 2010 to 2014. The majority used a quantitative approach (86%) and cross-sectional design (66%) to allow generalization about the health status of the population of a specific location (80%), mainly in the southeast (40%) and northeast (29%) (Table 2). The SIAB data were used to examine questions at the local level, and almost no studies involved the northern and midwestern populations of Brazil.
Distribution of analytical categories of studies that use the SIAB as data source.
SIAB: Brazilian Primary Care Information System.
Five classification categories have been created to describe what kind of SIAB information was used in such research. Some studies used registration data for the local population (26%), demographic and socio-sanitary data (26%), examined ailments – high blood pressure, diabetes and epilepsy (26%), obtained information about risk groups – children, pregnant women and the elderly (17%) and used attendance reports (questionnaire D) and a health unit production team (6%) (Table 2).
The aims of the studies using SIAB were also investigated, giving rise to the following distribution of categories: use of SIAB data for association with health indicators (20%) or comparison with other information systems (17%), sample or population calculus (9%), estimation of prevalence and characterization of the epidemiological profile of a population (26%) or, more generally, to carry out the assessment of the health situation (29%) (Table 2).
The registration data of the SIAB were used to determine the sample or to account for the population treated at health units.10,11 These data were also used in combination with other parameters to evaluate healthcare models 12 or health indicators.13,14 In the study by Pimentel et al., 15 for example, the SIAB was used to investigate possible associations between the number of people registered, the ratio between the Healthy Family and Oral Health teams, and the quality of oral health indicators such as first dental consultation and supervised brushing. Other researchers simulated the collection activity of CHWs in fieldwork, applying the SIAB registration form to feature their sample 16 or using secondary data already consolidated in the system to describe the health status of a population,17,18 covering the data of registered families, sanitation conditions and the main determinants of health.
The demographic data relating to birth and death, available through the SIAB platform, were used in five studies. In three of them, the authors sought to assess the quality of existing information in SIAB, comparing it in some cases with other systems such as SINASC and SIM.19,20,21 Tillvitz et al., 21 for example, conducted a survey of deaths from Death Statements Grouped by Basic Health Unit (BHU) and compared these records with what was available on Form A or in existing records in BHUs. In addition to assessing the quality of information about this parameter in SIAB, the authors characterized the factors associated with mortality. The other two studies used demographic data to assess and monitor child mortality 22 or the health of children and pregnant women. 23 Moreover, socio-sanitary data, such as age, education, occupation, water supply and sanitation, among others, were used to perform association studies with oral health indicators 24–26 or association with intestinal parasites. 27
Data on different diseases, such as hypertension and diabetes mellitus, were used to estimate the prevalence or characterize the profile of affected individuals11,26,28–34 or for data quality assessment on SIAB grievances compared with other information systems 2,36(Table 2). Li et al., 33 for example, used the SIAB information to identify patients with epilepsy to trace their epidemiological and clinical profile.
Regarding the risk groups, three studies evaluated the health status of children, pregnant women and the elderly.37,38,35 The other two estimated the prevalence of elderly populations,39,40 and one investigated malaria infection in pregnant women in the Amazon. 41 Details referring to attendance (Questionnaire D) collected by the Family Health teams were used to assess coverage and describe the morbidity and health indicators.42,43
Despite their apparent diversity, the studies rarely stated their working hypotheses or proposed questions to be answered using data from the information system. For example, to describe the health indicators of a municipality in Mato Grosso, the researchers 43 concluded that 28.4 per cent of the visits were for hypertension, 27 per cent for childcare, 16.9 per cent for prenatal care, 13.6 per cent for prevention of cervical cancer, 7.9 per cent for diabetes and so on. Therefore, most studies were descriptive and exploratory.
The SIAB as a study object
When the SIAB was the object of study in scientific research, interviews were conducted with primary care professionals to investigate their views, their knowledge and their everyday work, or to explore the use of the information system itself (78%). The approach of these studies was qualitative and exploratory (67% and 61%, respectively), their scope was local (56%) and concentrated in the southeast (61%), and most studies (67%) were published in the period from 2010 to 2014 (Table 3). All studies have proposed conducting evaluations of the quality of information produced by the system, the use of SIAB by the workers and training needs or specific aspects of the system, such as evaluation of mental healthcare (Table 3).
Distribution of analytical categories of studies that use the SIAB as object of study.
The registration data and reports were used to evaluate the quality and reliability of the information generated in the system. In the study by Tibiriça et al., 45 for example, the coverage and the quality of information of three health units involving a sample of households (64) was analysed using the comparison of questionnaire data filled in by CHW and independently by researchers. The interobserver agreement was greater than 95 per cent for registration information, which validated the use of this information for sample calculation; however, the number of cases of diseases was very small (n = 5), which impeded generalization.
Ribeiro et al. 46 studied the agreement of the BHU records compared with the consolidated data provided by DATASUS. There was no correspondence between the family master record number made by CHWs and the chart of the same family kept at the facility. The authors showed that the completion of the form by CHWs demonstrated failures, suggesting that this problem was due to the turnover of these professionals. The fact that the units do not standardize their records makes it difficult to check the quality of this information. Thus, the work of survey information by CHWs had become bureaucratic procedure, without dialogue regarding what was done in the health unit. Sala et al. 47 showed that there was no correlation between prevalence of diseases and these health conditions, revealing the difficulty of obtaining accurate and reliable data. Professionals complained of excess questionnaires used in routine work, resulting in lost time and hindering the revitalization of the work.
Silva and Laprega 48 presented the difficulties and criticism reported by health professionals for each of the questionnaires used in the system. For example, CHWs did not know how to properly fill in the B-GES form (pregnant), even though it was necessary to determine the nutritional status of pregnant women, and this difficulty also occurred with the B-HA questionnaires (hypertension) and B-DI (diabetes). Other questionnaires referring to low-prevalence diseases (B-TB and B-HAN) were barely used in primary care. It was observed that the units create questionnaires and observation notebooks to record aspects or guidelines about families that were not in the questionnaire system, such as type of breastfeeding, occurrence of diarrhoea, acute respiratory infection and use of oral rehydration therapy, among others. In the group of doctors and nurses interviewed, the reports criticized the content of the SIAB technical manual for its inadequacy of definitions of collection procedures, failures in the recordkeeping in the care of data-storage chips and poor reliability in the case of certain fields conducted by community agents.
Barbosa and Forster 49 showed some flaws in the SIAB software that hindered with the possibility of establishing generalizations. For example, ages were not automatically updated, data were erased when the person turned 15 years old and information overlapped. Daily records were not consolidated, such as those involving breastfeeding, weight, immunization and so on. Some authors reported the possibility of entering prevalent diseases into specific locations in the system.49–51
Freitas and Pinto 51 noted that family health strategy (FHS) professionals perceived limitations in the power and interpretation of SIAB data because their involvement with the system was sporadic due to everyday demands. The information flow in the SIAB implied a centralization of decision-making and decentralization of collection. Professionals’ concerns were to achieve monthly output of the data and to feed the system instead of discussing its meaning and using it for planning and decision-making on the health status of the population they served. Therefore, the data were an end in itself.52,53
Another aspect that limited the use of SIAB was the lack of systematic oversight of the completion of the questionnaires by CHWs. Illegible records, frequent loss of information, inability to perform searches and research, and concepts and procedures with little definition are examples of errors that could be corrected with supervision, allowing generalizations to be made from the data.51,54 Although the CHWs were aware of the tools and did not report difficulty in completing questionnaires, they did not understand why they performed this data collection monthly. 55 The lack of information and training, including among managers, 56 generated the production of data that were rarely used for diagnosis, planning and reorganization of actions in the health unit. 54 Therefore, the SIAB has not achieved its objectives.57,58
Some studies have suggested the potential for system expansion to incorporate new indicators involving mental health, epilepsy and infectious diseases.35,59 For example, in questionnaire A of the SIAB, there is a field for cases of individuals with epilepsy; however, it would be interesting to monitor and better evaluate these cases, since most cases of epilepsy are due to neurocysticercosis, the control information of which involves environmental monitoring and intestinal parasites. In the case of mental health issues, it would be important to create new indicators to record better the consumption of alcohol and drugs, the use of psychotropic drugs, and mental disorders. It should be considered that the records are made monthly by the CHWs and that the more indicators there are, the greater the time required for collection. The inclusion of new indicators would involve reviewing the distribution and frequency of this collection, which would not necessarily be daily or monthly.
Discussion
The Brazilian government has invested a significant amount of resources in the production and publication of data referring to primary care, supporting workers and maintenance of information systems. Although Tibiriça et al. 45 have shown that registration of the SIAB data are reliable for sample size calculation, no other studies were identified that have systematically evaluated the reliability of the information produced in this system. To qualify the data of the health information systems, it would be necessary to create mechanisms and strategies for checking the data collected by the CHWs based on research focusing on the quality and reliability of the information. In fact, our data showed a lacuna at the literature.
Most studies that took the SIAB as the object of study showed that CHWs, responsible for primary data collection, have not properly completed the forms because they have no knowledge, training or engagement for performing this task. Their work process was motivated by production targets, since the municipality can be penalized with cancellation of their funding resources if targets are not achieved. Thus, the completion of the sheets became a bureaucratic and alienating routine. The data, although collected monthly to feed the system, were not utilized for planning and problem-solving in health facilities. Such reports thus justified the distrust of the academic community regarding the quality of information produced by the SIAB for use in scientific research.
In 78 per cent of searches, data on registration, demographics, socio-sanitary conditions and diseases, particularly hypertension and diabetes, present in questionnaire A were used. In the reports of occupational health facilities, there has been criticism of the fact that they collect information about diseases that are not prevalent in their area to the detriment of others that were of interest. Furthermore, CHWs also reported that the information they collected on hypertension was not used by physicians or nurses at the clinics. Another issue was the absence of families’ records at the health units. The comparison of data provided by the SIAB and the medical records at the health units could not be performed, damaging systematic studies on the data reliability. Another factor that contributed to the community’s distrust of the SIAB concerns the lack of supervision of the process of producing and consolidating the data.
Regarding information about different diseases, the studies used only data about hypertension, diabetes mellitus and epilepsy. No research has analysed the information on PCD (primary ciliary dyskinesia), tuberculosis, leprosy, or Chagas disease, among others. This indicates that most of the data, collected monthly by primary care professionals for over 15 years, was not in fact taken advantage of to produce knowledge about these diseases. In 55 per cent of the studies in which these data were used, they served to characterize the sampled population, estimate prevalence or assess the health status of the population.
Although the authors recognized the importance of SIAB as a local management tool, that is that the continuous analysis of municipal profiles could help define and assess actions, this was not studied in their research. Roese et al. 37 reasoned that the information systems were one of the challenges of the SUS and suggested rethinking their organization to improve their integration. Some researchers noted a discrepancy between the records of the SIAB and the records of the CHWs (notepad) and the lack of access to data from CHWs consolidated by the SIAB; 11,34,35 others showed the incompatibility between SIM data and the SIAB, which complicates the monitoring of deaths in primary care because of the low population registration index of the studied population (33.6%). 21
Currently, the Ministry of Health will endeavour to implement the new strategy of replacing the SIAB with the e-SUS primary care and Health Information System for Primary Care (SISAB), responding, to some extent, to the questions that have arisen in scientific research. The literature has shown an increasing education and professionalization process for CHWs, responsible for part of this new data collection system,35,61–70 which could help to improve the quality of the information production process and thus the reliability and trustworthiness of the data.
The use of SIAB in scientific research was related to the distribution of postgraduate programmes on public health in the country, which are concentrated in the most populous and economically developed regions, leading to a clear asymmetry of knowledge production. The SIAB was used as the means and not as the end of the investigations.
The limitations of this study are related to the time cut used to review the literature, which just evaluated articles published between 2010 and 2014, and to changes in the Brazilian National Policy of Primary Care. Despite these changes, the issue of data quality in health information systems continues to be a challenge so that data can be used for decision-making and knowledge production.
Conclusion
In this review, we showed a gap in the literature and lack of evidence to evaluate the reliability of SIAB data and its underuse in scientific research. These data have been used for sample or population calculus, estimation of prevalence and characterization of the epidemiological profile of a population or, more generally, to carry out the assessment of health status, comparison or association with other information systems. The critical issues of the SIAB data are related to the process of production and information consolidation, mainly due to the lack of training and supervision of CHWs and bureaucratization of their work process. Despite changes in the Brazilian health policies, the issue of data quality in health information systems continues to be a challenge so that data can be used for decision-making and knowledge production.
Footnotes
Acknowledgements
The authors thank all the members of the research group who helped to improve the manuscript.
Authors’ contributions
The authors worked together to develop the study, read and approved the final manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was funded by Universidade Estadual da Paraíba (PROPESQ) and Fundação de Apoio à Pesquisa do Estado da Paraíba (FAPESQ/CNPq – PPSUS 015/2014).
