Research output of health research institutions and its use in 42 sub-Saharan African countries: results of a questionnaire-based survey

Abstract

Objective

To describe and analyse research output from surveyed national health research institutions in Africa.

Design

The survey used a structured questionnaire to solicit information from 847 health research institutions in 42 countries of the World Health Organization African Region.

Setting

Eight hundred and forty-seven health research institutions in 42 sub-Saharan African countries.

Participants

Key informants from the health research institutions.

Main outcome measures

Volume, type and medium of publications, and distribution of research outputs.

Results

Books or chapters for books accounted for the highest number of information products published (on average 16.7 per respondent institution), followed by patents registered in country (8.2), discussion or working papers (6.5) and conference proceedings (6.4). Publication in a peer-reviewed journal constituted only a minor part of research output (on average about 1 paper per institution). Radio and TV broadcasts on health research accounted for the highest number of products issued by institution staff (on average 5.5 per institution), followed by peer-reviewed journals indexed internationally (3.8) or nationally (3.1). There were, on average, 1.5 press releases, 1.5 newspaper or magazine articles, and 1.4 policy briefs per institution. Over half of respondent institutions (52%) developed briefs and summaries of articles to share with their target audiences, 43% developed briefs for possible actions and 37% provided articles and reports upon request. Only a small proportion of information products produced were available in institutional databases.

Conclusions

The research output of health research institutions in the Region is significant, but more effort is needed to strengthen research capacity, including human and financial resources.

Introduction

The Health Research System Analysis (HRSA) Initiative institutional survey ¹ constitutes the first attempt to collect large amount of comparable information across a large number of African countries on what health research institutions are doing, how they are organised and what their objectives are. The survey is an important part of a wider effort to describe and analyse national health research systems and intends to ‘provide estimates for benchmarks of national health research systems, as a means to describe, monitor and analyse national health research activities and to improve national research capacities’.

There are few studies addressing research outputs and use in African countries. Generally, these studies measure health research output for African countries using a narrow, though practical, definition of research output, i.e. the number of publications indexed in bibliographic databases.

One of these studies ² found that health research output, as measured by publications indexed in Thomson Reuters (formerly ISI) Web of Science®, is strongly clustered in a few high-income countries. Of the top 20 producers that accounted for more than 90% of research output in 1992–2001, only three (none of which was African) were not from high-income countries.

More recently, it has been shown using the same data that the World Health Organization (WHO) African Region was the only WHO region to have decreased its relative share of global health-related publications during 1992–2001. ³ The decrease in the (initially very low) share was over 15%. However, it is important to note that the absolute number of publications from the Region increased during this period.

A shortcoming of both of the above studies is that the only variable they use to measure health-related research output is the number of publications in high-impact international journals. Other important dimensions of the process of dissemination of research products, such as publishing in regional or national journals, editing working papers, strengthening or increasing human resources, or developing new products are not considered, as no such data were available.

A third study ⁴ considers a different period of time (1996–2005) and uses a different bibliographic database (PubMed®) to measure African research output. This database is not as complete as Thomson Reuters Web of Science® because it only records the affiliation of the first author and it gives roughly the same picture as PubMed®, i.e. African research production is strongly concentrated in a few countries, with Egypt, Nigeria and South Africa accounting for 60% of the continent’s production.

Finally, another study ⁵ of 158 African medical journals from 33 countries documents that most:

are owned by academic institutions

have considerable problems in increasing their relatively small circulation

lack funding

have serious problems in maintaining publication frequency, which keeps them out of international databases such as the Thomson Reuters Web of Knowledge

Considering the information gap about how African health research institutions are organised, what they produce, what type of interaction they have with each other and who funds them, the institutional survey is an important tool to start documenting and analysing some of these questions. In this paper, the main results of Module 4000 (health research outputs, synthesis, dissemination and knowledge management) are presented.

Methods

The methods followed to assess national health information systems are described elsewhere ⁶ but are summarised briefly here.

The survey used Tool 6 from the HRSA Initiative: Methods for Collecting Benchmarks and Systems Analysis Toolkit. ¹ Within the institutional survey, seven questionnaires, representing separate ‘modules’, were completed by the respondent institutions. This report draws on data from two of those questionnaires:

Module 1000 – Identification, introduction and background information

Module 4000 – Research output and use

The questionnaires were designed to focus on issues pertaining to the output and use of research at the institutional level.

For responses to questions where institutions were asked to rank items in the questionnaire, we used weighting schemes to arrive at composite ranks. For example, where the response required ranking an item on a 1–5 scale, a weight of five was given to the first rank, four to the second rank and so on, with the fifth rank getting the least weight of one. The average of these was used to derive a composite rank of items.

We used IBM® SPSS® Statistics Version 19 to analyse the data.

Results

The institution survey dataset included responses from up to 847 institutions in 42 countries in the WHO African Region (all except Algeria, Angola, Sierra Leone and South Africa). About two-thirds of the respondent institutions were aged 30 years, 70.3% belonged to the public sector, 12.5% were independent research institutions and 64.3% functioned at the national level (Table 1).

Table 1.

Characteristics of health research institutions in 42 sub-Saharan African countries, 2009.

Characteristics	Health research institutions
	n *	%
Age of institution (years) (n = 694)
<30	426	61
30–59	200	29
≥60	68	10
Sector the institution belongs to (n = 762)
Public	536	70
Private not-for-profit	132	17
Para-state	37	5
Private for-profit	26	3
Other	31	4
Type of institution (n = 847)
Government agencies	257	30
Hospitals	154	18
Medical schools	108	13
Independent research institutions	106	13
Other research institutions (non-governmental organisations, charities)	105	12
Other universities	95	11
Other	22	3
Level at which institution functions (n = 751)
National	483	64
Local	140	19
Regional	60	8
International	55	7
Other	13	2
Primary functions of institution (n = 697)
Conduct research on health topics	374	54
Academic	373	54
Provide health services	338	48
Conduct research on non-health topics	122	18
Product development or distribution	74	11
Other	128	18
National official or working language (n = 847)
French	445	53
English	285	34
Other	117	14
Institution has mandate on
Research of all types	571	79 (n = 723)
Health research	563	77 (n = 731)

*

Number of respondent health institutions, out of 847 surveyed.

The diversity of institutions taking part in this exercise was seen not only in the different types of institutions and their research level but also in their contributions and priorities. Respondents were asked to list the top three and bottom three contributions since 2000 from a list of alternatives. The most frequently mentioned most important option was ‘influencing health policies or programmes’, with 27% of respondent institutions putting this at the top of their list (Figure 1). The second most frequently mentioned category was ‘producing new knowledge’, cited by 25%. The third most frequently mentioned category was ‘publishing articles in peer-reviewed scientific journals’, cited by 23%.

Figure 1.

Health research institutions’ perceived most important contribution to research outputs, exchange and dissemination activities in 42 sub-Saharan African countries, 2009.

The most frequently mentioned least important option was ‘increasing profits’, cited by 38% of respondent institutions (Figure 2), followed by ‘developing products’ (cited by 11%) and ‘being technological leader’ (cited by 10%). These results clearly demonstrated that neither revenue generation nor being innovative in terms of products was among the main objectives of the institutions targeted by this survey.

Figure 2.

Health research institutions’ perceived least important contribution to research outputs, exchange and dissemination activities in 42 sub-Saharan African countries, 2009.

The production of information products was measured by a number of questions. Two basic indicators of research activity were measured by the set of questions:

In the previous 12 months, how many of the following items that contain or communicate research findings addressing health topics did the institution publish or edit?

In the previous 12 months, how many of the following research outputs addressing health topics were produced by the institute staff?

Books or chapters for books accounted for the highest information products published (on average 16.7 per respondent institution), followed by patents registered in country (8.2), discussion or working papers (6.5) and conference proceedings (6.4). Wider dissemination, non-technical information products were also published, although not as many as more academic, technical papers. There were, on average, 5.9 TV or radio broadcast materials per institution, 2.6 press releases or media briefs and 1.5 policy briefs or report series. Publication in a journal, peer-reviewed or otherwise, constituted only a minor part of the research output, with an average of about one paper per institution (Table 2).

Table 2.

Average number of information products issued per respondent health research institution in the 12 months preceding the survey in 42 sub-Saharan African countries, 2009.

Information product	Respondent institutions (n)	Products	Mean	95% confidence intervals
				Lower	Upper
Products published or edited
Books or chapters for books	276	4622	16.7	−11.8	45.3
Patents registered within country	238	1954	8.2	−7.6	24.0
Discussion or working papers	330	2151	6.5	2.1	10.9
Conference proceedings	340	2177	6.4	1.1	11.7
Radio or television broadcasts on health research	299	1759	5.9	−0.2	12.0
Press release or media briefs	294	756	2.6	1.3	3.8
Peer-reviewed journal editions, indexed internationally	312	542	1.7	1.3	2.2
Policy brief or report series	300	455	1.5	1.1	1.9
Peer-reviewed journal editions, indexed nationally	292	341	1.2	0.7	1.6
Clinical and health practice guidelines	272	306	1.1	0.3	2.0
Systematic reviews	268	236	0.9	0.5	1.3
Non-peer-reviewed magazine editions	267	229	0.9	0.6	1.1
Other peer-reviewed journal editions	271	159	0.6	0.3	0.8
Patents internationally registered	235	30	0.1	0.0	0.2
Products issued by staff
Radio or television broadcasts on health research	269	1472	5.5	2.2	8.8
Peer-reviewed journal editions, indexed internationally	307	1181	3.8	2.7	5.0
Discussion or working papers	313	1181	3.8	2.9	4.6
Peer-reviewed journals editions, indexed nationally	293	903	3.1	2.1	4.0
Press release or media briefs	261	399	1.5	1.0	2.0
Newspaper/magazine articles	265	400	1.5	1.0	2.0
Policy brief or report series	275	397	1.4	0.9	2.0
Other peer-reviewed journal editions	237	243	1.0	0.6	1.4
Books or chapters for books	268	226	0.8	0.6	1.1
Published products available in a database
Discussion or working papers	298	166	0.56	0.5	0.6
Conference proceedings	298	142	0.48	0.4	0.5
Policy brief or report series	299	91	0.30	0.3	0.4
Press release or media briefs	299	89	0.30	0.2	0.4
Peer-reviewed journals editions, indexed internationally	298	88	0.30	0.2	0.3
Radio or television broadcasts on health research	299	70	0.23	0.2	0.3
Peer-reviewed journals editions, indexed nationally	298	70	0.23	0.2	0.3
Books or chapters for books	298	70	0.23	0.2	0.3
Clinical and health practice guidelines	298	68	0.23	0.2	0.3
Non-peer-reviewed magazine editions	298	54	0.18	0.1	0.2
Other peer-reviewed journal editions	298	44	0.15	0.1	0.2
Systematic reviews	298	41	0.14	0.1	0.2
Patents registered within country	299	16	0.05	0.0	0.1
Patents internationally registered	298	15	0.05	0.0	0.1

Radio and TV broadcasts on health research accounted for the highest products issued by institution staff (on average 5.5 per respondent institution), peer-reviewed journals indexed internationally (3.8) or nationally (3.1). There were, on average, 1.5 press releases, 1.5 newspaper or magazine articles and 1.4 policy briefs per institution.

The frequency with which wider publication and dissemination events were organised by the different institutions was also surveyed. These questions aimed to measure the effort made by institutions to communicate research findings during the previous 12 months of answering the questionnaire. Over half of the respondent institutions (52%) developed briefs and summaries of articles to share with their target audiences, 43% developed briefs for possible action and 37% provided articles and reports upon request (Figure 3). Regarding the audience targeted for sharing research findings, the national ministry of health was targeted by 60% of institutions, followed by offices of international agencies (45%) and academic and research institutions (43%) (Figure 4).

Figure 3.

Percentage of health research institutions that have taken various types of action to share research findings with intended target audience in the five years preceding the survey in 42 sub-Saharan African countries, 2009.

Figure 4.

Percentages of health research institutions that have frequently involved target audience or users of research in activities aimed at sharing research findings in the five years preceding the survey in 42 sub-Saharan African countries, 2009.

Another form of communicating and disseminating research findings is through the maintenance and publication of databases containing information about ongoing research projects and research findings. Questions q4102 and q4103 collected such information. These questions asked about the existence of databases, a precondition for further inquiry (e.g. completeness of databases; frequency of update; whether user-friendly or not, as databases can be accessible but impossible to use by the public).

Table 2 shows the number of databases containing either ongoing project information or research findings. These databases may or may not be accessible to the public. The table shows that only a small proportion of information products produced were available in institutional databases. The top three items available in databases were discussion and working papers (on average 0.56 per respondent institution), followed by conference proceedings (0.48), and policy briefs and report series (0.3).

Health research institutions shared their research outputs with the public sector organisations, academic institutions, international agencies, civil society groups, healthcare facilities and private for-profit companies (Table 3). National ministries of health are the largest recipients of research output (466 products or an average of 0.82 products) per institution, followed by academic institutions (466/0.55). Products were mostly disseminated through the medium of scientific seminars at the institutions (1395 products, an average of 3.22 per institution) and at national conferences and seminars targeting specific groups such as practitioners and community members.

Table 3.

Users of research outputs from respondent health research institutions in 42 sub-Saharan African countries, 2009 and mean number of information products per respondent institution.

Users of research output	Respondent institutions (n)	Information products (n)	Mean	95% confidence intervals
				Lower	Upper
Products shared with targeted audience or user(s)
National ministry of health	571	466	0.82	0.78	0.85
Academic or research institutes	569	314	0.55	0.51	0.59
Subnational policy- or decision-makers	569	311	0.55	0.51	0.59
National offices of international agencies	569	299	0.53	0.48	0.57
Non-governmental organisations	569	279	0.49	0.45	0.53
Other national ministries	570	268	0.47	0.43	0.51
Primary or secondary care facilities	569	248	0.44	0.40	0.48
Tertiary care (non-university) hospitals	569	215	0.38	0.34	0.42
Mass media	569	208	0.37	0.33	0.41
National institutes	569	205	0.36	0.32	0.40
Local or community-based departments	568	199	0.35	0.31	0.39
Professional organisations	569	198	0.35	0.31	0.39
Biomedical-based healthcare providers	569	174	0.31	0.27	0.34
National offices of foreign governments	569	172	0.30	0.26	0.34
Patient or consumer groups	569	166	0.29	0.25	0.33
Medical or health research councils	569	157	0.28	0.24	0.31
Traditional or alternative care providers	569	124	0.22	0.18	0.25
Religious leaders or institutions	569	124	0.22	0.18	0.25
National legislative bodies	569	120	0.21	0.18	0.24
Special interest groups	569	111	0.20	0.16	0.23
Pharmaceutical or medical companies	569	104	0.18	0.15	0.21
Military or paramilitary organisations	569	89	0.16	0.13	0.19
Products shared or disseminated through events such as
Scientific seminars at the institution	433	1395	3.22	2.51	3.94
Conferences (only national participation)	391	677	1.73	1.30	2.17
Seminars on findings for practitioners	361	546	1.51	0.88	2.15
Seminars for community members	383	567	1.48	1.13	1.83
Seminars for civil society groups	349	473	1.36	0.64	2.07
Conferences (with foreign participation)	394	499	1.27	1.02	1.51
Forums to develop possible actions	351	415	1.18	0.60	1.77
Forums to identify new research needs	358	379	1.06	0.49	1.63
Seminars on findings for policy-makers	365	361	0.99	0.74	1.24
Seminars on findings for product developers	310	120	0.39	0.21	0.57

Discussion

These results show that while most of the respondent institutions have research as a high priority, publishing that research may not be so important for them. It may be that the data presented in other studies, which consider publications as an indication of research activities, give an incomplete picture and this may be the case in all countries, not only in African countries. It also appears that the institutions surveyed do not generate profits or have activities that could provide revenues, such as developing products or acquiring patents, as main priorities.

Although the survey has provided new, previously unavailable information, the results should be considered carefully as there are some limitations. The aim was to obtain a complete census of all health research institutions across the Region. However, as a complete census was not obtained, it is difficult to determine the representativeness of the results using statistical tools on the data gathered. The data described in this report are not representative, in a statistical sense, of what could be happening in some countries. The data only record research output and use at the institutions surveyed by this questionnaire.

The findings on research outputs do not provide information about productivity of these institutions. It may be that the more active institutions also employ more or better-trained staff. This information provides a baseline, and subsequent surveys may show how the information evolves over time. Based on online searches of publications in high-impact international journals by countries, it is quite possible that the downward share of African publications on health reported in the literature ² has not been reversed.

Patents registered, nationally or internationally, were not frequently reported by institutions. The acquisition of patents and other intellectual property depends on linkage between research institutions and productive firms, and it is to be expected that such linkages are more frequent and stronger in more developed and larger economies.

The findings of the study regarding the sharing of research outputs are useful in obtaining an overview of wider dissemination activities. However, individual institutions should ideally keep track of the number of people who are invited to these events or the range of stakeholders in the audience. Without this information, event numbers should be interpreted with caution in terms of the impact on individuals.

The first key piece of evidence gained from this survey is that research activities are broader in scope and richer in form than is shown by other databases such as bibliographic or reference databases such as PubMed® or Thomson Reuters Web of Science®. Health research institutions not only produce articles but also have a significant level of production of non-academic research output such as short policy reports, media releases and articles. According to the survey, publishing articles is not the top priority of research institutions; instead, priorities are influencing health policies or programmes or producing new knowledge.

The survey shows that a large number of health research institutions make an effort to disseminate and share their research production with their intended audiences. First, there is a sizeable effort to publish or edit academic and non-academic publications, with government institutions being the most active in this field. Second, a large number of academic and non-academic events such as scientific conferences, seminars, workshops and various types of public forums are organised and conducted, again mainly by government institutions. Finally, some institutions maintain a number of databases of ongoing research projects, most of which are open to the public. Government institutions most frequently host such databases.

Conclusions

The survey shows that even though health research institutions in sub-Saharan Africa are not the main worldwide producers of research, and they are not the main producers even among developing countries, their research activity is significant. It should be a policy goal to measure such activity in depth, as research in these countries may be the key to solving, or at least improving, the urgent health problems faced by individual countries in the Region. The institutional survey serves as an initial step in this direction, but further steps should be implemented, building on this extensive mapping of research institutions in the continent.