Evaluation of an Auditable Pharmaceutical Transaction Services Program in Ethiopia: A Case Study Evaluation Design

Abstract

In Ethiopia the government created an auditable pharmaceutical transaction system (APTS) to prevent low availability and high expiration rate of drugs. However, nothing is known about the implementation status of APTS in the study area. Therefore, this study aimed to evaluate the implementation status of APTS program by dimensions (availability, compliance, and acceptability) at Debark primary Hospital, North West Ethiopia. A single case study design was used to evaluate the process of the APTS program. Availability, compliance, and acceptability (client satisfaction) dimensions were measured using four, seven, and seven indicators respectively. In this evaluation, 423 clients participated in exit interviews, 600 records were reviewed, and five key informants were interviewed. To identify the predictor variables with acceptability, we used a multivariable logistic regression analysis. Statistically significant variables were defined based on the adjusted odds ratio (AOR) with 95% confidence interval (CI), and a p-value of less than .05. Thematic analysis was used for qualitative data. The overall implementation of APTS was 70% measured by availability (70.9%), compliance (70.4%), and acceptability (68.5%) dimensions. Ergometrine maleate injection, fluconazole tablet, and functional computer were not available. Government employees (AOR: 0.31, 95% CI [0.12, 0.76]) and clients’ who had a one-time visit (AOR: 2, 95% CI [1.11, 3.55]) were factors associated with the satisfaction of clients. The overall APTS program implementation was good. Unavailability of some tracer drugs, and functional computer were the main gaps. Therefore the program needs some improvement to enhance the implementation of APTS program.

Background

Medicines are the cornerstone and integral part of every health care system (Belete, 2018). Medicines contribute to achieving the Sustainable Development Goals (SDGs), including universal health coverage, by saving lives, reducing suffering, and improving health (UHC; World Health Organization, 2004). One of the most complex and vexing problems in the provision of better health is the lack of access to medicines (World Health Organization, 2007). The availability of essential medicines remains poor in many low- and middle-income countries, and a significant percentage of the global population cannot also access adequate, affordable, and effective medicines (Sachiko Ozawa et al., 2019)

Ethiopia has limited access to and affordability of essential medicines (Health, 2015; Sorato, 2019). In Ethiopia, 72.4% of key medicines are available at public health dispensaries, ranging from 66.1% to 81.4% depending on the region (Fenta, 2017). One third (35%) of health facilities reported running out of essential medicines, and 8.2% reported wastage due to expired medicines(FMOH, 2019). Absence of pharmacy professionals at all; inadequate space at dispensary and pharmaceutical store; inconsistent report and requisition form (RRF); poor utilization of internal facility report and resupply form (IFRR) and bin cards; poor counseling practices and poor handling of prescriptions were the persistent problems of the public health facilities in Ethiopia (FMOH, 2019; Gebremariam et al., 2019).

Therefore, corruption is a concern in the pharmaceutical industry (Plummer, 2012) and difficult to follow the pharmaceutical product and financial transactions as well as to ensure accountability and transparency (Ejigu, 2017; Walkowiak et al., 2018). The Amhara Regional Health Bureau (ARHB) has designed Auditable Pharmaceutical Transactions and Services (APTS) to close these gaps. In addition, Developed the Improved Access Pharmaceutical Service (SIAPS) project in 2011 as an intervention strategy was also developed for enhancing pharmaceutical services and transactions (Adinew et al., 2012). A better pharmaceutical management system can control poor selection, procurement, distribution, and inappropriate use of pharmaceutical products (Kwesi Eghan, 2017). APTS is a data-driven package of interventions designed to improve access to medicines and decrease wastage through accountable, transparent, and responsible pharmacy practice (Health, 2017). A good use of budgets, transparent and accountable transactions, accurate information, and effective development and deployment of the workforce improve customer satisfaction (Fenta et al., 2016; Health, 2017; USAID, 2017). The component of the APTS program was described using the program logic model (Figure 1).

Figure 1.

Logic model for Auditable pharmaceutical transaction service at Debark primary hospital adapted from logical framework in 2020.

Conducting a process assessment involves recognizing potential obstacles to service implementation and offering a structured information guide based on assessment criteria to enhance future service delivery. The assessment was crucial in identifying discrepancies between the current and desired practices during implementation. Hence, the outcome of the assessment may furnish useful suggestions to enhance deficient areas and fortify proficient domains, thereby enhancing the program. The stakeholders could also benefit from the findings of this evaluation. Moreover, this evaluation will augment the overall understanding of the subject matter and serve as a benchmark for future evaluators.

The APTS intervention has been implemented at Debark primary hospital since 2015 but has not yet evaluated its implementation status. Since APTS is a new intervention in Ethiopia, adequate evidence is not available on the evaluation of the implementation status of this intervention. Therefore, this study aims to evaluate the implementation status of APTS at Debark primary hospital. This evaluation aims to address the following evaluation questions: (i) Does the primary hospital have enough resources required to perform the APTS program? (ii) Do pharmacy and finance workers comply with the national guideline in delivering the APTS program? and (iii) What is the level of satisfaction of patients toward the APTS program?

Methods

Evaluation Design and Settings

A single case study design involving both quantitative and qualitative methods was used. The qualitative and quantitative data were collected concurrently, analyzed separately, and integrated during interpretation. The study was conducted at Debark primary hospital, North Gondar Zone Amhara National Regional State, Ethiopia from 03 February to 03 March 2020. Debark town is located 830 km far from Addis Ababa (the capital city of Ethiopia). The hospital has a total of 162 staff of which 75 health care providers and 87 administrative staff. The hospital is expected to serve over 1.2 million people.

Variables and Measurements

This evaluation focused on the evaluation of the process of APTS program implementation status. Availability of infrastructure and resources, compliance of health professionals with APTS guideline, and acceptability (client satisfaction) were the main dimensions of this evaluation. The dimensions of availability, compliance, and acceptability (client satisfaction) were measured by four, seven, and seven indicators, respectively. Moreover, the independent variables for satisfaction dimension were socio-demographic characteristics: (age, sex, residence, marital status, educational level, occupation, income, payment status), availability of key medicines, frequency of visit and courtesy and respect (Figure 2). The indicators were adapted from Ethiopian hospital service transformation guidelines and related literature (McNatt et al., 2015). Stakeholder analysis were done with each of the key stakes and agreed on the use of these indicators to evaluate the APTS implementation (Table 1).

Figure 2.

Conceptual framework for APTS evaluation; adapted from Logic Model Flowchart for Program Evaluation at Debark primary hospital in 2020.

Table 1.

Stakeholders’ Identification and Analysis Matrix for the Process Evaluation of Auditable Pharmaceutical Transaction and Service Program Implementation in Debark Primary Hospital North West Ethiopia, July 2020.

S.No	Stakeholders	Role in the program	Interest or perspective on evaluation	Role in the Evaluation	Possible communication strategies
1	FMOH	Training Coordinating Availing necessary inputs Funding	Improving the quality of pharmacy service	Formulate evaluation question Source of information	Telephone Email Letter
2	RHB	Training Availing necessary inputs Funding	Improving the quality of pharmacy service	Facilitation Funder Raise evaluation questions	Telephone Email Letter
3	PFSA	Supply medicines and medical equipment	Improving the quality of service	information source	Telephone E-mail Letter
4	Debark primary hospital	Training Availing necessary inputs	Improving the quality of service	Identify indicators information source Deciding on focus of evaluation	Face to face Interview
5	Health service providers	Providing service	Improving the quality of pharmacy service	Information source Use findings	Face to face Interview
6	Patients	Using service	Obtaining qualified Pharmacy service	Information source Formulating evaluation questions	Face to face Interview FGD Survey
7	Finance workers	Auditing Financing	Improving the quality of pharmacy service	Source of information raise evaluation questions.	Face to face Interview

Sample Size and Sampling Procedure

Single population proportion formula using an assumption of 50% proportion (p), 5% margin of error (d), 95% confidence level and 10% non-response rate. The final sample size was 423. For the satisfaction, a systematic random sampling technique was used to select the participants. We used the average number of patients 50 per day in Debark primary hospital that got pharmacy service at outpatient department. The value of “K” interval was determined by 30 days’ data collection through dividing 1500 patients to 423. The interval was found to be three and every three clients were interviewed to assess client satisfaction.

There have been five key informant interviews. For the key informant interviews, a purposive criterion-based sampling strategy (information-rich about the program, had experience with, and indicated by other samples) was conducted. These include the hospital’s chief executive officer, chief clinical officer, head of the pharmacy, head of finance, and auditor. Additionally, a stock report, a monthly report, a waste rate, and a tracer medication tally sheet are among the pharmacy reports (documents) that have been chosen for evaluation over the past 6 months.

Patients who got services during the data collection period whose age is 18 years and above, APTS service document from the last 6 months and assigned and working finance head, pharmacy head, CEO, chief clinical officer, and auditor were included in the evaluation. During the data collection period, patients who were very sick and unable to provide information were excluded.

Data Collection Tools and Procedure

A structured questionnaire for quantitative part and semi-structured interview guide for qualitative data collection tools were prepared from reviewing of APTS guideline (MILKESSA, 2019). APTS Program inventory tools were used to assess the availability of APTS program physical and human resources for the delivery of services and a list of resources was adapted (Wogayehu et al., 2019).The interviewer administered questionnaire and patient satisfaction tool was adapted from the APTS manual. It was used to assess the acceptability of the APTS program. APTS document review template tool was used to collect data from program documents (monthly APTS reports) and key informant interview guide tool was used to assess the availability of resources of the APTS program and barriers to implementation. This tool is comprised of components like a support system, training, and preparation, services organization, and delivery.

Three pharmacy technicians (diploma), one health information technician, and one pharmacist were recruited from other health facilities for data collection and supervision, respectively. The data collectors and the supervisor were received one-day training on objective of evaluation, data collection instruments, techniques, and ethical issues. Three pharmacy technicians conducted the interview and administered the questionnaires. Resource inventory and document review were conducted by the principal evaluator and health information technician whereas, a key informant interview was conducted by the principal evaluator.

A pretest was conducted on 5% (22) clients of the questionnaire at Sanga primary hospital and correction was made before the actual data collection. Qualitative data was collected by the principal evaluator for controlling data quality and it was audio taped.

Data Management and Analysis

Completed quantitative data were checked for completeness and consistency and then entered to Epi data version 4.6 software exported into SPSS version-25 software for analysis. For qualitative data field note was written as fair notes after data collection every night by the principal evaluator and audio record for a key-informant interview was properly recorded and transferred to a computer for the analysis. Descriptive statistics such as mean, standard deviation proportion was presented using tables, graphs, and texts. Independent variables with a p-value of less than .25 during bivariable logistic regression analyses were entered in the multivariable logistic regression analyses. Finally, Adjusted Odds Ratio (AOR) with 95% CI and p-value <.05 was used to identify factors associated with the client’s satisfaction. Qualitative data were also transcribed and translated into English. Finally, the qualitative data were analyzed using thematic analyses under each dimension. The result was compared based on the evaluation judgment matrix to determine the level of implementation status.

The matrix of analysis was developed and agreed with stakeholders along with their indicators using a rational approach. Accordingly, the matrix of judgment was developed for each sub-dimension of the evaluation with their agreed score, the weight given to each indicator. Based on the agreed parameter, the calculated indicator scores were judged as very good (85–100), good (75–84.99), fair (60–74.99), and poor (<60). Availability, compliance, and acceptability (patient satisfaction) sub-dimensions were weighted from 35%, 35%, and 30%, respectively.

Ethical Consideration

Ethical clearance was obtained from the institutional review board. Then permission letter was obtained from the Hospital. After explaining the benefit and risks of the evaluation, informed oral consent was obtained from each participant before participation. To ensure confidentiality, names were not used; instead, code numbers were assigned to depict the results, and the questionnaires were kept locked.

Results

Availability of Key Medicine

From 26 essential drugs only 80.8% and 83.5% of the selected essential drugs were available at the dispensaries and store respectively during data collection period.

The key informant interview indicated that there was a shortage of resources, particularly drugs and trained health care professionals in the hospital. “There is a shortage of drug supply to buy drugs from PFSA at Gondar and Bahir Dar cities. We will not get what we request or usually we got below 33% of the requested drugs in both by items and amount” [a 38-years-old male health officer]. This finding was also supplemented from the pharmacy department “There is supply shortage and we bought drugs directly without a bid from government organization which is known PFSA or currently the so-called EPSA, but we cannot find our consumption as per our demand. For example, we had requested 120 items, of which we got only 25 to 30 items. Shortage of drugs and logistics in the pharmacy department is obvious” [A 32 years old male pharmacist].

Availability of Trained Staff

There are twelve APTS implementing professionals, among these nine pharmacy profession and three finance professional working in the cashier window to collect daily transaction income. Of twelve pharmacy staff including APTS accountants eight were trained on auditable pharmaceutical transaction service which is organized by Amhara regional health bureau. The percentage of availability of trained professionals for Debark primary Hospital was 8 of 12 required (66.7%).

“We have a shortage of trained professionals to implement APTS program effectively. Only two APTS accountant, five pharmacists, and pharmacy head have been taken the training. However, many pharmacists are currently working without training. The hospital auditor also not took the training. This can affect our implementation since APTS is a new initiative” [a 31 years old male MSc in accounting].

Availability of APTS Tools and Pharmacy Work Flow Arrangement

Basic tools of APTS like standard treatment guideline, tablet counting trays, scissors, and calculators were avail but formulary manual and functional computers were not available in the dispensary class in Debark primary hospitals pharmacy unit. “Even though we raised the idea about necessary of computer in the dispensary class, at management meeting, nothing is done until now” [43 years old male MSc in management].

Debark primary hospital constructed the outpatient pharmacy premises and rearranged patient flow as per the APTS standards; they are having rearranged workflow, having two doors, waiting area, counseling room, standard counters in place, and staff adjustment and being cashiers within the dispensary. So, the work flow arrangement restructured as entry, evaluator/ biller, cashier, drug use counselor and exit. The assessment of APTS program in terms of program resource availability was measured to be 70.9 %, which was fair based on the judgmental parameter (Table 2).

Table 2.

Availability of Summary of APTS program resource at Debark Primary Hospital, Northwest, Ethiopia, 2020.

Indicators	E*	O*	W*	S*	A*	JP*
Number of pharmacy staff who trained on APTS	9	6	7	4.67	66.7	Fair
Number of pharmacy accountant who trained on APTS	3	2	7	4.67	66.7	Good
Proportion of clients who received 100% prescribed drugs within the last six month	15,600	10,012	8.75	5.60	64	Fair
Availability of key medicines used to treat top disease 10 in dispensing units	26	21	12.25	9.89	80.8	V.Good
Overall availability APTS program resources			35	24.8	70.9	Fair

Note. E = Expected, O = Observed, W = weight, S = Score ((observed × weight)/Expected), A = Achievement in percentage ((S/W) × 100), JP = Judgment Parameter.

Compliance of Health Care Providers to National Standards

Stock-out of Key Medicine

The minimum and maximum stock out durations in the last 6 months for most of these drugs in the hospital was 3 and 90 days, respectively. However, both ergometrine maleate injection and fluconazole tablets were out of stock for more than 90 days in the hospital. “Percentage of expired medicines were reported quarterly, Stock must be analyzed regularly to utilize the pharmaceuticals budget efficiently, in this regard, we analyzed the stock but not regularly” [A 32 years old male pharmacist].

Matching of Recorded Quantity With Physical Count at Medical Store

The actual physical count for each selected essential medicine was checked against the amount recorded on bin cards at the pharmacy store. Of 26 items, six of them were not accurately recorded. “Discrepancy between the quantities of medicines recorded on bin card and with the actual physical count is not regularly checked. I think that is why this variation has occurred” [43 years old male MSc in management].

The finding was also supported with the KI interview from the CEO office “Taking a sample of pharmaceutical products randomly and matching them with different financial documents is the most common way of auditing in the hospital. In doing so, we believed that APTS helped us in auditing pharmaceutical transactions. However, the absence of regular financial auditing and manual recording of a large number of items were reported” [43 years old male MSc in management].

The finding was also supplemented with the KI interview from the accountant department “Drugs and related equipment received with model 19 into the store and ratified by model 22. To control this movement there are bin cards, stoke cards, and other registrations based on the stoke management system. To conduct an audit, APTS accountants use APTS software but we did not receive training on it, thus we tried to conduct an audit by referring to the training manuals. However, the physical counts and ending inventory could not be balanced; the reason for this might be our knowledge gap” [A 31 years old male MSc accountant].

Adequate Labeling of Dispensed Medicine

It was reviewed labeling information on the package by randomly picking one of the medicines. Only 60% of reviewed medicines were properly labeled with the drugs’ duration information on the package. About 75% of medicines had a written name on its package and 78% of reviewed medicines had also a written strengths and frequency properly.

Transaction Recording Systems

Inventory management at Debark primary hospital was not computerized but they were used stock cards, requisitions and receipt vouchers (Models 19–22). In the last 6 months, 83%, and 100% monthly service reports and financial reports were reported respectively. Regarding transaction recording, the daily financial summary for cash sales was 53% and the daily financial summary for free provision of medicines was 35% for the last month. “This hospital is practicing only on the submission of monthly financial reports. Summation of service reports are still conducted, but not every month” [43 years old male MSc in management staff]. The assessment of the APTS program in terms of professional compliance was measured to be 70.4 %, which was fair although it needs improvement based on the judgmental parameter (Table 3).

Table 3.

Summary of Performance Indicators of Compliance to National APTS Guidelines at Debark Primary Hospital, Northwest, Ethiopia, 2020.

Indicators	E*	O*	W*	S*	A*	JP*
Proportion of stock status analysis reports of the last 6 months	6	2	3.5	1.17	33	Poor
Average stock-out duration of selected key medicine (days)	90	75	5.25	4.4	83.8	Good
Proportion of monthly financial report for the last 6 months	6	6	5.25	5.25	100	Very good
Proportion of prescribed medicine dispensed adequately labeled	100	65	5.25	3.41	65	Very good
Proportion of drugs matched recorded quantity with physical count	26	18	5.25	3.675	70	Fair
Proportion of daily financial summary report for cash sales for the last month	20	11	7	3.85	55	Poor
Proportion of utilizing transaction recording form/document	6	5	3.5	2.9	82.5	Good
Overall compliance of APTS program providers			35	24.66	70.4	Fair

Note. E = Expected; O = Observed; W = weight; S = Score ((observed × weight)/Expected); A = Achievement in percentage ((S/W) × 100); JP = Judgment Parameter.

Patient Satisfaction (Acceptability) on APTS Program

Socio-Demographic Characteristics of Patients

A total of 423 clients responded to the interviewer administered questionnaire with a response rate of 100%. Among the study participants, about 255 (60.3%) were males. Slightly above half (53.7%) of the participants reside in rural areas. The age of clients ranged from 18 to 83 years and their mean age was 37.7 (SD ± 13) years. Among participants 143 (33.8%) were unable to read and write. Of respondents’ majority (64.5%) were married. Nearly half, 201(47.5%) were farmers (Table 4).

Table 4.

Socio-Demographic Characteristics of Respondents in Debark primary Hospital, Northwest Ethiopia, 2020 (N = 423).

Variables	Category	Frequency	Percent
Sex	Male	255	60.3
	Female	168	39.7
Residence	Urban	196	46.3
	Rural	227	53.7
Marital status	Single	117	27.7
	Married	273	64.5
	Divorced	26	6.1
	Widowed	7	1.7
Educational level	Unable to read and write	143	33.8
	Can read and write	73	17.3
	Primary school	55	13
	Secondary school	77	18.2
	College and above	74	17.5
Occupation	Farmer	201	47.5
	Government employee	71	16.8
	Merchant	91	21.5
	Student	60	14.2
Payment status	Fee waiver users	81	19.1
	Out of pocket	302	71.4
	CBHI covered	39	9.2
Frequency of visiting	One	149	35.2
	Two	80	18.9
	More than two times	194	45.9

Level of Client Satisfaction

Patient satisfaction was also evaluated using seven APTS program indicators. Accordingly, patient satisfaction for the APTS program was measured to be 68.5%, which was fair based on the judgmental parameter (Table 5).

Table 5.

Summary of Performance of Satisfaction Indicators for APTS Program Debark Primary Hospital, Northwest, Ethiopia, 2020.

Indicators	E*	O*	W*	S*	A*	JP*
Proportion of clients satisfied with the information about the result of medication given by pharmacists	423	301	4.5	3.20	71.1	Fair
Proportion of patients satisfied with the simplicity/convenience to get OPD pharmacy at Hospital	423	283	4.5	3.01	66.9	fair
Proportion of patients satisfied with the cleanness/comfortably of waiting area	423	308	3	2.18	72.7	Fair
The proportion of patients satisfied with the adequacy of explanation about the treatment	423	310	3	2.20	73.3	Fair
The proportion of patients satisfied with the attention and respect of providers to their privacy	423	246	6	3.49	58.2	Poor
Proportion of patients satisfied with the fairness of the cost of medicines in the pharmacy	423	300	4.5	3.19	70.9	Fair
The proportion of patients satisfied with the time spent while waiting for service	423	297	4.5	3.16	70.2	Fair
Overall satisfaction of clients			30	20.43	68.5	Fair

Note. E = Expected; O = Observed; W = weight; S = Score ((observed × weight)/Expected); A = achievement in percentage ((S/W) × 100); JP = judgment parameter.

Overall Process Evaluation of APTS

The overall process of the auditable pharmaceutical transaction service program implementation was 70%, as measured by the availability of essential drugs and materials, healthcare providers’ compliance with the national guideline, and patient satisfaction (Table 6).

Table 6.

Summary Dimensions of the Process Evaluation of for APTS Program Debark Primary Hospital, Northwest, Ethiopia, 2020.

Dimension	Expected	Observed	# Indicators	Weigh	Result	Judgment
Availability	100%	70.9	4	35	24.8	Fair
Compliance	100%	70.4	7	35	24.66	Fair
Satisfaction	100%	68.5	7	30	20.43	Fair
Overall evaluation of the program			18	100	70	Fair

Factors Associated With Client Satisfaction

Residency, marital status, occupation, and frequency of visit of clients’ variables interred in multiple logistic regressions analysis. Among these occupational status and frequency of visit were significantly associated with client satisfaction with the APTS program. Accordingly, government employees were 69.4% (AOR: 0.31, 95% CI [0.12, 0.76]) less likely to be satisfied than a farmer. Moreover, clients who visited one time were 2 (AOR: 1.99, 95% CI [1.11, 3.55]) times more likely satisfied compared to those visited more than two times (Table 7).

Table 7.

Factors Associated With Clients’ Satisfaction on APTS Program at Debark Primary Hospital, Northwest, 2020 (n = 423).

	Satisfaction
Variables	Satisfied (%)	Dissatisfied (%)	COR	AOR [95% CI]
Residence
Urban	115 (50.6)	112 (49.3)	1	1
Rural	108 (55.1)	88 (44.9)	1.195	1.514 [0.760, 3.014]
Marital status
Single	59 (50.4)	58 (49.6)	1	1
Married	142 (52)	131 (48)	1.06	0.136 [0.015, 1.277]
Divorced	16 (61.8)	10 (38.4)	1.50	0.180 [0.020, 1.584]
Widowed	6 (85.7)	1 (14.3)	5.267	0.255 [0.026, 2.526]
Occupation
Farmer	106 (52.7)	95 (47.3)	1	1
Government employee	36 (50.7)	35 (49)	0.921	0.306 [0.123, 0.762]*
Merchant	46 (50.5)	45 (49.5)	0.916	0.805 [0.363, 1.785]
Student	36 (60)	24 (40)	1.344	0.488 [0.230, 1.033]
Visited frequency
One	87 (58.3)	62 (41.7)	1.49	1.99 [1.114, 3.559]*
Two	51 (63.75)	29 (36.3)	1.87	1.220 [0.750, 1.987]
More than two	94 (48.5)	100 (51.5)	1	1

*

Statistically significant at p-value <.05.

Discussion

When the drug supply management system is functioning successfully, the right drug items are always available in the right amounts, at the right rates, with the right use, at the right times, and of the right quality. As such, transparency and accountability will be given to the practice of pharmaceutical transactions and the provision of pharmaceutical services. Therefore, this evaluation was carried out to determine whether the standards are met or not. In this evaluation, the formative approach with driven indicators was used to measure the process of APTS program implementation. The overall implementation status of APTS program was 70%, which was fair as per the presetting judgment parameter (JP). Availability resources for the program implementation were 70.9% which was judged as fair. Compliance of APTS workers to the APTS guideline was 70.4% and satisfaction of the program services by clients was 68.5%, which were judged as fair as per the JP.

The results of this evaluation revealed that the study hospital fulfilled most of the required equipment for the APTS program according to APTS guideline standard with a shortage of some supplies and drugs. Our result showed that the availability of key medicines is about 80.8%, 83.5% at dispensary and store respectively. This result is different from the study conducted in Gamo Zone district hospital that fifteen out of 25 (60%) medicines were avail (Zone and Ethiopia, 2019). This difference might be due to the availability of implementation of APTS program at the study hospital. But this result is comparable with the study conducted in two APTS implementing hospitals in the Gamo Gofa zone that the availability of all prescribed drugs was reported more frequently in primary hospitals than in general hospitals 82% versus 78% (Mesfin, 2018). This similarity might be due to the implementation of the APTS program in both Hospitals.

The adequacy of trained manpower those were trained on the APTS in our study is different from the APTS standard guidelines where only 66.7% of health care providers had been trained on the APTS guideline. A considerable number of APTS implementers did not receive training on the APTS, thus might hinder for successfully implementation of the program in the hospital. The APTS guideline recommends, the drug information center (DIC) should be available in the APTS room but our observation showed that the drug information center is far away from the room.

The average wastage rate of the drug of this hospital was 0.93%, this result congruent with the recommended wastage rate of APTS guideline that is less than 1.5% (Adinew, 2012). Basic tools of APTS such as model 19 and model 22, cash sales ticket, and dispensing registers were available and in use correctly. That is similar to a descriptive study conducted for assessment on auditable pharmaceutical transactions and services implementation outcomes at Amanuel mentally specialized hospital (Alemayehu, 2017). The presence of stock-out of the drugs for over three months is another finding in this study. This result is different from the study conducted in Tikur Anbesa specialized hospital that showed the maximum stock-outs of drugs and equipment were not more than 60 days (Milkessa, 2019). This difference might be due to inadequate allocation of budgets for medicines at the study hospital.

An appropriate label should include at least the drug name and strength, as well as written instructions for how to take the drug on the envelopes. Written information on drug containers indicates the level of care given by the dispenser when dispensing the drug, and helps to remind patients\caregivers when to take the drug, how long to take it, and how much to take, in case they forget the verbal instructions for taking the drug. According to the data on drug labeling, Debark primary hospitals perform below the expected (100%) in terms of drug labeling performance (Tadeg et al., 2014).

Our assessment also showed that the number of the daily summary service reports and financial summary for cash sales for the last 1 month is 65%, 55% respectively which is low and incongruent with APTS implementation guideline. This might be due to a lack of training and capacity building including mentorship and supportive supervision. The absence of regular financial auditing and manual recording of a large number of items were reported to be the major challenges regarding the auditing task. It was also reported that the absence of regular auditing could be the reason for low discrepancy detection. With such inconsistent auditing practices, the pharmacy service will lead to mismanagement and inefficient utilization of resources (Ejigu, 2017; Walkowiak et al., 2018).

In general, all patients should receive essential information on drugs prescribed to them to maximize the benefits they obtain from their medications. Pharmacists have an important role in providing drug information for patients to minimize medication errors especially related to a dose causing unnecessary morbidity and mortality in patients receiving drugs.

The overall level of client satisfaction (acceptability) at this evaluation area with the auditable pharmaceutical transaction service was 68.5% which was less than the presetting judgment parameter (85%–100%). This evaluation in line with a study conducted at Hiwot Fana specialized hospital in Harar with overall patient satisfaction of 53.9% (Reta et al., 2017).

Concerning the availability of certain drug satisfaction was 280 (66.2%) but the study done at Hiwot Fana specialized showed that 154 (46.8%; Reta et al., 2017). This difference might be due to different study settings and different numbers of client flow. Our study showed that 330 (78%) of respondents in Debark primary hospital were satisfied with the amount of time that the pharmacy professionals spend with them for providing the appropriate medications information. However, the study done in Gondar university referral hospitals showed that only 9.2% of the respondents were satisfied with the amount of time spent with the pharmacy professionals (Surur et al., 2015). This might be due to there is a variation in pharmacy professionals’ approach to the respondents. Based on the findings of this study on Debark primary hospital 280 (66.25%) of respondents were satisfied with the availability of prescribed medications in the hospital. However, a study conducted in Jimma University Specialized hospitals showed that 30% of the clients were satisfied with the availability of the prescribed drug in hospitals pharmacy (Reta et al., 2017). The variation might be due to differences in study setups and client flow. About 83.9% of the respondents were satisfied with the privacy of outpatient pharmacy. This finding was higher than an assessment done in Addis Ababa teaching hospitals showed that 54.9% of respondents were satisfied with the privacy of pharmacy service (Tadeg et al., 2014). This may be due to having proper and well-organized premises or infrastructures.

Strengths and Limitations of the Evaluation

Our work used three dimensions to evaluate the process of APTS implementation which makes it more valid than measuring the process by a single dimension. Besides, using both qualitative and quantitative methods (triangulation) also helped us get accurate and detailed results. The finding of this study might have subjected to social desirability bias because the respondents were interviewed in the hospital compound. The improper registration of all of the services delivered was an additional limitation of the evaluation for the assessment of auditable pharmaceutical transaction service. Furthermore, patients experienced a relatively short-lived whereby they felt more satisfied immediately after their consultation than they did afterward. Since APTS is a new system implemented in Ethiopia only, adequate literature was not available to compare the implementation status of the program.

Conclusion

The overall evaluation of the program was fair; however, there were certain constraints in the program implementation. Unavailability of some tracer drugs such as ergometrine maleate injection and fluconazole tablet, functional computer, and drug formulary manual were the major constraints in the structural aspects of evaluation. On top of that, incomplete monthly service reports and daily financial service summary for cash sales were also the fundamental gaps in the compliance dimension of process evaluation. Even though the overall patient’s satisfaction was good, poor information exchange about medication, unavailability of the prescribed drugs and inadequate leveling off prescribed medicine were the major complaints of the users. Moreover, patient occupational status and frequency of health facility visits were the factors affecting the overall client satisfaction. It is preferable to train the pharmacy staff and finance workers who are responsible for APTS to ensure proper recording of services, which will improve the program’s execution. Supplies, medications, and other items must arrive at the hospital on schedule. Additionally, the government needs to keep track of, assess, and provide staff with feedback on the APTS program’s performance at the hospital.