Do You Like What You See? How Nonprofit Campaigns With Output,Outcome,and Impact Effectiveness Indicators Influence Charitable Behavior

Abstract

This study contributes to recent discussions on voluntary disclosure as a signaling approach among nonprofit organizations and its effects on stakeholders’ decision-making. Focusing on nonprofit program effectiveness, we test how nonprofit campaigns providing information on three effectiveness indicators—outputs, outcomes, and impacts (as part of the logic framework)—influence donation and lending behavior. An online survey experiment (N = 271) reveals that donors value outcome and impact indicators more than output information, without any differences between the two. Moreover, the three indicators have no statistical influences on lending behavior. We also consider the moderating role of reflective decision-making and find no influence on either donation or lending behavior.

Keywords

nonprofit program effectiveness logic framework (output,outcome,impact)charitable behavior decision-making mode signaling approach

Introduction

Increased competitive environments in nonprofit sectors worldwide put nonprofit organizations under increasing scrutiny to demonstrate their effectiveness (e.g., Carman, 2010; Mitchell & Berlan, 2016; Schubert & Boenigk, 2019). Stakeholders, and donors in particular, have shifted from a “trust-me” to a “show-me” attitude (Greiling, 2014). They pose higher demands and seek more elaborate information about the organization’s programs and services. To adequately address stakeholders’ demands, nonprofit scholars highlight the importance of voluntary nonprofit accountability and forms of voluntary disclosure among nonprofit organizations to positively affect stakeholders’ perceptions and signal organizational success (Becker, 2018; Parsons, 2007; Saxton, Kuo, & Ho, 2012).

One measure of organizational success is effectiveness, which consists of several domains, including managerial and program effectiveness (Lecy, Schmitz, & Swedlung, 2012). With the rise of social entrepreneurship, program effectiveness has gained importance more recently, but a precise definition is lacking (e.g., Grieco, Michelini, & Iasevoli, 2015). From the various theoretical models and approaches available, the logic framework (also known as logic model, results chain, or impact value chain) can form a better understanding. Discussions about this framework tend to be fragmented across disciplines (Jun & Shiau, 2012; Lecy et al., 2012), but three indicators are consistently mentioned: outputs, outcomes, and impacts (e.g., Carman, 2010; Costa & Pesci, 2016; Ebrahim & Rangan, 2014; Grieco et al., 2015; Mitchell & Berlan, 2016). Outputs refer to immediate effects, outcomes reflect the intermediate effects, and impacts describe the long-term, intended and unintended effects of a program on beneficiaries and overall society (Bagnoli & Megali, 2011; Ebrahim & Rangan, 2014). Building on Grieco et al.’s (2015) study, we, therefore, define program effectiveness as to how successfully a nonprofit program created value, measured by its effectiveness indicators outputs, outcomes, and impacts.

Despite the increasing importance of program effectiveness in nonprofit practice, evidence in terms of its influence of donors and their behaviors is scarce. Precisely, academic research lacks understanding of whether regular donors really rely on or differentiate among the different indicators of effectiveness (Lecy et al., 2012). Moreover, no study compares the effects of all three. Instead, scholars focus on how using one indicator influences charitable behavior (e.g., Aknin, Dunn, Whillans, Grant, & Norton, 2013; Cryder, Loewenstein, & Scheines, 2013) or compare two indicators at most (Karlan & Wood, 2014; Verkaik, 2016).

Next to donations, new forms of giving are emerging with the heightened transfer of for-profit concepts to the nonprofit sector (Grieco et al., 2015; Schrötgens & Boenigk, 2017). As one popular concept, charitable lending encourages private individuals to provide small loans to charitable causes, which they expect to get back (potentially with interest) after a certain period (Ly & Mason, 2012). Compared to donors’ decision-making processes, the proof of effectiveness (also called social impact assessment) might be even more important in a lending context (Costa & Pesci, 2016; European Union & Organisation for Economic Co-operation and Development [OECD], 2015). Extant literature focuses on the characteristics of social entrepreneurs and their businesses as key factors that drive funding success (e.g., Jenq, Pan, & Theseira, 2015; Ly & Mason, 2012). However, despite some indications that the projects’ perceived effectiveness plays a crucial role in determining investment decisions (Mittelman & Rojas-Méndez, 2013), this fact has been neglected so far.

Noting these research gaps, this study aims to assess how implementing each of the three program effectiveness indicators—output, outcome, and impact—in nonprofit campaigns influences individual donation and lending behaviors. In turn, it provides crucial insights for nonprofit organizations that currently incur high costs and undertake substantial efforts to measure their program effectiveness (Carnochan, Samples, Myers, & Austin, 2014). Particularly, impact indicators are difficult to measure (e.g., Ebrahim & Rangan, 2014; Mitchell & Berlan, 2016), so understanding which indicators donors and lenders prefer to rely on can help nonprofit organizations weigh the costs and benefits of collecting data about them.

By including an additional research stream, this study combines individual decision-making modes with the effectiveness indicators and its separate effects on donation and investment decisions. Prior research shows that people rely on different decision-making styles to process pertinent information when making donation or investment decisions (Dickert & Sagara, 2011; Erlandsson, 2016; Kahnemann, 2003, 2011; Small, Loewenstein, & Slovic, 2007). Depending on the organizational program and service, different indicators potentially assist the individual decision-making behavior, and thus contribute to organizational success. Therefore, this study considers the moderating effect of a reflective decision-making mode with the effectiveness indicators and its separate effects on both donation and investment decisions. Studying this moderating effect is salient, in that it reveals whether donors really respond to requests for “impact philanthropy” (Duncan, 2004) or “effective altruism” (Singer, 2015). Thus, findings of this study can help nonprofit leaders understand under which assumptions donors and lenders make their decisions, and accordingly shape organizational programs and campaigns.

Theoretical Foundation and Hypotheses

Effectiveness Indicators in the Logic Framework

The logic framework provides a clear structure for different effectiveness indicators applicable to many nonprofit organizations (Ebrahim & Rangan, 2014; Phineo, 2017). Due to inconsistent use of terminology, Table 1 explains each indicator.

Table 1.

Logic Framework.

Input →	Activities →	Effectiveness indicators
		Output →	Outcome →	Impact
		Immediate	Medium term (1-3 years)	Long term (3-10 years)
Examples
Human capital, funds , equipment	Addressing basic needs (e.g., food supply), providing services (e.g., education)	People trained , buildings built	People employed after training	Permanent job with sufficient income, sustained drop in poverty

Source. Adapted from Ebrahim and Rangan (2014).

Against the background of information asymmetries between nonprofit organizations and its stakeholders, the effectiveness indicators have potential to provide important signals of quality to the public. Donors and other nonprofit stakeholders lack information and face uncertainty in the process of assessing the organization’s behavior or the quality of nonprofit programs and services (Prakash & Gugerty, 2010). Along with their higher demands, nonprofit stakeholders seek for elaborate information about nonprofit organizations and its services (Prakash & Gugerty, 2010). One way to address stakeholders’ demands and increased skepticism is the use of signals in overcoming information asymmetries (Moleskis, Alegre, & Canela, 2019; Willems, Waldner, Dere, Matsuo, & Högy, 2017). In this vein, forms of voluntary accountability and disclosure (including financial and nonfinancial reporting beyond legal requirements, compliance with codes of conduct, or accreditation) have recently garnered much attention. Such measures intend to signal the quality and trustworthiness of nonprofit organizations and their services to the public (Becker, 2018; Parsons, 2007; Saxton et al., 2012). Buchheit and Parsons (2006) and Parsons (2007) first empirically analyzed the effects of voluntary nonfinancial disclosures and other accounting information in a nonprofit setting and found effects of greater organizational transparency (Buchheit & Parsons, 2006) and increased behavioral decision-making (Parsons, 2007). Against this background, a growing number of nonprofit organizations seek to create a favorable public response using accountability and disclosure instruments in their marketing and quality management efforts.

The provision of elaborated information on nonprofit programs can be another opportunity for stakeholders to adequately assess the organizations’ services and programs. It follows that the logic framework can depict an effective approach supporting stakeholders in their decision-making processes. Besides using such additional information to overcome uncertainty relating to the organization’s adequate provision of services, they also show their individual reputation and status such that they can inform others about their donation (Ariely, Bracha, & Meier, 2009; Bekkers & Wiepking, 2011; Glazer & Konrad, 1996). The latter has been found to be a core mechanism of why donors give. It thereby draws importance to the design of nonprofit campaigns so that donors are able to use information for social signaling and reputation matters (Ariely et al., 2009; Bekkers & Wiepking, 2011; Glazer & Konrad, 1996). On the contrary, the logic framework can be a powerful tool for nonprofit organizations to design their services and programs so it helps assessing program effectiveness, and thus organizational performance. The expected effects are hence based on the signaling approach, and on different mechanisms for giving, as outlined in greater detail below.

Donor Preferences for Effectiveness Indicators

As a central mechanism in charitable giving, Bekkers and Wiepking (2011) point out to the donor’s perception of his or her donation making a difference. Earlier studies also consider how perceptions of managerial effectiveness, such as in terms of governance, leadership, or cost ratios, relate to individual donor behavior (e.g., Lecy et al., 2012). More recently, the focus has shifted to the impacts of program effectiveness indicators. Duncan (2004, p. 2176) introduces “impact philanthropists,” described as people who “value making a difference” and want to see immediate effects. These donors prefer to support beneficiaries directly, not via an organization. Relative to the logic framework, this view suggests that donors likely value output indicators most. Several studies confirm donors giving more if they know exactly how their donation will benefit a specific purpose or beneficiary (e.g., Aknin et al., 2013; Verkaik, 2016). Furthermore, specific donation appeals including outcome indicators, such as “saving lives” (Cryder et al., 2013, p. 91), have a positive effect on donations as well.

The more recent effective altruism movement (Singer, 2015) goes even further to demand a focus on nonprofit programs that do the greatest amount of overall good. It calls for support of programs that not only reach the maximum beneficiaries (output) but those that ensure medium-term effects on beneficiaries and achieve long-term positive effects on societies. Thus, our first hypothesis is as follows:

Hypothesis 1a (H1a): Donors prefer outcome and impact indicators over output indicators.

The preference for either outcome or impact indicators is not clear though. According to effective altruism, donors prefer impact over outcome indicators, but there is no evidence of this prediction (e.g., Karlan & Wood, 2014; Verkaik, 2016). Regarding donations, previous research shows that emotions, for example, distress or sympathy toward a victim (Bekkers & Wiepking, 2011; Erlandsson, 2016), positively influence donations. Outcome indicators might evoke such emotions by providing tangible information about the medium-term effects for beneficiaries. Impact indicators instead may deflect the emotional focus, with their more intangible, long-term focus on both beneficiaries and society. We, therefore, hypothesize as follows:

Hypothesis 1b (H1b): Donors prefer outcome indicators over impact indicators.

Lender Preferences for Effectiveness Indicators

Literature on charitable lending thus far has been dominated by studies of the microfinance platform Kiva, which already has facilitated loans worth more than US$1.3 billion to people around the globe (Kiva, 2019). The studies identify several pertinent influences on funding success, mainly related to the appearances and characteristics of people borrowing money, sectors, and countries (Jenq et al., 2015). For example, women are generally funded faster, as they are more attractive, lighter skinned, and less obese borrowers. Other studies focus on narratives in the funding campaigns (e.g., Allison, Davis, Short, & Webb, 2014), with lenders responding more positively to narratives describing ventures as an opportunity to help others rather than those describing a business opportunity. Although these studies provide interesting insights into existing loans, little evidence indicates if lenders take the different effectiveness indicators into account and value the information they provide when making investment decisions.

In for-profit lending or investing studies, default and delinquency rates provide the primary measures of effectiveness, but such information is less relevant for charitable lending (Jenq et al., 2015). Instead, charitable lenders’ motives appear more similar to donors’, in that they prioritize the sense of making a difference (e.g., Costa & Pesci, 2016; Mittelman & Rojas-Méndez, 2013). However, unlike donors, lenders likely take a more long-term view on their investments (European Union & OECD, 2015). It follows that they might value impact indicators most, followed by outcome indicators, but both more than output indicators. Therefore, we predict the following:

Hypothesis 2a (H2a): Lenders prefer outcome or impact indicators over output indicators.

Hypothesis 2b (H2b): Lenders prefer impact indicators over outcome indicators.

Moderating Role of Decision-Making Mode

People rely on different decision-making styles to process the information they receive and arrive at a decision (Dickert & Sagara, 2011; Erlandsson, 2016; Kahnemann, 2003, 2011; Small et al., 2007). In cognitive and social psychology, the dual process model is a dominant theoretical framework that identifies two distinct decision-making modes: affective and calculative, or System 1 and System 2 in Kahnemann’s (2011) terminology. System 1 is an affective, automatic, and rapid decision-making style, often labeled intuitive decision-making. It relies on emotional processing and has been linked to moral attitudes and moral behavior (Erlandsson, 2016), as well as charitable giving (Small et al., 2007). People who display charitable behavior thus might rely predominantly on an intuitive decision-making style.

System 2, the calculative decision-making style, instead tends to be referred to as reflective decision-making, in that it implies conscious decisions that require substantially more effortful mental activities than intuitive decision-making. The effective altruism movement recommends such conscious decision-making about charitable donations and investments, and Erlandsson (2016) posits that people who consider effectiveness indicators in their charitable decision-making process employ reflective decision-making styles. Similarly, investment decisions tend to be deliberate and well thought out. If people rely on a reflective decision-making style when making charitable decisions, they likely take the time to reach a conscious decision. Therefore, more effectiveness indicators might be especially appreciated by reflective decision makers, for both their donation and their lending behaviors. Thus, the third hypothesis is as follows:

Hypothesis 3 (H3): More reflective decision-making interacts with outcome and impact indicators to positively influence donations and investments.

An overview of these hypothesized relationships appears in Figure 1.

Figure 1.

Conceptual model.

Method

Participants

Students offer an interesting and salient sample for this study; changes in philanthropy in terms of donors’ impact orientations, social entrepreneurship, and interest in investing are driven largely by younger people, including the latest generation of business students (Salamon, 2014). Previous studies of effectiveness indicators accordingly used student samples (e.g., Aknin et al., 2013; Cryder et al., 2013). Moreover, a recent study confirms that younger people are more likely to invest smaller amounts of money, which is pertinent to the current study context (Schrötgens & Boenigk, 2017). In addition, from a methodological perspective, student samples are highly suitable in experimental contexts due to the high homogeneity of this group (Koschate-Fischer & Schandelmeier, 2014). The characteristics of the 271 participants are outlined in Table A4 of Supplemental Appendix. Briefly, 58.7% of the participants identified as women and 39.9% as men. Their ages ranged from 18 to 43 years (M = 25.07 years), and 44.3% had already completed their bachelor’s degree. A considerable portion of the participants were majoring in business or economics (40.2%).

Research Design and Procedure

Hypotheses were tested by means of an online experiment and an online survey. First, each participant read a welcome message, along with a confidentiality statement. Second, to test the reflective decision-making as a moderator, the measures appeared in a 4-item scale adapted from Scott and Bruce (1995). To confirm the measurement quality of the reflective measurement models, item reliability (Cronbach’s alpha) should be above .7 (Hair, Black, Babin, & Anderson, 2010). This criterion was met by both reflective item measures (Table A1, Supplemental Appendix). Third, participants read a short paragraph about the charitable organization AEC-Foyer Lataste. Using an existing nonprofit organization helped ensure that the experiment was realistic, though opinions about an existing organization could influence participants (Pieters & Wedel, 2004). Therefore, this study used a French organization that mostly acquires funding from institutional donors; it was unlikely to be known by the sample group. The organization was presented as charitable, but without explicitly labeling as a nonprofit organization or social enterprise. Social enterprises can be nonprofits, but excluding any mention of the organizational form seemed appropriate, in light of Andersson and Self’s (2015) findings of a social entrepreneurship bias. That is, individual donors are prone to perceive social enterprises as effective, regardless of their actual effectiveness.

Fourth, on a subsequent page, participants were randomly allocated to one of the three experimental groups and saw a fictitious campaign (see Figure 2). To assess the effects of the different indicators, this study employed a 3 (Output, Outcome, Impact) × 1 Between-Subjects design. The campaign layout is the same in all three manipulations, and it reflects recent research on preferred donation sector and beneficiaries, including best practices related to attention capture and transfer in advertising (Pieters & Wedel, 2004; Verkaik, 2016).

Figure 2.

Original output indicator campaign.

To manipulate the output, outcome, and impact indicators, the headers and text boxes appearing in the campaigns differed (Table A2, Supplemental Appendix). The output campaign focused on the immediate effects, by telling donors that with their help, youngsters such as the portrayed girl could start a training program immediately. The outcome manipulation, based on the average time horizon of the training program, instead highlighted the effects of the successful completion of the program: a secure job with a regular income a year later. Finally, the impact manipulation stressed the long-term perspectives of the participants and the reduced poverty of their societies. No numbers appeared in any of the campaigns to avoid evaluability biases or proportion dominance effects (Baron & Szymanska, 2010; Erlandsson, 2016).

After questions on the manipulation, participants described their charitable (donation and lending) behaviors; the order of these two types was randomized. Because motivation to participate is crucial, this study followed standard experimental procedures from previous research (e.g., Becker, 2018; Freeman, Aquino, & McFerran, 2009). It thus entered participants in a lottery to win an Amazon gift voucher worth €30; the distribution of the gift vouchers was random. The measure of the donation-dependent variable asked participants to choose to donate some portion, in €5 increments, or the entire €30 of the Amazon gift card that they were promised a chance to win during the study (Freeman et al., 2009). The measure of the lending behavior variable instead indicated that AEC-Foyer Lataste had started a loan campaign, in which they could offer loan amounts in €25 increments (reflecting the actual practices of the online lending platform Kiva.org). For this measure, participants received no incentives; they simply indicated if they were willing to issue a loan in such increments. The measures of the dependent variables and further questions about the participants’ perceptions of the organization’s effectiveness and social impact are detailed subsequently.

Furthermore, participants indicated whether they knew the organization beforehand on a 7-point Likert-type scale (Cryder et al., 2013). They also rated their information preferences when selecting charitable projects, again on a 7-point Likert-type scale. Finally, they completed some sociodemographic items.

Manipulation and Confounding Checks Measures

Following previous research (e.g., Cryder et al., 2013; Lee, Winterich, & Ross, 2014), this study includes an instructional manipulation check at the end of the questionnaire. Specifically, all three campaign headers and texts reappeared, and participants had to indicate which one corresponded to the campaign they had previously seen. Manipulation checks can also test the effectiveness of the variation imposed by the experimental manipulation (Koschate-Fischer & Schandelmeier, 2014). Many researchers regard such manipulation checks as informative confirmations of internal and construct validity (Flake, Pek, & Hehman, 2017). The logic framework, which is central to this study, contains different indicators of effectiveness, but no empirical tests indicate if these indicators really offer signals of effectiveness. Therefore, a one-item manipulation check tests for perceived effectiveness (Andersson & Self, 2015; Table A1, Supplemental Appendix).

On the contrary, other researchers argue against the use of manipulation checks considering the fact that the manipulation might have influenced other variables as well (i.e., confounding factors; Fayant, Sigali, Lemonnier, Retsin, & Alexopoulos, 2017). Therefore, this study also included measures of other potential variables of influence to investigate the effectiveness indicators (e.g., Cryder et al., 2013; Verkaik, 2016). The check for how the three effectiveness indicators were perceived in terms of their social impact relied on four questions. One question appeared immediately after participants viewed their assigned campaign, together with questions about perceived innovativeness, sector attractiveness, trustworthiness, and perceived social impact (Schrötgens & Boenigk, 2017), which helped avoid participants’ reverse self-justification. After the dependent variable measures, the other perceived social impact items followed (adapted from Verkaik, 2016). All four questions were then combined into a single perceived social impact factor, with satisfactory factor loadings and item reliability (Table A1, Supplemental Appendix).

Manipulation and Confounding Checks Results

The experiment was pretested with 132 incentivized students, following a small, within-subjects pretest with a convenience sample of 44 people who viewed all three campaigns and indicated how effective they perceived each of them. The pretest revealed significant mean differences among the output, outcome, and impact indicator campaigns, χ²(2, 38) = 20.869, p < .01. However, asking the same person to rate each of the campaigns (within-subject design) is subject to various biases, so it would not be appropriate for the larger pretest or main study. It also revealed mean differences, such that perceived effectiveness varied across the three experimental groups, χ²(2, 111) = 7.985, p < .05, as did perceived sector attractiveness χ²(2, 119) = 6.370, p < .05. However, no significant differences emerged for perceived social impact, χ²(2, 119) = 3.400, p > .10; perceived innovativeness, χ²(2, 119) = .890, p > .10; or perceived trustworthiness, χ²(2, 119) = 1.615, p > .10.

The main study took place in December 2016 with university students. Because 41 of the original 312 participants failed the instructional manipulation check, they were excluded from the data analysis, leaving 271 participants. Table 2 details the questionnaire, including the results for the manipulation and confounding checks. In all cases, the means for the output indicator group were lower than those for the outcome and impact indicator groups. A series of Kruskal–Wallis tests confirm the significant differences among groups for the manipulation check, perceived effectiveness: χ²(2, 271) = 14.386, p = .001 and the confounding checks, perceived sector attractiveness: χ²(2, 271) = 40.466, p = .000; perceived innovativeness: χ²(2, 271) = 9.905, p = .011; perceived trustworthiness χ²(2, 271) = 14.704, p = .001; perceived social impact: χ²(2, 271) = 17.917, p = .000. The results further indicate the potential for mediation. As Hayes (2013) explains, simple mediation entails a causal sequence in which X affects Y indirectly through a mediator. For example, the results imply that outcome and impact indicators influence perceived social impact more positively than output indicators do. If perceived social impact then positively affects donation and lending behaviors, perceived social impact might function as a mediator. Further analyses of the correlations of the manipulation and confounding variables highlight some moderate correlations (Table A3, Supplemental Appendix).

Table 2.

Descriptive Statistics.

	Output(n = 86)		Outcome(n = 89)		Impact(n = 96)		Overall(n = 271)
	M	SD	M	SD	M	SD	M	SD
Behavior
Donation	0.36	0.48	0.54	0.50	0.52	0.50	0.48	0.50
Lending	0.07	0.26	0.05	0.21	0.63	0.24	0.06	0.24
Decision-making
Reflective	5.18	0.86	5.19	0.97	5.29	0.85	5.22	0.89
Intuitive	4.60	0.95	4.47	0.93	4.71	0.90	4.59	0.93
Manipulation check
Perceived effectiveness	4.83	2.01	5.42	1.91	5.97	1.91	5.42	1.99
Confounding check
Perceived sector attractiveness	4.26	1.47	5.24	1.25	5.57	1.37	5.04	1.47
Perceived innovativeness	2.93	1.43	3.48	1.45	3.52	1.43	3.32	1.46
Perceived trustworthiness	3.16	1.13	3.64	1.22	3.85	1.27	3.56	1.24
Perceived social impact	4.21	1.22	4.71	1.21	4.93	1.12	4.63	1.12
Previous knowledge of AEC-Foyer Lataste	1.01	0.11	1.13	0.53	1.22	0.86	1.13	0.60

Finally, to control for potential confounding factors, an assessment of differences in the sociodemographic characteristics and decision-making modes of all three experimental groups revealed no statistical differences. Thus, these variables appear in the further analyses as potential influences on charitable behavior. Moreover, the mean and standard deviation values for familiarity with AEC-Foyer Lataste were very low (M = 1.19, SD = .723). Considering the random order of the charitable behaviors, another check determined if there were any significant differences between the groups that first saw a solicitation for a donation versus those that were asked first to lend money. Because no significant differences arose, the following results combine the donation and lending behavior for both subgroups.

Results

Descriptive Statistics and Initial Analyses

To provide an initial understanding of the participants’ charitable behavior, Figure 3 highlights their donation and lending behavior across three experimental groups. For illustration, dummy variables were created for donation and lending behavior.

Figure 3.

Charitable behavior of participants across experimental treatments.

The graph for donation behavior implies differences among the three effectiveness indicators. In the output indicator scenario, 36% of the participants donate money; in the outcome and impact indicator scenarios, more than half of the participants donate. However, few differences mark the comparison of outcome and impact indicator groups. For lending behavior, the three experimental groups look similar: Less than 10% in each group make a loan, and there seem to be no notable differences across the groups. To confirm the descriptive findings, a series of chi-square tests apply to both donation and lending behavior. The chi-square tests for donation behavior confirm that effectiveness indicators and donation behaviors are not independent, χ²(12, 271) = 23.62, p < .05. Pairwise post hoc tests of each effectiveness indicator pair reveal significant differences between the output and impact experimental groups, χ²(6, 182) = 16.04, p < .05, and the output and outcome experimental groups, χ²(6, 175) = 13.39, p < .05, but not between the outcome and impact experimental groups, χ²(6, 185) = 7.010, p > .10. These findings provide initial, partial evidence in support of H1a but not of H1b. Similar analyses serve to test the independence of the effectiveness indicators and lending behavior. Without any significant differences across the three groups, χ²(8, 271) = 6.26, p > .10, and thus in the pairwise tests, there is no support for H2a or H2b.

Moderator Model

Binary logistic regressions assessed the moderating role of reflective decision-making, with donation and lending behavior as dummy variables. According to Baron and Kenny (1986), moderation exists if the interaction path between the effectiveness indicator and reflective decision-making is significant; the moderator variable also should be uncorrelated with the dependent and independent variables. Table 3 presents the regression results.

Table 3.

Logistic Regression Outputs for the Odds of Donating or Lending.

	Donation behavior			Lending behavior
Model	(1)	(2)	(3)	(4)	(5)	(6)
Outcome	1.967** (0.31)	1.019 (1.82)	0.993 (1.86)	0.580 (0.67)	0.122 (3.37)	0.186 (3.60)
Impact	1.878** (0.31)	1.264 (1.91)	0.840 (1.96)	0.915 (0.12)	2.357 (3.29)	1.377 (3.27)
Reflective decision-making	0.942 (0.14)	0.873 (0.27)	0.887 (0.27)	0.604** (0.255)	0.567 (0.50)	0.604 (0.48)
Interactions
Outcome × Reflective decision-making		1.135(0.35)	1.163(0.35)		1.383(0.68)	1.268(0.72)
Impact × Reflective decision-making		1.080(0.36)	1.163(0.237)		0.824(0.67)	0.886(0.86)
Controls
Previous knowledge of AEC-Foyer Lataste			1.234(0.23)			1.493(0.30)
Gender (male = 1)			0.586(0.26)*			0.897(0.59)
Age			0.992(0.03)			0.920(0.085)
Income			0.0877** (0.06)			0.841(0.174)
Business student			0.724(0.27)			0.417(0.631)
Observations	271	271	271	271	271	271
Pseudo R²	.02	.03	.09	.04	.05	.11
Model fit: χ²	6.066	0.135	13.302**	4.230	4.948	10.573

Note. Coefficients are odds ratios (ORs). Standard errors are in parentheses.

Significant at 10%. **Significant at 5%.

Donation behavior

Table 3 reveals that in the absence of reflective decision-making as a moderator, both outcome (odds ratio [OR] = 1.967; p < .05) and impact (OR = 1.878; p < .05) indicators have positive effects on donation behavior, but reflective decision-making does not (OR = 0.942; p > .10). The interaction effects on donation behavior are not significant, either between reflective decision-making and outcome indicators (OR = 1.135; p > .10) or between reflective decision-making and impact indicators (OR = 1.080; p > .10). The direct effects of outcome (OR = 1.019; p > .10) and impact (OR = 1.264; p > .10) indicators also are no longer significant. Among the control variables, only income seems to be a statistically significant predictor of donation behavior (OR = 0.887, p < .05).

Compared with the initial analyses that indicated a direct effect of outcome and impact indicators on donation behavior, the nonsignificant direct results of these indicators (columns 2 and 3) suggest that some other variables might influence the relationship of outcome and impact indicators with donation behavior. For example, the significant differences in perceived innovativeness, trustworthiness, sector attractiveness, and social impact across the three effectiveness indicators may imply that one of these factors mediates the relationship with donation behavior.

Investment behavior

Reflective decision-making exhibits a significant negative influence on investment behavior (OR = 0.604, p < .05; see column 4, Table 3). This finding already violates the prerequisite that the moderator and dependent variable are unrelated (Baron & Kenny, 1986). Thus, this moderation is not significant.

Discussion, Conclusion, and Limitations

This study examines the influences of different effectiveness indicators and reflective decision-making on charitable behavior, and thus contributes to the emerging body of research on the effects of voluntary disclosure and signaling on charitable behavior. With an experimental study, it offers first empirical insights of how three different effectiveness indicators can prompt two types of charitable behavior (Ebrahim & Rangan, 2014). The findings not only confirm some of the hypotheses and complement previous research, but also require interpretation and further research.

Implications

This study’s results show that outcome and impact indicators both influence donations more positively than output indicators. These findings contrast with Duncan’s (2004) notion of impact philanthropists and their primary interest in immediate effects. Instead, donors seem to care more about information on intermediate and long-term effects. Against the background of this study’s findings and the few, contradictory notions in prior literature, we recommend nonprofit scholars to further investigate differences across outcome and impact indicators, with focus on different campaigns and service contexts. The nonsignificant difference between outcome and impact indicators also suggests that nonprofit organizations, which often struggle to measure their impact, can embrace outcome indicators instead (e.g., Ebrahim & Rangan, 2014; Grieco et al., 2015; Mitchell & Berlan, 2016). Specifically, they should collect and provide information about how their programs help beneficiaries (even after project end), without the need to focus explicitly on overarching societal long-term effects. Nonprofit managers should then highlight such information in relating marketing campaigns, signaling the benefiting effects of the organization’s programs and services from an outcome perspective.

Another interesting finding relates to lending behavior. Contrary to previous research that highlights the importance of perceived social impacts for lending, this study indicates that different effectiveness indicators do not matter (e.g., Mittelman & Rojas-Méndez, 2013). In contexts in which people typically give more because they expect the money to be repaid, effectiveness and impact indicators should be considered in relation to the returns on investment (European Union & OECD, 2015). But as Costa and Pesci (2016) suggest, each stakeholder group has different information preferences when it comes to social impact measures. For lenders, the effectiveness indicators of the logic framework may not match their information needs. It follows that for nonprofit organizations providing lending options, it is not necessary to differentiate between different effectiveness indicators. However, against the specific context of charitable lending, and the lack of research in this relatively new field of giving, we advise nonprofit scholars to further investigate the lenders’ specific needs regarding the information provided. In this vein, charitable lending should be treated as a separate discipline by researchers, distinct from both donations and for-profit lending (Jenq et al., 2015). Further experimental studies are required to understand other mechanisms behind lending. That is, extant studies focus almost exclusively on existing lending data that cannot isolate or support manipulations of particular effects. Moreover, additional studies should also research and compare the effects of different social impact measurement approaches on the lending and donation likelihoods of donors among other influencing factors such as business sectors or culture.

Contrary to the hypotheses, reflective decision-making does not interact with any of the effectiveness indicators to influence donation or investment behavior. These findings indicate that the effective altruism movement, in its quest for more conscious decision-making, has not yet been embraced by most donors. Instead, donors seem convinced by the critique of effective altruism (e.g., Gabriel, 2017) that accuses the movement of neglecting especially vulnerable populations and featuring observational, quantification, and instrumental biases. Another potential explanation might stem from the general importance of emotions (Bekkers & Wiepking, 2011; Dickert & Sagara, 2011), sympathy in particular (Cryder et al., 2013), in donation decisions. Giving may be mostly a response that functions through an automatic decision-making process (Erlandsson, 2016; Kahnemann, 2011). In the context of these findings, nonprofit scholars should further investigate decision-making processes of donation and lending behavior, particularly in the light of increasing demands and skepticism among donors and nonprofit stakeholders.

Limitations and Further Research

This study is subject to several limitations that also provide avenues for future research. A key limitation relates to the specific nonprofit setting, that is, the use of a campaign nested in one particular organization. As part of recent developments in the field of nonprofit accountability and voluntary disclosure and the broader discussion on legitimacy and support of nonprofit organizations, we advise nonprofit scholars to replicate this experiment with different nonprofit organizations and across a variety of programs and campaigns. More specific and context-related information can help better understand changing demands for adequately designing quality management and marketing approaches in the nonprofit sector. Such approaches have the potential to address information asymmetries in the nonprofit sector and ultimately signal the organization’s effectiveness and success to the public.

Furthermore, the student sample may hinder the external validity of the findings. Despite the advantageousness of using student samples in experimental contexts or relating to recent trends in nonprofit sectors, other samples should be considered to avoid false conclusions drawn for the general public. Therefore, the study’s findings should be interpreted in the context of the specific sample population, and future research should extend this experiment to nonstudent sample groups. Particularly, this study reveals that donors seem to care more about information on intermediate and long-term effects. Given that the sample consists mostly of students, aged 25 years, and majoring in business and economics, existing literature provides evidence that such sample groups differently perceive effectiveness. Findings from Kraft and Singhapakdi (1991) reveal, for example, that students’ comprehension in terms of organizational effectiveness issues is different compared with those of managers such that they rate business ethics carried out by institutions lower, and social responsibility issues as more important than managers.

Moreover, the participants imagined a potential gift to distribute and, therefore, employed hypothetical money (Freeman et al., 2009), which may create a hypothetical bias. The participants did not know their odds of winning the voucher in the donation setting, so their expectations may have influenced their behavior. The lack of incentive in the lending setting might explain the low lending behavior across all three experimental groups. Further studies could test how the use of personal wealth affects the results and compare real and hypothetical behavior. In addition, participants self-rated their decision-making process, which may create a self-report bias (Donaldson & Grant-Vallone, 2002). More objective measures of decision-making such as experimental designs are helpful.

To fully understand the effect of outcome and impact indicators on donation behavior, additional analyses need to account for indirect influences. For example, several studies highlight the importance of perceived social impact (e.g., Cryder et al., 2013; Verkaik, 2016), whereas perceived sector attractiveness, innovativeness, or trustworthiness have not been considered in the debate. However, providing effectiveness information might help nonprofit organizations to not only increase the donors’ perceived social impact, but also enhance their perceptions of the importance of the focal nonprofit sector. From a methodological perspective, this aspect might also be interesting in the context of recent debates about manipulation checks (Fayant et al., 2017). That is, experimental manipulations potentially affect other variables providing additional plausible explanations. Further research clarifies that the indicators’ relationship with perceived social impact as well as other possible determinants as mediators is, therefore, considered worthwhile.

Finally, this study has been contextualized from a donors’ and lenders’ charitable giving perspective in Germany. However, extant literature on charitable giving questions the generalizability of results from a perspective of a single country. Indeed, the topic of individual giving is extremely disparate, given the wide variety of historic and cultural backgrounds, socioeconomic factors, state models, taxation rules as well as the wide variety of individual profiles and motivations. It follows that there is no one-size-fits-all portrait of the individual donor from a developed country, nor are there dominant models. Therefore, the results have to be considered within the German giving context, which is different from other European or U.S. American countries. Therefore, further research should consider different country contexts to draw a comprehensive picture of the questions outlined in this study.

Supplemental Material

20190720_Do_you_like_what_you_see_Final – Supplemental material for Do You Like What You See? How Nonprofit Campaigns With Output, Outcome, and Impact Effectiveness Indicators Influence Charitable Behavior

Supplemental material, 20190720_Do_you_like_what_you_see_Final for Do You Like What You See? How Nonprofit Campaigns With Output, Outcome, and Impact Effectiveness Indicators Influence Charitable Behavior by Jutta Bodem-Schrötgens and Annika Becker in Nonprofit and Voluntary Sector Quarterly

Footnotes

Acknowledgements

We are grateful to the graduate school of the School of Business, Economics and Social Sciences at the University of Hamburg for providing financial aid for the data collection of this study. Moreover, we would like to thank Silke Boenigk for supporting this study during our time at her Chair for Business Administration, especially Management of Public, Private and Nonprofit Organizations.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jutta Bodem-Schrötgens

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Jutta Bodem-Schrötgens obtained her PhD from the University of Hamburg, Germany, in 2017. Her research has focused on trends in the third sector, including social impact investing, social impact measurement, and the next generation of philanthropists. Currently, she is an engagement manager at McKinsey & Company, Inc.

Annika Becker obtained her PhD from the University of Hamburg, Germany, in 2016. She is a postdoctoral researcher at the University of Hamburg and a coordinator of an education program for refugees. Her current research projects focus on public perceptions of nonprofit organizations as well as services management challenges in a refugee context.

References

Aknin

L. B.

Dunn

E. W.

Whillans

A. V.

Grant

A. M.

Norton

M. I.

(2013). Making a difference matters: Impact unlocks the emotional benefits of prosocial spending. Journal of Economic Behavior & Organization, 88, 90-95.

Allison

T. H.

Davis

B. C.

Short

J. C.

Webb

J. W.

(2014). Crowdfunding in a prosocial microlending environment: Examining the role of intrinsic versus extrinsic cues. Entrepreneurship Theory and Practice, 39, 53-73.

Andersson

F. O.

Self

(2015). The social-entrepreneurship advantage: An experimental study of social entrepreneurship and perceptions of nonprofit effectiveness. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 26, 2718-2732.

Ariely

Bracha

Meier

(2009). Doing good or doing well? Image motivation and monetary incentives in behaving prosocially. American Economic Review, 99, 544-555.

Bagnoli

Megali

(2011). Measuring performance in social enterprises. Nonprofit and Voluntary Sector Quarterly, 40, 149-165.

Baron

Szymanska

(2010). Heuristics and biases in charity. In Oppenheimer

D. M.

Olivola

C. Y.

(Eds.), The science of giving: Experimental approach to the study of charity (pp. 215-235). New York, NY: Psychology Press.

Baron

R. M.

Kenny

D. A.

(1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistic considerations. Journal of Personality and Social Psychology, 51, 1173-1182.

Becker

(2018). An experimental study of voluntary nonprofit accountability and effects on public trust, reputation, perceived quality, and donation behavior. Nonprofit and Voluntary Sector Quarterly, 47, 562-582.

Bekkers

Wiepking

(2011). A literature review of empirical studies of philanthropy: Eight mechanisms that drive charitable giving. Nonprofit and Voluntary Sector Quarterly, 40, 924-973.

10.

Buchheit

Parsons

L. M.

(2006). An experimental investigation of accounting information’s influence on the individual giving process. Journal of Accounting and Public Policy, 25, 666-686.

11.

Carman

J. G.

(2010). The accountability movement. What’s wrong with this theory of change? Nonprofit and Voluntary Sector Quarterly, 39, 256-274.

12.

Carnochan

Samples

Myers

Austin

M. J.

(2014). Performance measurement challenges in nonprofit human service organizations. Nonprofit and Voluntary Sector Quarterly, 43, 1014-1032.

13.

Costa

Pesci

(2016). Social impact measurement: Why do stakeholders matter? Sustainability Accounting Management and Policy Journal, 7, 99-124.

14.

Cryder

C. E.

Loewenstein

Scheines

(2013). The donor is in the details. Organizational Behavior and Human Decision Processes, 120(1), 15-23.

15.

Dickert

Sagara

(2011). Affective motivations to help others: A two-stage model of donation decisions. Journal of Behavioral Decision-Making, 24, 361-376.

16.

Donaldson

S. I.

Grant-Vallone

E. J.

(2002). Understanding self-report bias in organizational behavior research. Journal of Business and Psychology, 17(2), 245-260.

17.

Duncan

(2004). A theory of impact philanthropy. Journal of Public Economics, 88, 2159-2180.

18.

Ebrahim

Rangan

(2014). What impact? A framework for measuring the scale and scope of social performance. California Management Review, 56, 118-141.

19.

Erlandsson

(2016). The when and why of helping: Individual and organizational decision-making from a psychological perspective. Stockholm, Sweden: Elanders Sverige.

20.

European Union & Organisation for Economic Co-operation and Development. (2015). Policy brief on social impact measurement for social enterprises. Luxembourg City, Luxembourg: Publications Office of the European Union.

21.

Fayant

M.-P.

Sigali

Lemonnier

Retsin

Alexopoulos

(2017). On the limitations of manipulation checks: An obstacle toward cumulative science. International Review of Social Psychology, 30, 125-130.

22.

Flake

J. K.

Pek

Hehman

(2018). Construct validation in social and personality research: Current practice and recommendations. Social Psychological & Personality Science, 8, 370-378. doi:10.1177/1948550617693063

23.

Freeman

Aquino

McFerran

(2009). Overcoming beneficiary race as an impediment to charitable donations: Social dominance orientation, the experience of moral evaluation, and donation behavior. Personality and Social Psychology Bulletin, 35, 75-84.

24.

Gabriel

(2017). Effective altruism and its critics. Journal of Applied Philosophy, 34, 457-473.

25.

Glazer

Konrad

K. A.

(1996). A signaling explanation for charity. The American Economic Review, 86, 1019-1028.

26.

Greiling

(2014). Qualität und Transparenz von NPOs: Pflichtübung oder Chance? [Quality and Transparency of NPOs: compulsory exercise or opportunity?] In Zimmer

A. E.

Simsa

(Eds.), Forschung zu Zivilgesellschaft, NPOs und engagement (pp. 231-244). Wiesbaden, Germany: Springer Fachmedien.

27.

Grieco

Michelini

Iasevoli

(2015). Measuring value creation in social enterprises: A cluster analysis of social impact assessment models. Nonprofit and Voluntary Sector Quarterly, 44, 1173-1193.

28.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Prentice Hall.

29.

Hayes

A. F.

(2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: The Guildford Press.

30.

Jenq

Pan

Theseira

(2015). Beauty, weight, and skin color in charitable giving. Journal of Economic Behavior & Organization, 119, 234-253.

31.

Jun

K.-N.

Shiau

(2012). How are we doing? A multiple constituency approach to civic association effectiveness. Nonprofit and Voluntary Sector Quarterly, 41, 632-655.

32.

Kahnemann

(2003). Maps of bounded rationality: Psychology for behavioral economics. American Economic Review, 93, 1449-1475.

33.

Kahnemann

(2011). Thinking, fast and slow. London, England: Penguin Books.

34.

Karlan

Wood

(2014). The effect of effectiveness: Donor response to aid effectiveness in a direct mail fundraising experiment (Economic Growth Center Discussion Paper, No. 1038). Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2421943

35.

Kiva . (Ed.). (2019). Kiva Impact. Retrieved from https://www.kiva.org/about/impact

36.

Koschate-Fischer

Schandelmeier

(2014). A guideline for designing experimental studies in marketing research and a critical discussion of selected problem areas. Journal of Business Economics, 84, 793-826.

37.

Kraft

K. L.

Singhapakdi

(1991). The role of ethics and social responsibility in achieving organizational effectiveness: Students versus managers. Journal of Business Ethics, 10, 679-686.

38.

Lecy

J. D.

Schmitz

H. P.

Swedlung

(2012). Non-governmental and not-for-profit organizational effectiveness: A modern synthesis. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 23, 434-457.

39.

Lee

Winterich

K. P.

Ross

W. T.

(2014). I’m moral but I won’t help you: The distinct roles of empathy and justice in donations. Journal of Consumer Research, 41, 678-696. doi:10.1086/677226

40.

Mason

(2012). Individual preferences over development projects: Evidence from microlending on Kiva. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 23, 1036-1055.

41.

Mitchell

G. E.

Berlan

(2016). Evaluation and evaluative rigor in the nonprofit sector. Nonprofit Management & Leadership, 27, 237-250. doi:10.1002/nml.21236

42.

Mittelman

Rojas-Méndez

J. I.

(2013). Exploring consumer’s needs and motivations in online social lending for development. Journal of Nonprofit & Public Sector Marketing, 25, 309-333.

43.

Moleskis

Alegre

Canela

M. A.

(2019). Crowdfunding entrepreneurial or humanitarian needs? The influence of signals and biases on decisions. Nonprofit and Voluntary Sector Quarterly, 48, 552-571.

44.

Parsons

L. M.

(2007). The impact of financial information and voluntary disclosures on contributions to not-for-profit organizations. Behavioral Research in Accounting, 19, 179-196.

45.

Phineo . (Ed.). (2017). Doing good—Achieving the best? Retrieved from http://www.phineo.org/english

46.

Pieters

Wedel

(2004). Attention capture and transfer in advertising: Brand, pictorial, and text-size effects. Journal of Marketing, 68(2), 36-50.

47.

Prakash

Gugerty

M. K.

(2010). Trust but verify? Voluntary regulation programs in the nonprofit sector. Regulation & Governance, 4(1), 22-47.

48.

Salamon

L. M.

(2014). New frontiers of philanthropy. New York, NY: Oxford University Press.

49.

Saxton

G. D.

Kuo

J. S.

Y. C.

(2012). The determinants of voluntary financial disclosure by nonprofit organizations. Nonprofit and Voluntary Sector Quarterly, 41, 1051-1071.

50.

Schrötgens

Boenigk

(2017). Social impact investment behavior in the nonprofit sector: First insights from an online survey experiment. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 28, 2658-2682.

51.

Schubert

Boenigk

(2019). The nonprofit starvation cycle: Empirical evidence from a German context. Nonprofit and Voluntary Sector Quarterly, 48, 467-491.

52.

Scott

S. G.

Bruce

R. A.

(1995). Decision-making style: The development and assessment of a new measure. Educational and Psychological Measurement, 55, 818-831.

53.

Singer

(2015). The most good you can do: How effective altruism is changing ideas about living ethically. London, England: Yale University Press.

54.

Small

D. A.

Loewenstein

Slovic

(2007). Sympathy and callousness: The impact of deliberative thought of donations to identifiable and statistical victims. Organizational Behavior and Human Decision Processes, 102, 143-153.

55.

Verkaik

(2016). Do donors really care about impact information? A dual process account. Retrieved from https://osf.io/paf2b/

56.

Willems

Waldner

C. J.

Dere

Y. I.

Matsuo

Högy

(2017). The role of formal third-party endorsements and informal self-proclaiming signals in nonprofit reputation building. Nonprofit and Voluntary Sector Quarterly, 46, 1092-1105.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.48 MB