Mobilizing Group Membership

Abstract

A randomized experiment was conducted to assess the effectiveness of three forms of e-mail appeals to prospective members of a newly formed professional group. The baseline condition consisted of an impersonal appeal; prospective members were sent a mass e-mail encouraging them to join. Participants in the personal condition received an e-mail with the same content, prefaced by a personal note from the group president. Participants in the social pressure condition received a personal note that called attention to the fact that they had previously signed a petition to form the professional group and urged them to make good on their earlier pledge (i.e., signing of the petition). Personalization is found to generate strong and statistically significant treatment effects. Even stronger are the effects of social pressure.

Groups form to bring together individuals who share common interests. Groups also often pursue agendas on behalf of their members. For example, an environmental group connects individuals interested in protecting nature and may lobby government for stronger environmental regulations. Likewise, a college’s alumni club unites those with a common experience who may attempt to raise money and recruit potential students. The success of any attempt to form a group often depends on mobilization efforts by the group’s founders. What makes for successful mobilization?

We build on the literature on group mobilization by exploring the impact of e-mail mobilization efforts. E-mail mobilization has become an increasingly important tactic given the ease with which e-mail reaches large numbers of potential group members. At the same time, efforts to encourage membership by e-mail must overcome a well-known problem: recipients of e-mail are often indifferent or hostile to mass e-mail communication, sometimes characterized as “spam.” Mass e-mails have proven ineffective, for example, as a means of encouraging college students to register to vote (Bennion & Nickerson, 2011; Nickerson, 2007a, 2007b). Several large-scale randomized experiments show that relatively small proportions of intended recipients open mass e-mail, and those who do are no more likely to register than recipients in the control who were omitted from the e-mail list.

This article addresses the challenge of crafting effective e-mail messages as part of a membership drive. Specifically, we study how incorporating personalized content and invoking social pressure affect the success of a membership drive. A randomized field experiment was conducted by a newly formed professional group. In what follows, we explain the theoretical backdrop of the interventions we test, showing how personalization and social pressure stimulate action in other contexts. We then describe the design and results of our experiment.

Personalization and Social Pressure as Mobilization Tactics

Using e-mail messages to mobilize participation has obvious benefits in terms of cost. The effectiveness of such efforts, however, is less clear—given the proliferation of e-mail solicitations, it is difficult to attract attention and make an appeal that culminates in action.

That said, certain types of e-mail messages may be more successful than others; we focus on two aspects of communication that may enhance their influence. First, personalized messages may prompt attention and be more persuasive. Tam and Ho (2005) speculate that

a recipient may form a more favorable attitude toward the message when a promotion e-mail is addressed to him/her personally (e.g., addressing the recipient using his/her first name) than when a generic message is received . . . the recipient may invoke a rule that “personalized recommendations are tailored for me and therefore can be trusted.” (p. 275)

This hypothesis has its intellectual origins in social exchange theory (Cook, 1987; Homans, 1958), which posits increased responsiveness when messages generate a feeling of trust and personal interaction. The theory posits that social behavior involves the exchange of material and nonmaterial goods where sources of approval and/or prestige come into play. Researchers have tested social exchange theory in the context of invitations to respond to surveys, and although the results are mixed, the general finding is that personalized invitations stimulate participation (e.g., Duncan, 1979; Fan & Yan, 2010; Porter & Whitcomb, 2003).

Personalization is thought to work especially well when the message is from a source known by the recipient. For example, although mass e-mail seems to do nothing to increase voter turnout, e-mail from one friend to another may be effective (Davenport, 2012). Personal interaction further stimulates trust, suggests the possibility for later social exchange, and may invoke social pressure to perform the desired behavior (Fan & Yan, 2010). ¹

The second message aspect we explore is explicit social pressure. When individuals decide whether to take an action on behalf of a public good, they may be swayed by the extrinsic benefits of their action. ² Extrinsic benefits include feelings of shame or pride in anticipation that others may learn about one’s actions. The expectation that a behavior will be monitored by others can induce someone to act: He or she may do so to obtain the social rewards of conforming to a norm or to avoid embarrassment by nonaction. Scholars have demonstrated the power of social pressure in stimulating a range of social, economic, and political behaviors (e.g., Bolsen, 2009; Cialdini & Goldstein, 2004; Gross, Schmidt, Keating, & Saks, 1974; Schultz, 1999; Sinclair, 2012).

For example, Gerber, Donald, and Larimer (2008) randomly assigned tens of thousands of potential voters in Michigan to one of five experimental groups. The control group received no turnout message. One treatment group received a message stressing that voting is a civic duty. A second, stronger treatment said that voting behavior among members of the recipient’s household was being studied for academic purposes. A still stronger treatment encouraged recipients to do their civic duty and showed that voting was being monitored by displaying whether members of the household had voted in two previous elections. The strongest treatment showed not only the household’s voting record but also the records of several neighbors, promising to inform the recipient and the neighbors about who turned out in the upcoming election. The results show nearly a 5 percentage-point increase in turnout among registered voters who were told that their behavior would be sent to their household and an 8 percentage-point increase among those told their neighbors would learn whether they voted. Telling registered voters that they were being studied, by contrast, produced much weaker effects. These results suggest that social pressure, and not merely being observed, has the greatest effect, a finding that has since been replicated by several follow-up experiments (Gerber et al., 2010, Mann, 2010; Panagopoulos, 2010, 2011; Sinclair, McConnell, & Green, 2012)

In sum, effective e-mail mobilization is far from guaranteed, but we expect the likelihood of success to increase with personalization and explicit social pressure. We further predict that the social pressure message will have a greater effect than personalization alone. In the next section, we describe a field experiment designed to test these hypotheses. To our knowledge, it is the first randomized experiment to assess the effectiveness of personalized and social pressure messages conveyed by e-mail. We want to be clear that while we have directional hypotheses, we generally were not entirely certain of the outcome, and our intent was to increase membership and add to the knowledge base of what mobilization tactics work.

Experiment

We conducted the experiment during a membership drive for a new section of a large professional organization. The organization, which counts more than 15,000 members, includes more than three dozen sections, each devoted to a subfield in the profession. Each section organizes panels at the organization’s annual meeting, distributes subfield awards, and engages in other activities aimed at bringing together members who share a common interest. To form a new section, a subgroup must file a petition that justifies the need for the section and includes the signatures of at least 200 current members of the larger organization. Once formed, a section must maintain a minimum of 250 dues-paying members to remain active.

The new section, which is the focus of our study, was formally recognized by the parent organization in 2010. Members of the parent organization can join the section at any time for a small additional fee. At the start of 2011, the section had 214 members and thus, to survive, needed to recruit at least 36 new members to reach the 250 minimum. Joining a section is nontrivial as indicated by the fact that the average member pays the extra dues to join 2.56 sections of 40 possible (at the time of our study).

As the inaugural and incoming presidents of the section, we launched an e-mail mobilization drive that focused on recruiting the 450 individuals who earlier had signed the petition to form the section. The practical question was how to structure the message so as to successfully recruit members. Although social pressure messages might be thought to work based on previous experiments, those experiments involved mailings from strangers with no prior relationship with the recipient. Here, the situation was different. The prospective targets were for the most part acquaintances of the e-mail sender. For that reason, the authors thought it possible that exerting social pressure could lead to negative reactions that might make undercut recipients’ propensity to join. Given the fact that good arguments could be made in favor of alternative messages, we decided to take an empirical approach and put these messages to a head-to-head test.

Specifically, we sent e-mail messages to the 280 individuals who had signed the petition but failed to join the section. The message began with the salutation “Dear Colleague,” and then provided details of section activities (e.g., mentoring, awards, and a newsletter) and explained how to join the section (see appendix for full version). In an effort to identify the most effective mobilization technique, we randomly assigned respondents to receive one of three messages, as follows.

• Impersonal (n = 93). This consisted of the basic “Dear Colleague” message.

• Personalized (n = 93). This included a short note prior to the “Dear Colleague” message that named the recipient:

Hi NAME,

I hope your 2011 is off to a good start. I’m writing to urge you to join the new XXX section. ³ Our organizing petition last year drew hundreds of signatures, and now we have to turn those signatures into paying members. I’m pasting a form letter below (sorry!) with some details about the section and instructions on how to join. Thanks for helping to launch the new section, and let me know if you have questions.

Social Pressure (n = 94). This included a short note prior to the Dear “Colleague” message that not only was personalized but also applied social pressure (as italicized below; the text was not italicized in the e-mails that were actually sent):

Hi NAME,

“I hope your 2011 is off to a good start. I’m writing to urge you to join the new XXX section. Our organizing petition last year drew hundreds of signatures, and now we have to turn those signatures into paying members. We are really grateful for your support last year, but the XXX Section list shows that you haven’t yet joined. Now’s the time! You do not have to wait to renew your XXX membership, and it costs only $8. I’m pasting a form letter below (sorry!) with some details about the section and instructions on how to join. Thanks for helping to launch the new section, and let me know if you have questions.”

Note that all three experimental groups were sent some type of appeal, which means that the treatment effects we estimate represent the effect of varying message content. All messages were sent from the personal account of the current section president between February 23, 2011 and March 22, 2011. To assess the results of our efforts, we checked the updated membership list maintained by the parent organization on March 25, 2011.

As expected, random assignment produced experimental groups that were similar in terms of their observable background attributes. Men comprised 68% of the impersonal group and 66% of the personal and social pressure groups. Prospective members from outside the United States comprised 10% of the impersonal group, 9% of the personal group, and 10% of the social pressure group. In addition, we measured the amount of contact the participant had had in the past year with the sender of the e-mail by creating three categories based on an archive of the sender’s e-mail: 0 e-mails, 1 e-mail, or more than 1 e-mail (which typically meant several). We found near-identical percentages across conditions with 52%, 52%, and 51% receiving 0 e-mails; 16%, 17%, and 17% receiving 1; and 32%, 31%, and 32% receiving more than 1. This last bit of evidence suggests that random assignment generated groups that were on average equally familiar with the sender of the e-mail.

To ensure all respondents ultimately received what we believed to be an influential appeal to join the section, we conducted a follow-up by sending an additional message to the impersonal message recipients who failed to join by March 25th. ⁴ These respondents were reassigned to randomly receive either the personalized (N = 43) or social pressure (N = 43) version. ⁵ Again, random assignment produced groups with similar observable attributes. Men comprised 68% of the personal group and 70% of the social pressure group. Prospective members from outside the United States comprised 11% of the personal group and 12% of the social pressure group. Finally, contact with the pressure group was 47% with 0 contact, 15% with 1 contact, and 38% with more than 1 while the personalized group, respectively, had 51%, 14%, and 35%.

We sent these follow-up e-mails between May 9, 2011 and May 17, 2011 and checked membership on May 31, 2011. During both rounds of e-mail, we also tracked whether respondents replied to the e-mail by sending e-mail back to the section president; nearly all responses included a promise to join the section.

Results

We report the results from the initial mailing in Table 1. The table suggests the powerful effects of personalization and social pressure. Among those receiving the generic, nonpersonalized e-mails, only about 5% joined the section, and just one person e-mailed a response in reply to the section president. Simply adding a personalized greeting and note boosted membership by 15 percentage-points, with more than 20% joining (and nearly 30% e-mailing back). A test of difference-in-proportions is highly significant, z = 3.06, p < .01. ⁶ A social pressure message had an even more dramatic impact with more than 30% joining, which is a significant increase over the control group and marginally significant when compared with 20% rate observed in the personalized condition (z = 1.63, p = .06).

Table 1.

Rates of Joining the Newly Formed Section during Phase 1, by Experimental Condition.

Condition	N	Joined	Percentage joined	e-Mailed	Percentage e-mailed
Impersonal	93	5	5.38	1	1.08
Personalized	93	19	20.43	27	29.03
Pressure	94	29	30.85	20	21.28
Total	280	53	18.93	48	17.14

Interestingly, compared with the personalized group, fewer (albeit not significantly fewer) social pressure respondents e-mailed back (21.3% vs. 29.0%, z = 1.22, p = .21 two-tailed). This pattern may stem from nonsocial pressure respondents’ desire to ensure their decision to join was appreciated by the sender, whereas in the social pressure condition they were told that the sender would know. ⁷ Overall, the results suggest that personalization, and even more so social pressure, are useful tactics in mobilizing group membership, even if the latter generates a bit less subsequent communication.

The second phase of the experiment recontacted individuals in the impersonal condition who had failed to join, and randomly sent them either a personalized or social pressure e-mail follow-up. Although participants in this study had received a previous e-mail, they were equivalent in this regard; the question is how they would respond to a follow-up e-mail. We present the results in Table 2. The results suggest that the social pressure message had a remarkably large effect, with nearly 42% joining the section upon receiving that e-mail. The personalized message also was somewhat successful with more than 16% joining, although this effect remains significantly smaller than the effect of the social pressure message (z = 2.61, p < .01). We suspect that the sequence of repeated e-mails heightened social pressure, thereby making it an even stronger treatment despite the fact that it was applied to individuals who were initially unresponsive to an impersonal appeal. In contrast to our first-round results, e-mail replies are no more likely to come from the personalized group; in fact, more were sent by the pressure group (z = 1.99, p < .05, two-sided test). This pattern may reflect the strong social pressure effect of a follow-up message and the individuals’ perceived need to implicitly apologize for not initially joining.

Table 2.

Rates of Joining the Newly Formed Section, by Experimental Condition,” to “Rates of Joining the Newly Formed Section during Phase 2, by Experimental Condition.

Condition	N	Joined	Percentage joined	e-Mailed	Percentage e-mailed
Personalized	43	7	16.28	4	9.30
Pressure	43	18	41.86	11	25.58
Total	86	25	29.07	15	17.44

Conclusion

The advent of e-mail has reduced the marginal cost of communicating with prospective members. Like most membership drives conducted by e-mail, ours led us to draft several potential messages, each of which was equally costly (or costless) to send. Our question as practitioners and social scientists is which of the messages was most effective in terms of encouraging petition-signers make good on their pledge of support. Uncertain about how to proceed with our mobilization drive, we conducted a rigorous empirical test of competing e-mail messages. Both waves of our experiment reveal the power of personalization and social pressure.

Unlike many previous studies of social pressure, ours applies this social–psychological force to people who had previously pledged their support. This feature of our study may explain why the treatment effects we observed are large even by the standards of this literature. As mentioned, we were not certain of what our results would show, and thus, we believe our findings add to an important knowledge base on how to build organizations. A fruitful line of future research would be to randomly vary whether individuals are initially invited to pledge their support for a cause or group. The question is whether social pressure is especially powerful among those who have been (randomly) encouraged to pledge their support. If so, the practical implication may be that it pays to invest substantial resources in an initial pledge drive, which provides the target list for a subsequent social pressure intervention that induces people to honor their pledge. Moreover, responses may vary across types of organization and senders. We note that the new section formed because of a new interest among potential members; another open question concerns the success of such tactics on older organizations.