Abstract
Promotion of public diplomacy (PD) policy has been growing worldwide. In the US, support rebounded after 9/11, as the government sought to curb anti-American sentiments. Educational exchanges relying on assumptions about social contact benefits became key to American PD policy. However, equally widespread efforts to objectively evaluate whether educational exchanges based on such assumptions serve governments’ PD goals are lacking. This study therefore examines a critical PD case, the Fulbright English Teaching Assistant program in Turkey, and offers a longitudinal look at one cohort of 94 ETAs and the nearly 2000 students with whom they regularly interacted. Using pre/post measurements of both groups’ intercultural sensitivity levels, it seeks to assess the effectiveness of the program from a PD perspective. The article concludes with suggestions for modified conceptual emphasis on contact theory based PD initiatives; and accordingly, suggests revisions to how educational exchanges like the ETA program are implemented.
Keywords
Introduction
Countries engage in a wide range of public diplomacy activities with the general aim of positively influencing foreign public opinion, which, in turn, is expected to help create support for more congenial and amenable government behaviors. 1 While public diplomacy efforts by the United States government slowed with the end of the Cold War, they saw a dramatic revitalization after 9/11, as the United States sought to improve its image abroad, particularly in the Middle East and Muslim majority countries. 2
Evidence of this renewed interest could be seen in the rebounding of budgets for public diplomacy, the creation of new training and scholarship programs, the establishing of focused research centers, and the launching of new academic journals devoted to the topic. While post 9-11 conflict drove practitioners to rediscover the tools of public diplomacy, geopolitical realities and technological changes also drove academics to reconsider the field’s dynamics and paradigms. The so-called ‘New Public Diplomacy’ 3 reflected changes in the methods, actors, and aims of public diplomacy activities and emphasized relationship building and mutual understanding. 4
The renewed interest could also be seen in the growth of the national educational exchange programs that make up a key component of US governmental public diplomacy efforts, in what has been alternately labeled as ‘citizen diplomacy’ 5 or ‘parapublic activity’. 6 The prestigious Fulbright program for example, which had been subject to declining budgets before 9/11, 7 received a nearly 30% increase in its budget, from US$95 million in 1999 to over US$122 million in 2001, another 30% rise in 2004, and continued increases yearly up until 2011. 8 In countries that were a particular target of post-9/11 US public diplomacy, the percentage growth was even more significant. Turkey, for example, saw the US portion of its annual budget for Fulbright go up 100% in 2004, another 50% in 2009, and 122% in 2012, making Turkey in 2024 one of the largest US contribution recipients among European countries.
Of the various programs falling under the Fulbright exchange umbrella, none have seen as significant a growth as the English Teaching Assistants (ETA) program, which arranges for recent American university graduates to spend a year employed as assistants to language instructors while also serving as ‘U.S. cultural ambassadors’ 9 in one of 53 foreign countries. From an average of approximately 35 ETAs being posted around the world each year in the 1990s, the number grew over 400% in 2003 to 178, and then rapidly increased each year after that. By 2013, more than 1000 ETA grants were being awarded annually, settling at around 1300 in recent years. Again, the growth has been particularly rapid in countries that could be considered as prime targets of post-9/11 public diplomacy efforts. Turkey, for example, which began receiving 1–3 ETAs annually in 1998, saw the numbers of incoming ETA grantees grow from 12 in 2007 to over 50 in 2010, and reaching over 100 by 2015.
Despite this massive expansion in program budgets and numbers, fundamental gaps remain in the field of public diplomacy. Primary among them are long-standing claims that public diplomacy lacks in grand theorizing 10 or that theorizing follows practice rather than guiding it. 11 Arguably related to this, questions about the effectiveness of public diplomacy practices and their assessment also abound. 12 In a political era in which the value of all soft power initiatives are being fundamentally questioned in various countries, particularly the United States, 13 and continued governmental support for educational exchanges is under threat, 14 effective measurements of these programs are crucial.
In this article we argue that a fundamental – yet largely unquestioned – construct underlying the improved relations and interactions these programs are argued to help build; a construct which may therefore provide a logical starting point for evaluating today’s public diplomacy, is social contact. We therefore offer a contact-based evaluation of a major educational exchange case, the ETA program in Turkey, via a longitudinal exploration of 94 ETAs and their approximately 2000 students. Using a pre/post measure of Intercultural Sensitivity to assess the effectiveness of the contact between the ETAs and their students, we aim to provide an example of how to evaluate a public diplomacy program based on conceptual foundations that reflect ‘new’ public diplomacy principles.
Public diplomacy and contact theory
Social contact theory refers to the assumption posited in the mid-20th century by Gordon Allport that positive, that is, friendly and cooperative, intergroup contact can reduce prejudice and improve race relations. 15 Allport cautioned that these positive effects would be facilitated if the participants were of relatively equal status; were acting in a cooperative, un-competitive environment with commonly shared goals; and if the contact occurred with institutional support.
Allport’s original hypothesis has been tested hundreds of times in studies around the world 16 and has gained complexity and depth, with additional factors being suggested as affecting the impact of contact on intergroup relations. Primary among these is Pettigrew’s reformulation of the theory, incorporating two main emphases: societal/institutional context and time. 17 For the latter he proposed a longitudinal model, arguing that for constructive contact to occur it must be related to ‘long-term close relationships’ rather than just ‘initial acquaintanceship’. Accordingly, he recommended a 5th facilitating factor be added – time for the contact situation to ‘provide the participants with the opportunity to become friends’. Subsequent studies testing this idea seemed to show that closer friendship-based contact is more effective at changing attitudes than less intimate forms of contact, 18 and emphasizing the importance of ‘extended contact’. 19
Other studies have gone on to explore the potential impacts of contact beyond prejudice, for example, in reducing intergroup anxiety 20 and improving intercultural competence and understanding, 21 leading to the conclusion that contact theory can be extended to situations beyond racial and ethnic encounters. 22 There have, however, also been counter-arguments to the overall idea that contact is positive. In the early 20th century, a belief that intergroup contact was more likely harmful than helpful was not uncommon, 23 and even in more recent years the theory’s principle has been questioned. 24 Post 9/11, it became common to cite cases like that of the political theorist Sayyid Qutb, who became radicalized in his anti-Western views while an exchange student in the United States. 25 Recent scholarship however, leans more toward acknowledging the complexities of contact, rather than dismissing the broader well-accepted fundamentals of its premise.
Social contact theory remains largely overlooked in the public diplomacy literature, with those few who refer to it decrying how infrequently it is discussed. Cuhadar and Dayton try to fill this gap by providing an overview of three seminal social psychology theories that have been used to design Track Two diplomatic solutions in conflict scenarios, one of which is social contact theory. 26 They emphasize the importance of doing so by arguing that failure to understand these theories leads to a theory/practice gap. Bettie also notes contact theory’s absence from public diplomacy research – ironic given that the ‘contact is good’ idea is a primary theoretical assumption behind public diplomacy. 27 She too warns of the risks of inadequately understanding underlying social psychology theories like these by reminding that failure to fully consider Allport’s four conditions for contact can lead to negative effects. Cowan and Arsenault refer to contact theory as key to dialogic public diplomacy tools – one of three layers along with monologic and collaborative. 28 They cite the works of philosophers like Buber, who wrote that dialogue involves reciprocal and multidirectional exchanges of information and ideas, in a relationship of mutual respect and willingness to listen. Cowan and Arsenault then gently criticize dialogue-based public diplomacy activities by presenting examples of leaders just ‘listening’, that is, engaging only in conversation rather than in the kind of collaborative activities that might serve as the basis for forming relationships. Ultimately, all three of these works provide powerful warnings that inadequate or surface-level understandings of the social psychology theories that underpin many public diplomacy practices, key among them being contact theory, will have adverse results.
The U.S. government’s post-9/11 renewed interest in public diplomacy has largely emphasized promotion of programs that are based on contact theory principles. The ETA program, for example, involves large numbers of individuals who, because they serve as teachers rather than researchers, are guaranteed to interact extensively with host country citizens during their grant period. The expected impact of programs like these hinges on a normative assumption that contact will be beneficial, as opposed to the ‘opinion leader’ model 29 that lies behind the core Fulbright programs for foreign citizens. The latter anticipates that the exceptional foreign awardees of Fulbright study and research grants will eventually become local political, educational, and societal leaders, and by returning home from their US grant period with a deeper appreciation of American values and people, will gradually influence their fellow citizens to feel similarly and ultimately push for more peaceful (pro-American) foreign policies overall. 30
In principle, the ETA program appears to meet Allport’s facilitating conditions for positive contact. While there are obvious hierarchical characteristics in any teacher-student context, the ETAs are by title ‘teaching assistants’, and generally come in with no teaching experience, so the teacher-student status gap is greatly reduced. The program also incorporates a social dimension of encouraging ETAs to create extracurricular opportunities for interaction and dialogue, participation in which increases the chance for friendship-building and further reduces classroom-based hierarchies. Moreover, in most countries the ETAs are employed in high schools or universities. Not only does this reduce possible age-related imbalances, it has the added public diplomacy benefit that the target local population is comprised of students in their mid-teens to mid-twenties – a crucial formative period during which, according to the impressionable years hypothesis, core political attitudes and beliefs are formed. 31 In addition, the language classroom is typically a cooperative environment in which everyone is working toward a clear common goal of language acquisition. Finally, institutional support for the overall program from the host schools, the Fulbright Program administrators, and the two funding governments, is predictably very high.
Contact theory may provide important insights into what makes for successful interactions and relationship-building, which are increasingly considered key to effective public diplomacy practices. Evaluation studies in public diplomacy however, have neither questioned the assumptions behind contact-based public diplomacy, nor have there been significant efforts to reliably measure its impact.
Evaluating public diplomacy impact
There is an abundance of criticism about how public diplomacy activities are evaluated. Overall it has been argued that such evaluation is inadequate 32 and that there is a lack of empirical data. 33 Studies have been criticized for being overly positivist 34 and for having hinged on ‘collecting data that proved an organization was meeting its objectives’ 35 rather than looking outward at the target groups/cultures.
Turning specifically to the evaluation of contact-based public diplomacy programs like educational exchanges, there are cautionary works about whether they actually lead to positive political benefits, 36 and assessments of the effectiveness and impact of these various efforts agree that measuring the connection remains problematic. 37 Perhaps because governments would like to see immediate results for money spent, methods of assessment have been likened by one scholar to asking a forester to measure the growth of their trees on a daily basis 38 and, given the complexity of operationalizing public diplomacy’s effects, to do so without the benefit of a ruler. The literature is even more lacking in terms of evaluating exchange programs specifically in terms of their public diplomacy impact.
True, there is a fairly large body of literature examining various student mobility programs as public diplomacy tools, 39 but attempts to systematically evaluate these programs from a public diplomacy perspective are rare. Some studies have explored the underlying public diplomacy assumption that exchange programs help change attitudes, with mixed results. Yun examined the attitude/behavior connection following educational exchanges, showing that positive changes in the former do not necessarily lead to the latter (a critical failure if governments are expecting concrete soft power impact), 40 and in another study, Yun raises further questions about attitudinal change with findings that even positive change is unlikely to be symmetrical between the participants of the two countries involved. 41 Ayhan et al. also explored such changes, and while they found exchange participants’ cognitive knowledge of the host country increased, affective attitudinal change results were much more mixed, with current students revealing no change of attitude, and alumni revealing only somewhat more positive attitudes – as compared with the feelings they recalled having prior to their exchange. 42
One study that particularly addresses the question of educational exchange programs’ effectiveness from a public diplomacy perspective is Wilson. 43 Using a comprehensive and carefully designed survey, he examines Erasmus exchange students’ social, political, and world values before and after their grants, and finds no net change in political attitudes, and a general failure to build up bonds with foreign nationals. 44
The largest assessment studies that exist are government-sponsored reports extolling the positive personal outcomes on the participants of well-known exchange programs. 45 These studies conclude there is tremendous ‘success’, with claims such as ‘the Visiting Fulbright Scholar Program is resoundingly meeting its legislative mandate of increasing mutual understanding between the people of the United States and the people of other countries . . . a multiplier effect that makes the Program such a powerful and important tool of public diplomacy’. 46 This assertion is made despite the fact that these studies only indirectly address public diplomacy-related issues by including a few questions about whether former grantees have continued collaborating with colleagues from their host countries, and responses to these questions tend to be mixed, for example, only 44% of US Fulbright scholars report even ‘moderate’ collaboration after their grant period, and virtually none of the Chevening grantees report maintaining any organizational ties with the host country after completing the program.
Reflecting similar methodological limitations, a 2014 U.S. State Department commissioned study of the ETA program aimed to assess three aspects of its ‘success’, including the program’s impact on ‘mutual understanding’, defined as considering how well the ETA program ‘fosters mutual understanding and strengthens relations between the US/US citizens and other countries/their citizens’. 47 While the positive results are quite overwhelming, it cannot be ignored that, like earlier cited studies, they are based on a survey of former grantees’ perceptions of their own impact. For example, the ETAs were asked such questions as how much they felt they had helped break stereotypes about US citizens (90% felt they had done so to a ‘great’ or ‘moderate’ extent), and how much they believed they had changed people’s impressions of the US and its citizens (83% said they had done so to a ‘great’ or ‘moderate’ extent). In response to even broader impact questions, more than 75% of the ETAs reported that they had helped their students and colleagues ‘expand their worldview’ and just over half (51%) thought they had helped them ‘think more critically about their own country’.
While these results led the report’s authors to conclude that the ETAs serve successfully as ‘cultural ambassadors’ who ‘deepen[] their students’ understanding of U.S. society and culture . . . break down stereotypes about the United States . . . [and] challenge people to rethink their perceptions of both the United States and their own country’, 48 they do little to truly respond to calls over the years for better evaluation of public diplomacy. 49 Surely, a more robust measurement of such claims would be helpful to a State Department hoping to justify continued funding to programs like these. Alternative assessment studies that go beyond self-reporting surveys could also go a long way to providing deeper insights into the potential effectiveness and impact in general of key public diplomacy activities like educational exchanges.
Evaluating contact in public diplomacy
One possible way of exploring the impact of contact-based public diplomacy initiatives is to assume a longitudinal approach, and to test for evidence of attitudinal changes among not only the ‘citizen diplomats’ but the target, host country citizens themselves. This study looked therefore at one of the largest ETA programs worldwide, in a country deemed critical for U.S. public diplomacy efforts, Turkey, and conducted measurements of both the ETAs and their students at the start and end of the academic school year. This design allowed us to not only examine possible correlations between various factors and their initial overall attitudes, but also to explore changes in attitudes toward and understanding of the ‘other’ following 8 months of regular contact.
To examine possible longer-term changes in attitudes without simply asking participants directly, the current study relies on measurements of their Intercultural Sensitivity, a concept often used together with Intercultural Understanding and with the capacity to navigate intercultural interactions, that is, Intercultural Communication Competence. According to Chen and Starosta, 50 the three are closely related but separate concepts, with Intercultural Communication Competence serving as an umbrella term comprised of the affective, cognitive, and behavioral abilities of interactants. The cognitive aspect of intercultural understanding refers to our actual knowledge of cultures – our own, others, and the similarities and differences among them, 51 and the behavioral aspect of intercultural communicative competence addresses our ability and adroitness to actually succeed in reaching our communication goals in any intercultural interaction. The affective aspect, which in a sense underlies the other two, is represented by the concept of Intercultural Sensitivity, which Chen and Starosta define as someone’s ‘active desire to motivate themselves to understand, appreciate, and accept differences among cultures’. 52 Intercultural Sensitivity can thus be viewed as a kind of precondition to the other two, as the ability to behave in interculturally appropriate ways is based on learning about cultural similarities and differences, and that process of achieving greater cognitive understanding is supported and enhanced by one’s intercultural sensitivity.
In order to create a valid measure of Intercultural Sensitivity, Chen and Starosta 53 attempted to isolate the affective aspect from the cognitive and behavioral. Their 24-item Likert based Intercultural Sensitivity Scale (ICS) explores five factors: Interaction engagement (e.g. ‘I often give positive responses to my culturally different counterpart during our interaction’); respect for cultural differences (e.g. ‘I can tell when I have upset my culturally distinct counterpart during our interaction’); interaction confidence (e.g. ‘I feel confident when interacting with people from different cultures’); interaction enjoyment (e.g. ‘I often get discouraged when I am with people with different cultures’ – reverse scored); and interaction attentiveness (e.g. ‘I am sensitive to my culturally distinct counterpart’s subtle meanings during our interaction’). Chen and Starosta found the ICS to have good internal consistency 54 (α = 0.89), an evaluation that has been echoed in a number of other studies conducted in different cultural contexts. 55
With growing interest in internationalization of education and consciousness of the importance of intercultural understanding, studies testing changes in Intercultural Sensitivity (IS) have increased in recent years, generally with mixed results. True, some research has shown higher IS scores among study abroad students compared with peers who did not go abroad, 56 and also higher IS scores among language learners engaging in online discussions and activities with intercultural participants rather than with just their peers. 57 These were, however, mostly small-scale studies without random assignment or pre-post measurement. Moreover, other studies have failed to show any quantitative gains following study-abroad programs. 58 Given our cautiousness about relying on self-reported data, Jackson’s study is particularly interesting as it provides a comparison between perceptions and actual measurements of IS, and reveals how most of the participants ‘significantly overestimated their level of intercultural sensitivity’. 59 The unfortunate impact, the author concludes, was an unawareness of how their behaviors and communication styles might have been impeding efforts to build up intercultural relationships.
Public diplomacy in the post-9/11 era regained a focal position in the US State Department practices, and contact-based public diplomacy efforts like educational exchange programs are increasingly seen as a critical means for meeting public diplomacy goals. In light of the above discussion, the question remains however, how can we more accurately qualify and quantify the results the programs achieve, and correlate input to outcomes both for predictive and operational purposes?
Methodology
Context and participants
This study offers a unique opportunity to examine social contact theory as a basis for public diplomacy efforts in a critical context. While Turks and Americans do not have a particular history of grievances, there has been an increase in negative attitudes about the U.S. government and policies in recent years. 60 Our data, however, were collected during a period of relative calm and positive relations (2014–2015). It is worth noting that this is the same year that the aforementioned U.S. State Department commissioned study was conducted with this same ETA cohort in Turkey and other countries. At that time the ETA program in Turkey was the largest in the world at the university level, making this a critical case. The participants consisted of the 104 ETA grantees to Turkey and the approximately 2000 students with whom they interacted. The ETAs were all recent US university graduates, representing a diverse group in terms of regional background (coming from 33 different states) and previous experience abroad. Nearly all their students (98%) were between 18 and 23 years old, and were enrolled in the intensive English language preparatory programs of 34 public universities across Turkey, all located outside the country’s largest and most cosmopolitan city of Istanbul. While there was some variation in the ETAs’ teaching loads, the ETA contract restricted them from more than 20 classroom hours per week, and most were teaching on average 18 hours. Because students had classes with other local (Turkish) teachers as well, their average classroom contact time with their school’s ETAs was approximately 8 hours per week, however additional contact between the ETAs and students took place through extracurricular activities facilitated and designed by the American grantees. Such activities are considered a recommended element of the ETA program, and, in the Turkish case, the grantees are asked to report on them at the end-of-year evaluation meeting. Examples of such activities range widely, from theater, sporting or cooking groups to podcast creation and volunteer projects.
The ETA program in Turkey was designed with a developmental mission, so the host schools were all recently founded public universities. While 4 of the 34 universities were located in or near Ankara and Izmir (cities of several million people), the rest were in smaller, less cosmopolitan cities. Unlike the country’s leading universities, these institutions were unlikely to have any foreign instructors on staff, and for those in some of the smaller Eastern cities in particular, the ETAs were likely to be some of the few foreigners living in the city and the first foreigners with whom many students had extensive, regular contact. Just 13% of the students in the study reported having ever traveled outside of Turkey. The ETAs on the other hand tended to be much more experienced with foreign travel, with nearly all (91%) reporting having previously visited or lived outside the US prior to this experience in Turkey.
Data collection and analysis
The American ETAs and their Turkish students were surveyed shortly after the start and again at the end of the academic year. To maximize participation, both surveys were completed by students during class hours. In lieu of consent forms, the students and the ETAs were informed both orally and in writing that completion of the survey was entirely optional and that choosing not to do so would in no way affect their grade in the class or their assessment as grantees. The results below reflect data from those American ETAs (94) and Turkish students (1938) who completed both the pre and post surveys. The pre-survey collected demographic data about the grantees/students, including education, past intercultural contact experience, and foreign language knowledge. Additionally, for the ETAs, it included questions about in-group identification, knowledge about/familiarity with Turkey/Turkish people/Turkish culture, and their experience so far settling in to their host cities. For the students, the pre-survey had questions about their previous experience with foreigners in general, and their knowledge of/familiarity with the United States. Both the pre and post-surveys also included Chen and Starosta’s 24-item Intercultural Sensitivity Scale. To prevent linguistic misunderstandings, for the students, a previously translated and validated Turkish version of the ICS was used. 61
For both the ETAs and their students we ran two main types of inquiries: First, we explored correlations between various factors and their initial ICS scores; second, we examined whether there were any changes in their ICS scores – both overall and its five sub-categories – between the start and end of the school year.
In terms of the factors considered in the correlations, the ETAs’ initial ICS scores – and the five sub-categories – were examined for possible relationships with four factors: familiarity with Turkey; proficiency in Turkish; the degree of their identification as an American; and a self-rating of how well they deemed their ‘settling in’ process was going. Overall ‘familiarity with Turkey’ was calculated based on their responses to five questions about whether they had had any previous visits to Turkey; read books by Turkish authors; listened to Turkish music; watched Turkish films; and had Turkish friends in the US. A language proficiency score was assigned based on their reported oral and written proficiency in Turkish (advanced, intermediate, beginner, none). The ‘Americanness’ score was calculated by adding up the Likert-scale answers to statements about being a patriot, being proud of being an American, supporting American foreign policies, and missing the US. Finally, ‘Settling in’ scores were based on their responses to a Likert scale question asking how easily they were finding the process of settling in to their host cities.
For the Turkish students, correlations were explored between their initial ICS scores and two factors: context and previous experience with foreigners. ‘Context’ was considered in terms of the location, size, and ranking of the university in which they were studying. Eleven host universities were in ‘large’ cities of 1 million or more people, thirteen in ‘medium’ cities of between 500,000 and 1 million people, and ten were in ‘small’ cities of fewer than 500,000. The universities themselves were also categorized according to student population size: 12 ‘large’ (over 40,000 students); 11 ‘medium’ (between 20,000 and 40,000 students); and 11 small (fewer than 20,000 students). In terms of ranking, the average scores for each school’s total admitted students that year on the Turkish central university entrance exam were obtained from the Turkish Higher Education Council, and then the schools were divided into 3 groups – 10 high, 13 medium, and 11 low. To determine ‘previous experience with foreigners’ an overall ‘score’ was given based on a calculation of their responses to questions about: (1) the frequency of their interactions with foreigners (face-to-face as well as online); (2) the number of times having traveled outside of Turkey (and for how long spent outside the country); and (3) whether they had ever had English instruction by a native English speaker.
Results
The American ETAs
Pre-post ICS
In terms of overall ICS, paired sample t-test analysis of the American grantees’ initial average scores of 101.5 and end-of-year average scores of 101.6, showed no significant change, though with a slight increase in standard deviation. This finding held true for both the overall ICS score (Figure 1) and for all five sub-categories.
ETAs’ pre-post ICS overall scores.
Optimistically, this can be considered an encouraging result, as earlier studies have shown declines in at least one aspect of the ICS – reduced respect for the local population after contact. 62 This did not happen among these 94 ETAs, despite their being in a wide variety of sometimes challenging contexts. One ETA who spent the year in the conservative Eastern city of Bayburt, reported how the new university itself was facing a reaction from the local population. When there was little snow that winter, some local residents blamed it on the female students’ ‘provocative’ dressing habits. Despite such circumstances, over the course of the school year these ETAs did not exhibit a decrease in levels of ICS, presumably attesting to their pre-existing high intercultural sensitivity. Most had previous exposure to foreign cultures, and clearly had the volition for additional contact as shown by their willingness to come to Turkey for this program. The ETA experience just gave them an opportunity to consolidate their readiness for this process.
In looking for measurable antecedents that might have impacted their initial ICS scores, we considered their attachment to their own national identity (homophily) to see whether that might be related to initial ICS. A Pearson’s product-moment correlation was run to assess the relationship between scores on the five sub-categories and ‘Americanness’ among ETAs. While most found no significant correlation, a statistically significant negative correlation was found with ‘respect for cultural differences’ (see Figure 2). Though weak, the correlation suggests that the higher the degree of ‘Americanness’, the lower the initial score on questions related to ‘respect for cultural differences’.
Respect for cultural differences and homophily among ETAs.
We also explored correlations between three factors – ICS, language proficiency, and familiarity with Turkey – and the ETAs’ reported ease of settling into their host communities, and found significant positive findings, both with overall ICS scores (Figure 3) and with specifically interaction confidence scores (Figure 4).
ICS and ETA acclimation.
Interaction confidence and ETA acclimation.
Neither of these findings can be considered surprising – one might expect grantees with higher ICS and, in particular, higher levels of interaction confidence, to experience an easier time when attempting to settle into a culturally unfamiliar context. What was surprising however, is that neither Turkish language proficiency nor overall familiarity with Turkey showed any significant correlation. While this appears to suggest that ICS levels may be a greater indicator of an individual’s ability to adapt to unfamiliar cultural contexts than even actual familiarity with that culture or local language proficiency, it may simply be that the established support networks in place, from host university contacts, prepared guide booklets, and contact with previous years’ grantees, were sufficient to smooth over major linguistic and cultural barriers during the early stages of settling in. Individuals’ personal readiness however, as measured by their ICS scores, played a role in easing the challenges that did remain, or at least perceptions of how difficult those challenges were.
The Turkish students
The Turkish students’ initial average ICS score of 91.8 was significantly lower overall than that of the ETAs (101.5). As noted above, the ETAs were all college graduates, mostly experienced travelers, and, given their participation in the Fulbright program, obviously open to the idea of travel and intercultural exchanges. Their Turkish students were all enrolled in the preparatory English programs of newly established public universities. As such, they were slightly younger and presumably less mature than the ETAs; only around 10% of them had ever traveled abroad, and very few had ever had regular contact with a foreigner. Although various correlations were explored between the students’ initial ICS scores and their previous experience with foreigners in general and their school context, none of the findings were statistically significant.
Pre/post ICS
A paired samples t-test was run to determine if there were differences in overall ICS scores among the students at the beginning and end of the school year. Although a decrease was noted (from 91.8 to 91.4), no statistically significant difference was found (Figure 5).
Students’ pre-post ICS scores.
Analysis of the five subgroups indicated that ICS scores declined significantly in four and increased significantly in only one, interaction confidence (Figure 6).
Students’ pre/post interaction confidence.
This significant increase suggests that regular contact with the ETAs may have contributed to these students’ comfort levels when interacting with people from other cultures and helped boost their confidence in such interactions. Given that the majority of the students had never had direct contact with foreigners before, this opportunity to meet regularly and become comfortable with the ETAs was a positive growth experience.
The remaining four sub-categories all saw statistically significant drops. These categories included Interaction Attentiveness, which assesses the efforts people make to understand what is happening in any intercultural interaction – basically, to what extent does someone actively pay attention to the cultural signals they are giving and are being given; Interaction Engagement, which measures emotional involvement during an individual’s participation in intercultural communication; and Interaction Enjoyment, which measures participants’ positive or negative reactions to the communicative process with people from different cultures. Interpreting these findings is complicated. While Chen and Starosta’s instrument clearly considers decreased engagement and awareness as indicators of lower intercultural sensitivity, one could also posit that they reflect the amount of work and effort required for interaction, and therefore a reduction in both might suggest less explicit effort and, therefore, greater comfort – in line with the students’ increased confidence. It is harder to explain the reduction in enjoyment and, perhaps most critically, the final sub-category of intercultural respect, which also saw a significant decline (Figure 7).
Students’ pre/post respect for cultural differences.
While all five factors combined make up the construct of intercultural sensitivity, one might argue that respect for other cultures’ differences underlies the other four in indicating an individual’s preparedness to reduce prejudicial behaviors and interact positively with the ‘other’. On this critical front, these students saw a significant drop in their respect for cultural differences during the extended period of regular contact with the American ETAs.
Discussion
Public diplomacy initiatives like Fulbright’s ETA Program are based on the principle of contact. ETAs are not expected to directly communicate particular ideas or messages about the U.S. or its policies. Rather, the anticipated positive impact is expected to emerge through the ETAs interacting with the host nation’s citizens. Given this, the results of the current study, which aimed to apply a more robust evaluation of the results that programs like this achieve, reveal gaps in both the practice and conceptualizing of public diplomacy. While the program has seen tremendous growth in the past 20 years and is clearly viewed as a valuable public diplomacy practice, it remains understudied in the public diplomacy literature. This oversight serves neither to practice nor theorizing as it increases the chances of both ineffective implementation in exchange practices and misdirected emphases on what the principles of public diplomacy should be based on and how it should be defined.
The findings of this study could broadly be judged as ‘mixed’. While some previous studies of educational exchange participants have shown respect for the host country population actually decreasing after contact, this did not happen among these 94 American ETAs – nor, though, did it increase. The lack of any change may be because the ETAs could be characterized as highly ‘culturally sensitive’ individuals, who were well poised for intercultural contact, and this experience simply gave them an opportunity to consolidate their already high intercultural competence.
Their Turkish students started out with significantly lower ICS scores. This is unsurprising, as the ETAs were college graduates and experienced travelers who were clearly open to the idea of travel/intercultural exchanges. A majority of the students were 18–20 year olds without foreign travel or contact experience. Disappointingly, their low ICS scores did not rise even after extended intercultural contact. Indeed, in critical aspects, such as enjoyment in interacting with and respect for people from other cultures, their scores saw a significant drop.
These findings showing a mixed impact from contact suggest the need to consider adjustments to contact theory and careful revisions to how educational exchanges like the ETA program are implemented. The following sections explore this study’s findings’ contributions to the discussion of ‘time’ in contact theory and how, therefore, public diplomacy theorizing may most effectively draw on it, and then move on to recommendations for public diplomacy practice.
Contact-based public diplomacy – conceptual underpinnings and application to public diplomacy theorizing
While the Turkish students’ overall ICS scores in this study did not change, and some aspects even fell, one aspect saw a significant increase – their confidence in engaging with people from other cultures. This is critically important because with confidence, the likelihood of further attempts at contact, and thus increased intercultural sensitivity, grows greater. Pettigrew and others have argued in the literature that the element of time is critical for producing positive impacts from intergroup contact. In this case we were not measuring reduced prejudice toward a single target group, nor were we studying two groups with specific historical grievances. Instead, we were looking at possible growth in more generalized intercultural sensitivity. Not unlike the extended contact that Vezzali and Stathi recommend, such change may more likely come about with repeated intercultural contacts.
Pettigrew cautions that initial contacts between different cultures may at best lead to ‘liking but without generalization’. 63 For most of the students in this study, this was their first time in physical contact with foreigners. 64 However, with the greater confidence gained following this period of contact, we may project that students’ subsequent contact experiences will not only be more likely, but will be ‘better’ in the sense of positively affecting other aspects of their ICS. The potential importance of confidence in ensuring effective contact has been noted. Turner and Cameron argue that confidence serves as a condition that could make people ‘contact ready’. 65 In other words, with increased confidence from this first encounter, we can speculate that the students will engage in subsequent contact encounters with enhanced readiness, which could enable them to build up overall greater ICS.
We are not the first to emphasize the significance of a factor that does not itself directly impact attitude, but rather helps create conditions that contribute to possible future contact, which in turn may have an impact. Husnu and Crisp focus on ‘imagined contact’, which they argue is an important preparatory measure that can instigate greater intention for future intergroup contact. 66 Interaction confidence may do the same.
The findings suggest therefore, that contact-based public diplomacy, both conceptually and in practice, should place greater emphasis on repetition – or what we might call ‘resilient contact’. Other features of international relations, such as strategic alliances, are always based on long-term repeated practices, through which consolidated institutionalized relations are built up. To draw an analogy from a hard power example, no governments would declare they were in a strategic alliance after engaging in a single joint military training, even if that training lasted for several months. Such understandings and subsequent trust – however imperfect they may be – can only develop with repeated interactions over time. Through the confidence gained in such repeated encounters and experiences, countries build up alliances, and individuals may develop their intercultural sensitivity.
Contact-based public diplomacy in practice – what should the U.S. be doing?
Public diplomacy practitioners certainly recognize the importance of and make efforts to create the conditions for such ‘resilient contact’. This is evidenced in the U.S. State Department’s extensive work to engage with program alumni through its Office of Alumni Affairs and their many efforts to create long-term networking/benefit structures for exchange participants. These efforts are, however, increasingly under direct attack, with recent cancellations to over 20 State Department sponsored educational and cultural exchange programs, 67 and the failure to follow up on genuinely long-term efforts like the Quadrennial Diplomacy and Development Review – last produced in 2015. 68
Moreover, current internal assessment practices may inadvertently work against ‘resilient contact’, such as the pressure for ‘quantity’. If contact is ‘good’, then more contact is generally deemed ‘better’. This phenomenon has been exacerbated by measurement practices like the Mission Activity Tracker, a central database into which foreign service officers regularly enter quantitative information about their activities, including numbers of people contacted, and which has been criticized as leaving no room for qualitative or original comments. 69 Because such measures are reference points for job success, their use promotes practitioners to seek to multiply their contact events with the target populations. Rather than focusing deeply on limited targets, practitioners are incentivized to increase their overall numbers. Lengthy and repeated engagements signifying resilient contact become discouraged – a practice that, according to our results, seems destined to produce negative results. Short, single contact events are unlikely to help build up confidence that such contact will continue – which is the basic bedrock of possible long-term public diplomacy success.
From this observation, some practical implications may be drawn. At the most fundamental level of policy planning, U.S. public diplomacy conceptually and in practice would benefit from more openly acknowledging certain inescapable realities that have been discussed in earlier works about public diplomacy evaluation surrounding the inherent tension between the need to achieve short-term deliverables with the idealistic aims of reaching long-term results. 70 Ultimately, the programs being evaluated have political goals, so even if described with friendly terms of ‘soft’ power or ‘public’ diplomacy, it is unreasonable to expect them to serve exclusively apolitical goals. We can, however, seek ways of shielding programs from the worst of political interference.
Definitions and conceptualizations of public diplomacy may highlight different features, but they tend to agree on at least two things: (1) the need to distinguish public diplomacy from traditional (‘behind closed doors’) diplomacy; and (2) that public diplomacy is a ‘good’ dialogical practice, unlike the more stigmatized one-way propaganda. 71 It is important, therefore, that official State Department definitions of public diplomacy openly emphasize these distinctions. Accordingly, they should seek a balance between public diplomacy’s short and long-term goals. If contact-based public diplomacy efforts like cultural and educational exchange programs are seen primarily as tools for short-term political goals rather than as long-term public-diplomacy investments, they cease to be public diplomacy efforts at all, and instead become examples of short-sighted opportunism – thereby risking the potential real benefits. For public diplomacy to live up to the positive label of dialogic practice, a starting point is to at minimum rhetorically include reference to its multilateral nature. Newly-set goals should identify and overtly recognize mutual benefits particularly in contact-based public diplomacy efforts, and all efforts should be taken to avoid having established programs like Fulbright become seen as being manipulated for unilateral U.S. objectives.
Assessment of what constitutes ‘success’ in public diplomacy should follow in line with this defining of public diplomacy and its goals. Pressure to find ways of measuring the success of public diplomacy programs is inevitable, as Congress, now more than ever, is demanding justification to continue funding. Evaluators should more transparently engage with the inherent dichotomies in stakeholder goals – from the international level (each involved country has its own expectations) to the domestic level (a Congressperson, a state department desk officer, and a foreign service officer abroad may all be on the same ‘team’, but hold different priorities and goals).
Current methods of assessment need to move beyond counting events or encounters, and find ways of instead valuing resilient contact – a system that promotes officers working on pre-established themes and relationships rather than constantly seeking new ones, and a system in which personnel rotations do not leave behind orphaned institutions and issues (which are arguably the most damaging of all to public diplomacy’s true success), but rather encourages incoming officers to focus their innovativeness on improving existing engagements.
Finally, assessment of existing programs like educational exchanges and thus of public diplomacy’s overall impact must also include studies conducted by external, non-government-affiliated entities or individuals, using robust, mixed methodological tools. Commissioned studies or those done by partner agencies to the State Department as well as studies that rely solely on self-reported data, or which collect data from only the ‘citizen diplomats’ and not the targeted populations, are all wide open to the risk of structural self-deception. While they may provide results that temporarily comfort everyone, real long-term success requires objective evaluation and, if necessary, appropriate adjustment in implementation practices.
Conclusion
In this study we conducted a measurement of Intercultural Sensitivity as a way of evaluating the impact of a contact-based public diplomacy initiative. We applied the measure to the young ‘citizen diplomats’ and the students with whom they interacted over one academic year. While the results showed shifts in some measurements of attitude, many were not in the anticipated positive direction. Critically, there was a decrease in various measures of the Turkish students’ sensitivity toward other cultures, including their enjoyment in engaging with people of other cultures and their respect for cultural differences. However, there was an equally critical increase in the students’ confidence in interacting with people from other cultures.
While we ultimately conclude that contact-based public diplomacy efforts are an effective way to achieve new public diplomacy goals of relationship building and mutual understanding, it is clear that our findings leave much to be explained. Broad studies similar to ours but conducted in different contexts are important to see whether the same results occur in countries that have different historical and political relations with the US, with populations of different ages and intercultural experiences, and with different examples of contact-based public diplomacy programs. Moreover, in depth, qualitative studies of contact-based public diplomacy programs are crucial to provide insights into the possible causes for when contact failure occurs. These may include deeper explorations into such factors as the characteristics of the involved participants and the conduct of the programs themselves. Critically, the time factor has to be explored further. This requires longitudinal studies to follow up with populations like the students in this study to see whether our projections that increased interaction confidence indeed conditions them for future interactions and results in more and better intercultural sensitivity and, ultimately, better public diplomacy outcomes. Only with additional studies like these can we build up a better understanding of whether, and if so, under what circumstances, current public diplomacy practices are actually helping to build greater mutual understanding between citizens of different nations, and can thus be considered effective.
Footnotes
Data availability statement
Due to Fulbright Commission policies we are unable to make the data publicly available at this time.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Ethical approval and informed consent statement
Participation was voluntary. Informed consent by those who opted to complete the survey was taken verbally. Further explanation in the article text.
