Abstract
Content analysis is a powerful tool for investigating attitudes expressed in naturally occurring language data. It is a useful tool to help researchers develop an understanding of a specific research field through identifying how particular issues or topics have been conceptualized or where fieldwork can be limited or prohibitive. This is especially true for research on postconflict reconstruction, where large-scale quantitative surveying or metareviews of the literature can be prohibitive. The present study provides a case study of how a particular content analysis software program—Leximancer—was used to map factors associated with institutional legitimacy in postconflict societies. The case of Timor-Leste is used as an example. We examine texts at three levels of discourse: at the academic, official, and primary levels. Results indicate differing perspectives on legitimacy at each level of discourse. This article offers a snapshot of a potential method for understanding how particular topics are conceptualized within a specific research field and can thus help in the development of evaluation methods or data collection instruments.
Introduction
Although the legal and international legitimacy of peacebuilding missions is regarded as one of the most important assets of efforts to rebuild failed states (Barnet & Zurcher, 2009; Brinkerhoff, 2005; Chesterman, Ignatieff, & Thakur, 2005; Lemay-Hebert, 2009; Mersiades, 2005; Roberts, 2008; United Nations, 2008), there is a lack of research examining factors that influence domestic legitimacy among local populations. Domestic legitimacy refers to whether peacebuilding processes (here we include statebuilding as well) are accepted and supported by local groups. It incorporates beliefs and attitudes and has a behavioral outcome, in that it impacts on people’s willingness to cooperate with institutional authorities. Currently there is no defining methodology for assessing the attitudes of local groups and stakeholders toward reconstructed institutions and the agents responsible for rebuilding them. Evaluation methods may include population-based studies of public attitudes in postconflict societies, though achieving a reliable nationwide sample in a postwar country is logistically difficult. By the time results are collected and analyzed, the shifts in political circumstances may have rendered the surveyed public opinion irrelevant. Likewise, although in-country data collection such as ethnographic fieldwork or qualitative interviewing does provide opportunities to gauge local insights, they can suffer from small sample sizes and hence lack external validity.
The aim of this article is to test a potential alternative approach (i.e., content analysis) that could be used as a “complement” to other forms of data collection or as a stand-alone method. Content analysis examines the conceptual structure of existing opinions expressed in text-based sources of information. We recognize that such an approach has its limitations, particularly in postconflict contexts where low literacy rates, limited access to the Internet, and an embryonic print media mean local opinions can be underpresented in different text-based data sources. However it can provide a viable starting point for understanding how particular issues or concepts are understood or conceived, which then provides useful data to prepare fieldwork methods or undertake closer analysis.
To gain insight into the issue of the legitimacy of reconstructed institutions, not only are local attitudes measured via analysis of online forums and blogs but also other groups with varied involvement in peacebuilding processes are included in our analysis: that is, “officials” (including local government, United Nations, and nongovernmental organization [NGO] text sources) and “academics” (journal articles published by experts in the field). These three sources of information can provide differing perspectives on the legitimacy of state and international institutions involved in postconflict reconstruction and in the process, allow one to gain insight into the rationale for those opinions. Leximancer, a text analytics program, is used to facilitate content analysis, with it helping to visualize the meaning of content in texts by extracting concepts, and then graphically mapping relationships between concepts (the use of Leximancer for content analysis will be explained in the “Method” section). This study is exploratory in nature and we recognize that there are limitations in our approach. Our aim is to demonstrate a specific methodology and how it is relevant to studies on postconflict reconstruction and outline a replicable approach for achieving this.
Timor-Leste has been chosen as a case study for this method due to its history of violent conflict, institutional damage, and subsequent reconstruction. Indonesian forces invaded East Timor in 1975 and occupied the country for 25 years, and Indonesian rule throughout this time was characterized as oppressive, with the army silencing dissent. Regardless, opposition to the Indonesians increased in the 1980s, and in 1999, under strong local and international pressure, the Indonesian government announced it would hold a referendum on granting independence to East Timor. While the independence vote was overwhelmingly passed, Indonesian soldiers and prointegration militia embarked on a campaign of violence, resulting in the death of 1,400 Timorese, the destruction of basic infrastructure, and the displacement of approximately 300,000 people. One month later, an international mission called INTERFET (Inter-national Force for East Timor), led by Australian troops, was deployed to stabilize the country. This administration was followed by the establishment of UNTAET (United Nations Transitional Authority in East Timor), which oversaw all aspects of governance of Timor-Leste. Democratic elections were held in 2002 and UNTAET was disbanded. Although the security situation in Timor-Leste remains fragile (including unrest in 2006 that led to the deaths of nearly 40 people and an attempted assassination of President Jose Ramos-Horta in 2008), a recent UN Human Rights Report praised improvements, especially in reconstructed security and justice institutions (United Nations Integrated Mission in Timor-Leste [UNMIT], 2010). Nonetheless, research has drawn attention to the problems Timor-Leste has faced in establishing the legitimacy of its transitional authorities as well as the new democratic government that followed (Croissant & Schacter, 2008; Olssen, 2009; Saldanha, 2008). For example, the 2007 elections produced a majority vote for the FRETILIN (
Theoretical Framework
Effective content analysis relies on a theoretical framework through which to collate textual data and categorize the emergent ideas. As stated above, the focal point of the content analysis in this article is domestic legitimacy, that is, the extent to which Timorese perceive their authorities, institutions, and social arrangements to be appropriate, proper, and just (Brinkerhoff, 2005; Tyler, 2006), and how these perceptions compare with those expressed by officials and academics. M. Weber (1964) links legitimacy to a population’s willingness to comply with a system of rule, and identifies three sources of legitimate authority:
Studies of domestic legitimacy have provided varied insights into the micro-level factors that influence the success of peacebuilding and statebuilding programs “on the ground” (e.g., Harmer & Frith, 2009; Mersiades, 2005; Rubinstein, 2008). One factor contributing to perceptions of legitimacy is the notion of voice. Voice refers to the opportunity for groups to comment on or inform an authority’s decision, allowing them to have some level of input into processes that affect them (Folger, 1977). Voice provides for some level of local control and is premised on the concept of local accountability and participation in the reconstruction process. This is essential to ensuring that reconstruction efforts are perceived as being aimed at meeting local needs and avoids the problem of them simply being perceived as a vehicle for local elites and international or corporate interests (Chesterman, 2007; Harmer & Frith, 2009; Richmond & Franks, 2008). In short, voice builds trust within and across social groups, and is one of the biggest predictors of whether people perceive processes to be just and in their best interests (Folger, 1977). Hence, we have selected voice as one variable through which to study domestic legitimacy.
Past research also indicates other factors that have the potential to influence perceptions of legitimacy. For example, Rubinstein (2008) draws on a cultural perspective and argues that any explanatory framework aimed at conceptualizing local-oriented legitimacy needs to take account of the social stratifications among local populations that generate boundaries along cultural and ethnic lines. These stratifications make the engagement and inclusion of various local actors complicated and contested but is essential to achieving a stable peace. Social identity theory (Tajfel & Turner, 1979; Turner, 1991; see Hornsey, 2008, for a recent review) provides further insight, proposing that people are quick to categorize the world into in-groups and out-groups, based on religion, ethnicity, age, culture, socioeconomic status, and political beliefs. Furthermore, these group allegiances produce systematic biases in perception and behavior (e.g., Brewer, 1979; Cohen, 2003; Hewstone, Rubin, & Willis, 2002; Tajfel & Turner, 1979). One consequence of these biases is that group membership can act as a heuristic that tells people who can be trusted and who cannot, independent of any history of interpersonal exchange. It is assumed that in-group members can be trusted because they are motivated to protect the common good. Out-group members, however, are instinctively mistrusted because it is assumed that they are motivated to protect their own constituency at one’s own expense (Foddy, Platow, & Yamagishi, 2009; Hornsey & Imani, 2004). These biases could potentially have profound effects on the extent to which processes central to statebuilding are seen as legitimate. This is because citizens can draw assumptions about which groups are responsible for rebuilding institutions (i.e., who is “pulling the strings”). For example, a local population may find themselves speculating about who is “pulling the strings” when it comes to developing a constitution or a parliament: Is it the local populace, the United States, the United Nations, the European Union, spoilers allied to the government, big business, or a complicated combination of all of these? The answer to these questions (which can also be based on fact, myth, intuition, or conspiracy) would be a key determinant of whether people see rebuilt institutions as valid and legitimate. Hence, the second contextual factor that provides a focus for this study is social identity.
The variables of voice and social identity have been adopted for theoretical and methodological reasons. First, we wish to employ a flexible theoretical framework to guide the analysis of the qualitative data but still allow these data to speak for themselves by giving space to new emerging themes. Second, given the data comprise a large collection of text collected from the Internet, there was a substantial amount of material that was not relevant to our general research question. Our theoretical framework helped to filter out this irrelevant material. However, it needs to be stressed that the concepts and themes discovered within the data are emergent, revealed through the use of Leximancer, a text analytics software program. Although this software and the analytical method underpinning it (described in the next section) has been demonstrated to be a reliable and valid method of content analysis (e.g., see Smith & Humphreys, 2006), the interpretation of the results Leximancer produces still requires a theoretical lens through which to comprehend the various patterns revealed in our data sources. As new sources of data are added to Leximancer and different techniques of exploring the data are adopted, the content of concepts and themes revealed in the data can shift. A theoretical framework is essential to identifying generalizable results and interpreting instances that do not fit with consistent trends emerging from the data.
Method
Content Analysis
Content analysis is a research method that uses a set of procedures to draw valid inferences from texts (R. P. Weber, 1990). Such inferences can relate to the attitudinal and behavioral responses of organizations or people to particular issues (Krippendorff, 2004; R. P. Weber, 1990). Content analysis techniques determine the presence of words or concepts in collections of textual documents and can be used to break large amounts of materials into manageable categories (Leximancer Manual, 2010; Stockwell, Colomb, Smith, & Wiles, 2009; Krippendorff, 2004). It can be used to understand the conceptual structure of a set of documents and identify the most important occurring themes evident within text-based data relating to a particular domain (e.g., political, social, or economic).
Leximancer
Leximancer is a content analysis computer software that identifies core concepts within textual data (conceptual analysis) and clarifies the properties of these concepts and how they are related (relational analysis). Leximancer identifies what concepts exist in a set of texts, allowing concepts to be automatically coded. For example, Leximancer can take a set of 1,000 documents, containing millions of words, and produce a map containing the 50 most important concepts within that text, along with their relationships with each other. The overall advantage of Leximancer is that it analyzes textual data in a more reliable fashion and produces visual representations that aid interpretation (Smith, 2003; Smith & Humphreys, 2006; Travaglia, Westbrook, & Braithwaite, 2009).
Leximancer concepts and themes
A concept in Leximancer is a set of words that travel together (co-occur) in the text. Leximancer identifies relationships by the frequency with which words occur together in a block of text and the frequency with which they occur separately. This means that concepts themselves are emergent, rather than selected by the author. Concepts that co-occur often within the same two-sentence block attract one another strongly and are clustered in the visual map produced by Leximancer. Similar concepts tend to settle together in close proximity, which aids interpretation by the researcher. Leximancer “themes,” however, are clusters of concepts, denoted by circles on the maps (see figures provided). Concepts that are not close enough to any existing theme will begin their own theme. The size of themes can be controlled by the software user and set to the level of explanatory detail desired. Each theme is named after the most prominent concept in that group, which is also indicated by the largest dot in the theme cluster.
Data Sources for the Current Study
A variety of sources were used for analysis. A total of 1,653 documents were included in the analysis, distributed equally between each source. Postcollection, all literature was organized by source (see appendix for source lists). The purpose of this organization by source was to allow for subsequent comparisons from differing points of view. For this reason, the three categories of textual sources were deemed academic, official, and primary, to reflect the differing viewpoints of those observing, leading, and personally affected by statebuilding. This is not to say that these groups are mutually exclusive, nor entirely homogeneous: for example, UN officials may publish in academia, “local” sources may be politicized elites, and government and UN sources may have differing viewpoints within the same text. But for the purposes of analysis, these categories represent three important populations who have different levels of involvement in, and views of, statebuilding.
Academic literature was retrieved primarily from refereed political and social science journals (published between 2000 and 2010). Official literature comprised text produced by those responsible for delivering peacebuilding programs in Timor-Leste. This included UN official documentation, including resolutions, mandates, and progress reports. It also included material sourced from government agencies and NGOs, from specific project outlines to country assessments and official committee reports. Primary sources encompassed any text representing the viewpoint of locals. To retrieve this information, a variety of sources were used: online discussion forums, social media, personal blogs, local online media (including comments sections), and transcripts from investigative journalism programs in which locals were interviewed. Many of these sources included texts translated from Portuguese and Tetun into English. The aim of gathering primary text was to access the perceptions of citizens: their thoughts, beliefs, and attitudes to the conflict itself and subsequent reconstruction. Nonetheless, “local” data in this study does not reflect the entirety of Timorese society, given its reliance on Internet access and literacy levels.
Analysis Process
Analyzing all text sources in Leximancer created an initial exploratory figure. After perusing the resulting visual figure, concepts deemed to be irrelevant were removed. We excluded common function words (
From this point, it was possible to assemble a list of words that could be used as a starting point for Leximancer analysis, termed
Results
Academic Sources
The academic legitimacy profile figure indicated that different institutions requiring rebuilding after conflict were given well-balanced coverage in the academic literature when discussing perceptions of legitimacy (see Figure 1). The theme of “government” emerged as central on the visual figure, surrounded by overlapping themes encompassing other institutions: “health,” “economic,” “security,” and “police.”
East Timor academic legitimacy profile: Main themes
As the concept visibility in Leximancer was increased, the central theme that grouped concepts together relating to legitimacy was “government.” Among the top-ranking concepts that were identified as occurring with government were those connected to issues of voice, for example, “participation” and “include,” indicating voice as an important issue identified in the academic literature. Also appearing in the East Timor legitimacy profile were characteristics or properties of legitimate institutions, in accordance with academic theory and research: “functioning,” “appropriate,” “responsible,” “accountable,” and “democratic.” Closely related was a “reconstruction” theme, in which legitimacy was a central concept (see Figure 2).
East Timor academic legitimacy profile: “Reconstruction” theme
A query on the legitimacy concept revealed a negative co-occurring concept: “crisis.” Text retrieved from original text sources showed that in the perception of the academic community, the erosion of the legitimacy of UNTAET, particularly in its governance role, had created instability in East Timor in the period of 2000-2002. Below are some example quotes from the “legitimacy” and “crisis” query:
3./east timor academic/ By the early months of 2000, a growing crisis of legitimacy had begun to develop, catalyzing what appeared to be a radical reorientation in UNTAET’s approach to political authority and participation. On May 30, UNTAET announced its intention to move toward a period of “co-government” prior to the transfer of full authority to the East Timorese. 6./east timor academic/ The legitimacy crisis that resulted from UNTAET’s early paralysis confirms the observation by several analysts that the first weeks of a mission can be crucial to its ultimate success.
It is interesting to note that UNTAET is the main institution identified in the co-occurrence of legitimacy and crisis. This runs counter to previous research suggesting that the United Nations benefits from a level of legitimacy beyond that of unilateral and even multilateral foreign agents, with UNTAET enjoying high levels of support among Timorese people (Higashi, 2009). Given that UNTAET is the most ambitious transitional administration in the history of the United Nations, we anticipate polarized reactions, and these criticisms leveled by academics will also be examined in the context of the results from the official and local sources. An investigation of the notion of “local” voice and participation resulted in a “knowledge pathway” between “local” and “consultation.” A pathway is a Leximancer feature that describes a relationship between two concepts. A start concept is selected, followed by an end concept, and the links between them are illustrated. The relationships between these concepts are best thought of as correlations, with text segments describing the relationship between each concept in the pathway and the next (for further information, see Leximancer Manual, 2010). Figure 3 charts the links between each step as well as provides illustrative quotes.
Knowledge pathway between “local” and “consultation” concepts, academic sources
We can observe from this result that in the academic literature, voice is explicitly linked to notions of legitimacy, particularly in the context of civil society, capacity building, and good governance. Also in the top 10 concepts co-occurring with legitimacy are the concepts of “local” and “power.”
Summary of academic texts
From the academic legitimacy profiles, the notion of “voice” is considered an essential element of postconflict reconstruction and is regarded as tied to the legitimacy of agencies directing the reconstruction processes and the emerging institutions allocated responsibility for governance. Implicit in these academic conceptualizations are notions of active participation in the peacebuilding process, but a central question is whether such forms of active voice have been achieved to the satisfaction of local populations. This is essential to whether local groups will voluntarily cooperate and support the activities of agencies involved in peacebuilding missions and defer to their authority. Social identity processes are alluded to in these academic sources, though not explicitly linked with perceptions of foreign forces in Timor-Leste. These initial results will be further explored in the official and local texts.
Official Sources
In the legitimacy profile of official sources, the representation of institutions was less specific than in the academic literature (Figure 4). The main themes evident in this literature were “government” and “system,” which between them thematically grouped the majority of concepts to appear on this figure.
East Timor official legitimacy profile: Main themes
The “system” theme comprised the majority of concepts and called for more in-depth analysis (see Figure 5). As in the academic figures, it illustrated a variety of concepts associated with legitimacy that are particularly applicable to notions of voice (e.g., “access,” “consultation,” “cooperation”).
East Timor official legitimacy profile: “System” theme
Building participation among locals was stated as an aim of most United Nations, NGO, and government policy documents relating to statebuilding in East Timor. “Participation” was 1 of the top 10 concepts associated with “institutions” in the official literature, with a 16% likelihood that the 2 concepts would co-occur in the same text segment. Also the top 10 concepts co-occurring with “institutions” were related concepts of “involved” and “consultation”—reflecting different elements of voice. Figure 6 below illustrates the section of the map where these concepts were located.
Official map section containing concepts “institutions,” “participation,” “involved,” and “consultation”
Figure 6 reveals relationships between “voice” concepts in the official sources. Voice is referred to in the context of institutions, co-occurring with concepts such as involved, consultation, and participation. Also within this cluster is the concept “responsibility,” wherein official sources discuss empowerment through involving local institutions. For example, the following quote from a local NGO illustrates this point:
24./east timor official We hope that this time the government will put more efforts into making this consultation a true beginning for popular participation in Timor-Leste and not just use it to please international institutions.
Overall, results indicated that improving voice (i.e., participation) is identified as a key aim of reconstruction by the majority of government, United Nations, and NGO sources. However, the above quote illustrates an important problem with statebuilding: the perception that government attempts to increase voice are aimed at appeasing international stakeholders, rather than underpinned by a sincere and goodwilled attempt to actually encourage local participation. The official sources also contained assessments of attempts to enhance local participation through donor supported programs, for example, CEP—Community Empowerment and Local Governance Project—which is administered by the World Bank and the Asian Development Bank. One of the aims of CEP was to empower Timorese through the creation of village and subdistrict councils to make funding decisions supporting the local development of projects. However, in its attempt to empower locals, CEP may have had the opposite affect, as indicated through the following quote:
7 ./east timor official/ CEP’s attempt to bypass preexisting governance structures with a new Council structure was received ambiguously by communities and in many cases, generated local conflict that inhibited the project. Instead, the CEP model should have adopted the positive features of Timorese local authorities and challenged its regressive features through continuous dialogue.
Another key issue to emerge related to rebuilding legitimacy in the official literature was holding appropriate investigations into human rights violators as part of transitional justice processes (see Figure 7).
Official sources: “International” theme containing human rights–related concepts
Within the official literature there were a number of process-related issues identified as important to institutional legitimacy, one being the necessity for commissions or trials into human rights violators to meet international standards and principles of fairness in the eyes of the general population. The official figures revealed some doubts among NGOs and legal experts relating to the perceived legitimacy of 15./east timor official/ While various governments on friendly terms with Jakarta applauded the long-stalled beginning of the court, human rights advocates have been far more critical. There are too many loopholes that could prevent the effectiveness of the trial, such as the skill of the judges, the law, as well as the independency of judges from elements of power.
There is also some evidence that this perceived lack of legitimacy is related to Indonesian interests dominating the Commission:
19./east timor official/ The trials have failed fundamentally in fulfilling the “truth function,” a central part of the mandate of human rights and war crimes tribunals. Indeed, by clinging to a version of the violence in East Timor that is accepted nowhere outside of Indonesia (and rejected there by many), the trials have lost a unique opportunity to set the historical record straight, inform the Indonesian public of the accountability of their institutions for the gross human rights violations perpetrated in their name, and provide a basis for reconciliation.
This provides preliminary support for the social identity hypothesis, with the motives and outcomes of the Commission being innately distrusted and disputed by local Timorese due to suspicion over who is in control of the process.
Summary: Official sources
Concepts relating to voice (i.e., participation and consultation) were closely bound to notions of legitimacy in the official profiles. In some cases, government attempts to run community programs aimed at increasing voice were viewed as attempts to appease international stakeholders, rather than a genuine attempt to use local opinion to inform policy implementation. In this case, the concept of voice is tied to social identity processes, with locals distrusting institutions due to perceptions that they are influenced by outsiders and reflect foreign interests. This points to one of the key challenges peacebuilding missions face, in that while efforts might be made to provide opportunities for locals to have voice through processes of consultation, if those strategies do not translate into actual action that reflect the preferences of local populations—compared with donor agencies—then the process and the outcomes will be viewed with suspicion and loose domestic legitimacy. Official sources also emphasize the path to legitimacy through transitional justice and forgiveness mechanisms such as the
Primary Sources
The first figure resulting from the primary sources showed a contrast to the academic and official figures (Figure 8), with an emphasis on more humanitarian concerns: for example, “culture,” “rights,” “people,” “group,” “agree.”
East Timor primary legitimacy profile: Main themes
Results from the primary texts indicated that for the Timorese people, a sense of transitional justice was of great concern. Close inspection of the main themes in the primary literature showed that human rights considerations were central to perceptions of legitimacy and connected to almost all other issues in the profile.
When the theme size was decreased to reveal smaller clustered themes, variations on Xanana Gusmao’s name appeared on the primary figure within the people theme, very close to a “leadership” concept (see Figure 9). Xanana Gusmao is a central figure in Timorese politics, the previous leader of the armed resistance turned first head of state. Interestingly, his name did not appear on either the academic or official legitimacy profiles. His importance to the Timorese people may reflect a source of legitimacy that Max Weber (1964) calls “charismatic authority,” wherein legitimacy for a political party or movement is gained through the power of an individual’s personality. The following quotes emphasize the important legitimating role of Gusmao:
49./east timor primary/global voices online (4)~1.html/1/1_69 It’s fair also to hope that the national hero Xanana Gusmão makes the best use of his well known charisma in order to unite the country around its infant institutions. The whole nation has its eyes turned to him right now. 52. /east timor primary/human rights and post-conflict transitional justice in east timor~6.html/1/1_947 Gusmão’s personal charisma and moral authority increases his influence beyond the formal powers of the law.
East Timor primary legitimacy profile: “People” theme with Xanana Gusmao concept
This is in contrast to the official map, where no concepts relating to individual leaders appeared on the official map. While the concept “leaders” was found on the academic map, quotes indicated this concept to be mostly about the leaders of UNTAET.
Issues pertaining to voice also arose in the primary legitimacy profile. A query of the concept “including” across the primary texts yielded the following quote, again referring to a general theme of there being a gap in understanding between locals and official institutions:
24. /east timor primary/Timor ba nafatin~3.html/1/1_376 There is a visible lack of communication between most Western-type institutions, including those created locally, and a considerable part of the East Timorese society.
Another common idea relating to voice that emerged in the East Timor primary literature was the choosing of Portuguese as the national language. This was perceived in East Timor as provocative and elitist, given that the majority of Timorese spoke Tetun or Indonesian, with few speaking Portuguese. The concepts of “Portuguese” and “English” were queried together to tap text revolving around language. The quote below was especially interesting as it alluded directly to the marginalization suffered by people when officially denied voice:
10. global voices online (1)~1.html/1/1_89 The catastrophic decision to make Portuguese the national language of East Timor perfectly illustrates the dogmatism and unreality of Alkatiri’s approach. This decision disfranchised young East Timorese who speak Tetun, Indonesian or English.
Another theme that emerged was an East Timor theme in which an “Australian” concept was prominent (see Figure 10).
East Timor primary sources: “Australia” theme
Closer inspection of the text associated with this theme revealed that the dominant perception was one of suspicion toward Australia’s involvement in East Timor. The text data indicated that from the time of the launch of UNTAET, there was speculation that the Australian government was more interested in Timor’s natural gas and oil supplies than in a humanitarian mission.
1
The strength of these reactions arose from the perception that Australia was the driving force behind the formation of UNTAET, which effectively relieved Timorese people from the governance of their country. Due to the scope of UNTAET, there was evidence of specific concerns about Australia exerting control over East Timor:
12. /east timor primary/AR2006060201400_pf~1.html/1/1_40 Now the Australians have disembarked with Police, investigators, and magistrates. They will want to take over the Justice and then the Public Administration. Do not fool yourselves: there is a strategy behind that. They did exactly the same in the Solomon Islands. With the pretext of fighting the gangs, they pushed the Police against the military and were able to put in the Government whoever they wanted to.
As stated previously, a significant issue for locals was the concept of human rights and justice, as opposed to the academic figures where these themes were not as prominent. The lack of faith in the judicial system in East Timor was overwhelmingly reflected in the primary literature and was identified as a potential obstacle to institutional reform in East Timor. The following is a fraction of the opinions represented in the primary text regarding the judicial system in East Timor:
30. /east timor primary/human rights in east timor~5.html/1/1_755 Fear and mistrust of authority and lack of faith in the system to deliver justice are consequences of the long occupation.
A related topic to perceptions of justice was the issue of using the reconstructed judicial system as opposed to relying on traditional methods of justice. This issue was common across academic, official, and primary sources. The feeling of foreign forces changing the way a central issue such as justice is handled reflects social identity processes, that is, “our way” versus “your way.” At the same time, there exist inherent tensions in dealing with lawbreakers via traditional, nonjudicial justice mechanisms, as they potentially undermine the legitimacy of the “official” justice system and can create tensions between the operation of formal and informal justice processes. The following quotes are an example of opinions relating to this tension, taken from each text source:
Academic legitimacy profile:
4. /east timor academic/united nations transitional administration in east timor~2.html/1/1_346 Because the traditional resolution of gender-related crimes was so deeply etched into East Timorese legal culture, respondents considered police intervention in such matters to be disrespectful of traditional law and inconsistent with customary norms. In East Timorese culture only certain types of criminals go to the police.
Official legitimacy profile:
13. /east timor official/www.laohamutuk.org/Bulletin/2001/Oct/bulletinv2n6~1.html/1/1_80 As Amnesty International noted in a recent report on East Timor, the use of alternative, non-judicial criminal justice mechanisms can lead to serious human rights violations where they operate in an unregulated way without adequate protection.
Primary legitimacy profile
34. /east timor primary/human rights and post-conflict transitional justice in east timor~3.html/1/1_377 through traditional mechanisms of mediation, and this practice continues today, partly because the formal justice system still does not work properly, and partly because people have no confidence in the formal justice system. There is also little understanding of the legal process even among educated people in Dili, let alone people in the villages.
Tensions between traditional and formal systems of justice have been recognized more broadly with the peacebuilding literature (e.g., Baker & Scheye, 2007, 2009). The desire for a liberal democracy and functioning institutions while still maintaining a sense of traditional culture was an issue alluded to in the primary sources and was raised as a significant challenge for reconstruction efforts.
Summary: Primary sources
Primary sources emphasized human rights and the judiciary as the most important elements of obtaining legitimacy through peacebuilding. As with the official sources, there were some conflicting opinions about the use of traditional justice versus establishing a centralized justice system. Xanana Gusmao was singled out for his influential position as a leader and as a source of enhancing government legitimacy via charismatic authority. Notions of voice were discussed, particularly with regard to local disenfranchisement over the selection of Portuguese as an official language and were tied to social identity factors, as Portuguese is seen as an out-group language, spoken by former colonizers and by the elite. This link between voice and social identity was particularly strong in our primary sources. Further connections between legitimacy and social identity were found in discussions of UNTAET, where locals expressed disapproval of the intervention and whether foreign interests dominated the United Nations intervention.
Discussion
Implications for Research and Policy
The development of legitimacy profiles across academic, official, and primary texts related to peacebuilding in East Timor were characterized by a disjunction between the ideals of the academic literature, the aims of reconstruction programs, and the reality on the ground. Each text source and the development of legitimacy profiles represent a different form of aggregation as it pertains to varying stakeholder assessments of peacebuilding. Our concern was to explore how notions of domestic legitimacy were understood and conceived within the statebuilding and peacebuilding literature. A similar methodological approach could be used to explore other facets of postconflict peacebuilding.
Our results indicated that the academic literature discussed legitimacy with equal attention across a broad range of institutions, including economic, health, and security. The official literature, however, focused on government, and the primary literature focused on human rights and the judiciary. The presence of Timorese individuals such as Xanana Gusmao in primary profiles—but not academic or official—suggests a possible underestimation of the power of the individual to influence perceptions of legitimacy, via what can be termed charismatic authority. Though building participation and increasing voice is discussed in the academic literature and explicitly stated as an objective in the official literature, the primary literature shows that East Timorese still struggle with the impression of having little influence over the rebuilding of key institutions. This leads to perceptions of systemic bias and ineffectiveness, which thereby undermines domestic legitimacy. There is a strong indication that a lack of trust regarding Australia’s involvement in the reconstruction of East Timor impacted on the legitimacy of UNTAET.
In the case of East Timor, the text analysis indicated that suspicions abound of outsiders and their motives for involvement, at the stage of the international intervention and subsequently in reconstruction. This was exacerbated when other high-status groups were operating in the country in question. For example, locals in East Timor were generally accepting of UN involvement, though it was evident that this support was tempted by perceptions that foreign governments were “pulling the strings” of the operation. One can conclude that social identity processes are at work, arising from the innate distrust of the motives of particular out-groups as represented by international agencies and their representatives in East Timor.
It needs to be recognized that there are limitations with the methodology adopted in this project. For example, access to textual data was limited to English language sources. Although many of the sources (especially blogs and forums) included English translations of Portuguese and Tetun, sections of local opinions may not be represented in this analysis. Similarly, populations that do not have access to technologies that allow them to communicate their opinions to a wider audience will not have been included. In addition, secondary sources are the subject of filtering and editing processes of their authors and in the case of official literature reflect the bureaucratic and authorized stance of authorities. Hence, the accuracy of official reports and documents need to be kept in mind, given there can be variance between what is officially stated and what actually occurs in practice. We have attempted to correct for this potential bias by exploring key concepts across a variety of data sources, allowing for triangulation. It is important to note that we do not see this analysis as a definitive guide to study local processes in Timor-Leste. Rather the methodology should be seen as a way to explore predefined theories or concepts through rapid content analysis of a large literature.
The theory of domestic legitimacy as employed in this project and the intersecting variables of social identity and voice provide a useful basis by which to analyze available text sources. The text analysis allowed us to understand how these variables are linked and identity their key characteristics. Such results give insight into how the short- and long-term success of postconflict peacebuilding can be assessed. Our methodology indicates how agencies (whether international, governmental, or nongovernmental) can undertake site assessments through the use of available textual sources prior to engaging in interventions in the field. This can help identify local issues that may have a bearing on the way agencies are received and accepted by specific populations, and which may also complicate the implementation of policies and programs. For example, agencies can undertake similar country assessments when looking at entering unstable regions and could use a similar methodology to assess not just country stability but also specific issues that may affect their operations in particular regions. The United Nations and NGOs could perform similar assessments as undertaken in this article to help tailor development programs to specific community needs. Researchers can also undertake similar analysis of the type outlined to prepare for in-country fieldwork or in the design of data collection instruments. Rapidly reviewing texts through content analysis can help researchers systematically grasp an understanding of a certain research field, whose body of literature maybe beyond the capacity of researchers to remain updated on. Using computer-aided content analysis software to develop profiles on core concepts should be understood as an ongoing process with profiles updated when newly discovered and emerging text sources are accessed. This can help researchers or agencies to measure change over time, thus allowing for a more longitudinal assessment of postconflict peacebuilding.
Conclusion
Our methodology highlights how content analysis tools (i.e., Leximancer) can provide a means by which to assess important features of postconflict peacebuilding. When combined with a specific theory and methodology by which to categorize text sources, content analysis can allow one to undertake concise and efficient evaluation of texts within a particular research field. This is not to argue that assessments such as the one undertaken here are superior to “on the ground” fieldwork. It provides one methodology among many that could be adopted as part of a mixed-methods approach.
Footnotes
Appendix
Authors’ Note
The opinions express in this article are those of the authors.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This project was supported through a financial award provided by United States Air Force Office of Scientific Research (AFOSR) and the Unites States Asian Office of Aerospace Research and Development (AOARD)—Award Number FA2386-08-1-4094 and AOARD-124052.