Abstract
We propose a framework for citizen social science that brings together three reinforcing elements of a research project – scale, citizen-leadership, and publicness – to improve qualitative research. Our framework was born out of necessity; a desire to involve ordinary citizens, in researching public issues, with limited funding. We illustrate the application of our framework using insights from research we have led, involving first, a series of qualitative studies of state and civil society organizations working on community engagement by three separate years of public policy students; and second, a qualitative study on the system for processing salary and injury disputes for low-waged migrant workers in Singapore conducted by over 100 volunteers and activists. Drawing on a review of the literature and our own experiences, we speak to the advantages and trade-offs of adopting this approach and suggest practical methods for conducting citizen social science.
Introduction
Our framework for scaled, citizen-led, and public qualitative research is a way of conducting public sociological interventions as a collective enterprise (Collins et al., 2017) involving large numbers of non-professional researchers in researching an important issue, and, through this, improving scientific knowledge and empowering non-professional researchers. We are motivated by both our personal experience doing qualitative research on public issues, and our normative orientation toward the public mission of universities to educate for citizenship and public service while inculcating the scientific norms of free and open debate (Calhoun, 2011).
Our framework builds on the growing body of work under the label of citizen social science, which, similar to the more established citizen science approach to natural sciences (see, for example, Bonney et al., 2009), prioritizes the ‘greater hands-on involvement of lay people in scientific research: doing it, designing it, understanding it and debating it’ (Richardson, 2016: 207). However, in contrast to the dominance of positivist epistemology in the natural sciences which is the basis for citizen science, citizen social science draws on hermeneutic and critical perspectives that challenge the objectivity of scientific knowledge and highlight the necessity of the human role in knowledge production (see, for example, Giddens, 1993). Citizen social science presents an opportunity to bring citizens into the knowledge production process, in service of democratizing and emancipatory ideals (Purdam, 2014). In this role, the researcher is encouraged to look beyond data analysis and scientific writing as their only contributions of value, and directly engage with the citizens who are both the research subjects and co-producers of knowledge (Bonhoure et al., 2019). Putting citizen social science into practice is challenging, and there are potential issues of data integrity, the ethics of engaging volunteers, and mobilizing a large research team (Heiss and Matthes, 2017). Citizen social science shares the twin objectives of the problem-solving sociology agenda, in which the research generates new scientific knowledge as well as contributing to the solution of the social problems in question (Prasad, 2018). As with utopian co-production, we seek to expand the ‘cramped space of the possible’ for academic research (Bell and Pahl, 2018). We present our framework of scaled, public, and citizen-led qualitative research as a practical response to the challenges of putting citizen social science into action.
Literature review
Scaled Research. We define scaled qualitative research as research that (1) involves a sizable research team and (2) conducts a substantial amount of interviews and/or other fieldwork. A scaled research team has the advantage of a greater capacity to gather and analyze data. We are oriented toward research teams engaged in shared data collection and analysis on a single project rather than producing a collection of individually authored studies linked by topic or approach (see, for example, Auyero, 2015; Burawoy et al., 2000). In traditional qualitative research, scaling up is done by either a lead researcher who finds funding for a team of graduate research assistants (Blee, 2012; Lareau, 2011; Newman, 1999), or a single researcher who finds time and funding for extended periods of fieldwork (Hoang, 2016; Rao, 2020). One exception to these traditional approaches to scaling is Grazian (2008), who used 811 firsthand narrative accounts by undergraduate students in his research.
Scaling has multiple potential benefits. Scaling can serve the traditional hallmarks of excellence in qualitative research – attentiveness to detail, clarity, or tightness of argument (Lamont and White, 2008) – by adding depth to the cases one is studying through more fieldwork by teams of researchers. Depth is distinguished from an attempt to legitimate qualitative research using quantitative logics of sample size; depth is produced by capturing ‘high-quality data’ and ‘trenchant data analysis’ (Lareau and Rao, 2016). This means detailed attention to social interaction by several sets of eyes and ears, and through fieldnotes, reflections, and memos which are subject to ongoing scrutiny by the research team. Scaling supports a sequential approach to qualitative research involving literal replication, in which like cases are examined, as well as theoretical replication, in which the same mechanisms of interest are observed in different circumstances, and allows the researcher to achieve saturation (i.e. when observations no longer yield new knowledge) while sampling for range across a greater variety of situations (Small, 2009). We use scale to enable us to make stronger logical claims about processes, and not to imitate quantitative logics of statistical inference. Scale also enables one to make a more compelling ontological claim that a certain phenomenon exists, which is especially important when the evidence is contested by other policy actors or when systematic sampling of the targeted population is not possible.
Conducting research with a large team does bring trade-offs compared to solo qualitative research. Most scaling of research requires significant investment in supervision in the form of recruitment of researchers, training them, establishing standards and protocols, checking the quality of data collection or analysis, and maintaining communication. In addition, the creation of such systems can generate its own inertia, meaning that flexibility is sacrificed as the scale increases. Small (2009) suggests that a scaled-up research team may risk resembling a survey, particularly if the lead researchers disengage from the field.
Citizen-led. We define citizen-led research as research in which non-professionals (1) fill the role of co-researchers, participating in the project design, data collection, analysis, and writing and (2) are empowered to use the research for their benefit. We note that our definition differs from that used in traditional participatory or action research, even as we draw on those rich literatures. In our definition, non-professional researchers need not be study subjects or community members, but can be students, volunteers, or any interested non-professional researchers. 1 Note also that our definition requires non-professionals to be engaged in design, analysis, and writing, and not only data collection.
Citizen-led research can provide low-cost scaling, through involving volunteer, non-professional researchers, in a similar way to citizen science in the natural sciences. In addition, citizen-led research can empower subaltern perspectives by incorporating them into the research process; improve research quality by having team members inform and crosscheck the design, analysis, and study conclusions (Lamont and White, 2008; Morrill et al., 2005); build a scientifically literate public; and further normative goals of democratic organization and self-actualization through participation as equals in the research team (Freire, 2000).
There are at least three main approaches to citizen-led research in the academic literature: traditional participatory research, action research, and citizen science.
In traditional participatory research, research subjects contribute beyond their typical role as passive respondents by being co-researchers, shaping both the design and research outcomes (Kral, 2014). In addition, benefit from the research is conferred onto study participants (Israel et al., 2012; Kral, 2014). This approach includes allowing marginalized groups to gain new competencies and have their voices heard (Russo, 2012); contribution of local knowledge by marginalized groups; improved investigation of context, meaning, and dynamism of social action (Bergold and Thomas, 2012); and a general increase in research effectiveness (Cornwall and Jewkes, 1995).
Action Research (Reason and Bradbury, 2007) emphasizes participation by the research subjects, or at least members of the community being studied, with an additional emphasis on social action and collective self-experimentation by subjects during the research process.
Citizen science is a method of enquiry associated with the natural sciences, and involves the public helping to collect and/or analyze large amounts of data (Bhattacharjee, 2005). Citizen science projects aim to educate citizens, who are non-professional researchers, on the subject matter or research methodology (or both) through their participation (Bonney et al., 2009). As with scaling, and similar to peer research (Kelly et al., 2020), the investment in training non-professionals for participatory research can be significant. In addition, participation involves ceding some control to a larger group or democratic process, and that takes time. Non-professional researcher involvement can, depending on their skill sets, experience, and supervision, lead to uneven quality of data or analysis.
Public Research. We define public research as research that (1) aims to promote dialogue, (2) among relevant non-academic publics, potentially including civil society, policy makers, and business, (3) often about values or goals that are not automatically shared and that thus are politically salient (Burawoy, 2005). Public research can intrinsically motivate professional and non-professional researchers through the potential to better society. Public research promises such betterment by focusing on issues that are relevant to society but understudied, or that involve marginalized groups without the power or social status to raise their concerns.
The idea of ‘publicness’, within sociology, was revived in Michael Burawoy’s (2005) presidential address to the American Sociological Association where he called for sociology to return to its initial purpose of ‘taking knowledge back to those from whom it came, making public issues out of private troubles, and thus regenerating sociology’s moral fiber’. Since then, there has been much debate about public sociology’s importance and its use in improving societal welfare (Brady, 2004; Clawson, 2007). Public research intends to create knowledge on topics that affect the public so as to empower the public to engage in deliberative discussions. Different ways in which we can evaluate the possible impacts of public research include audience engagements, feedback from research subjects, and policy outcomes (Collins et al., 2017).
Burawoy’s call for public sociology to reach a broader audience and improve people’s lives is one that many sociologists are sympathetic toward, but the lack of ‘concrete proposals for practice’ can confound the would-be public sociologist (Brady, 2004). Our framework is intended to contribute a viable model of practice which can serve the goals of public sociology.
Engaging in public rather than purely academic research can have potential trade-offs for the academic researcher. Public research often finds itself advocating for or against changes to existing public perceptions or legislation, and this is different to academic research objectives, such as creating generalizable knowledge. This can slow the academic publication process, and there is the danger that such public research just does not ‘count’ for appointment, tenure, promotions, or national and international research leagues tables, although there is a growing role for ‘impact’ in career assessments for United Kingdom and, potentially, Australian researchers.
As can be seen in Figure 1, the three elements – scale, citizen-leadership, and publicness – can have a reinforcing synergy. These three elements each have benefits and trade-offs and the combination of elements can enhance the benefits of the others while mitigating their trade-offs: (1) participation can help achieve scale through providing low-cost volunteer researchers; (2) public research can help achieve participation by motivating the efforts of volunteer researchers; and (3) scale educates participants on the issues, bringing more evidence to light, and thus promoting informed public discussion.
Self-reinforcing motivations of scaled, citizen-led, and public qualitative research.
Further examples of prominent studies are listed in Figure 2.
A Venn diagram of the different types of qualitative research, with examples.
Our experiences with research that is scaled, citizen-led, and public
This next section illustrates how all three elements of our framework – scaled, participatory, and public – were applied in two research studies we undertook: (1) an undergraduate course on participatory practices of community engagement among state and civil society organizations, and (2) a collaboration between university academics, students, and nongovernmental organization (NGO) volunteers to study the wages and injury disputes of low-waged migrant workers.
Our two research studies engaged with questions of democracy and the welfare of migrant workers, topics which lie at ‘the limits of open and consultative government’ in Singapore. 2 Our research took into account this political context by engaging in private consultations with government officials before, during, and after the research took place to navigate the associated risks. We did so to avoid the research effort being shut down, and positioned the research so that might have a policy impact.
Case study 1: public policy task forces on community engagement
One of us led three semester-long courses (Public Policy Task Forces) of 13–20 students between 2015–2017 with the dual pedagogical objectives of teaching qualitative research methods while studying community engagement in Singapore. The courses were explicitly presented to students as being participatory, with Fung and Wright’s (2003) empowered participatory governance as the touchstone for both the class’s organization and the analysis of the civil society organizations which were studied.
Motivation
Studying community engagement in Singapore appealed because of the contrast between the egalitarian and emancipatory ideals of bottom-up participation and the city-state’s technocratic approach to governance and open disavowal of liberal democracy (Chua, 2017). That state and civil society organizations so actively embraced participation in this context presented a compelling intellectual puzzle. The public issues in which a range of participatory practices were being employed centered on community engagement, civil society, and poverty, all of which were issues that Singaporean society and academics were engaged (see, for example, Loh et al., 2017) and thus attractive to students as well. The instructor’s goal was to connect to their new home through this fascinating topic, fulfill their teaching obligations, and take steps toward producing publishable research through the accumulation of the necessary data and by establishing their own familiarity with the field. Almost as an afterthought, the instructor decided to internalize the participatory practices we were studying in the student research team itself, and this decision proved vital to the research effort as well as for the pedagogy of the class.
Method
Scaled
Mobilizing a research team of 13–20 undergraduates so as to both train them on qualitative research and systematically collect and analyze data presented a logistical challenge. Students were organized into thematic as well as functional sub-teams with distinct responsibilities such as ensuring the consistent application of codes for our field notes, editing the final report, and preparing and delivering the final presentation. Every student was required to do weekly fieldwork and to produce three analytic memos during the semester. The class met for a single 3-hour session each week, during which the students worked in their sub-teams with the instructor (and occasionally editors) circulating among them. Periodically, a sub-team would present on its progress to the entire group. Fieldwork was conducted in teams of 3–5 who jointly produced 2000 words or more of field notes (within 24 hours) per field visit. In addition, students were required to produce ‘notes on notes’ for every field exercise in which they documented their reflections, concerns, or persistent questions. The instructor participated actively in the fieldwork and contributed to the field notes accordingly. Meeting in preparation for fieldwork was usually an opportunity to engage with students informally and get them to share their opinions and observations. We used a cloud-based qualitative research software (Dedoose) which facilitated multiple students accessing the same documents, reviewing coding choices, and contributing to our analysis. Our university supported the course with $2000–4500 per semester for student transport, qualitative research software (Dedoose, 2018), and catering for the final presentation event. We also benefited from an ‘Active Learning Classroom’ designed to facilitate groupwork and student-centered learning.
Citizen-led
The task force was citizen-led in that it internalized the norms of participatory democracy within a research team of undergraduate non-professional researchers. Students were informed of this in the first week of the class when we read Fung and Wright (2003), and we discussed what it meant to internalize the empowered participatory governance framework throughout the semester. This reading provided a vital normative reference for the entire structure of the class, as well as a key framework for analyzing community organizations. Staff members of the client organizations contributed to our deliberations throughout the semester. They, and the community members where we did our fieldwork, participated in the closed-door discussion which followed the presentation of our report.
Taking a citizen-led approach required a lot of preparation, constant readjustment, and imposed major constraints on any ability to direct the project in the mode of a traditional Principal Investigator. For example, prior to the semester, the instructor had to establish the initial link with target organizations and organize Institutional Review Board (IRB) approval, while leaving scope for substantial student innovation and also protecting the interests of participants. Student researchers – as a group and as individuals – had substantial leeway to follow their own interests under the broad rubric of studying ‘community engagement’, subject to the team’s ability to gain the necessary access to interviewees and field sites. The semi-structured interview guide also gave students a lot of leeway to tailor their questions to the situation at hand.
Public
Singapore’s peculiar political context was a persistent constraint on the level of publicness that was possible during this research. The lack of protection for civil liberties and the concentration of power in the state (Tan, 2016) limited the willingness of the research team or study participants (including public servants) to make public comments. We did not disseminate the findings of our study publicly (other than in the final presentation). We avoided media coverage of the study and avoided identifying our field sites. In fact, the access we had to the field sites and interviewees was often obtained in exchange for our agreement to keep a low public profile (over and above the usual anonymity).
In response to the situational constraints on publicness, students presented the research at a closed-door event under the Chatham House rule for an audience including our partners, civil society organizations we met and studied, academics, and assorted government organizations with responsibilities for social services and community engagement. Both the research team and the final presentation audience operated as mini-publics 3 (Dryzek, 2012); they were small groups who engaged in deliberative discussion about contested political and ethical issues. The strengths of this approach were evident when many more civil society and government officials attended the final forum than would participate in the study, although very few even asked any questions. For us, this pointed to our research reaching the limits of publicness in the context we found ourselves.
Outcomes
Scaled
The aggregate data collection – a total of 157 field visits and interviews across three semesters – was well beyond the capacity of the academic alone. However, to serve the learning of the students, the three research teams were kept insulated from each other. Consequently, the fieldwork provided more breadth of coverage than depth. The instructor benefited from the breadth of fieldwork, but also supervised the fieldwork quite closely in order to be able to make connections across visits to related sites. The students did produce highly detailed fieldnotes by reconciling multiple perspectives on a single set of social interactions, thus capturing details that one person might miss – or mis-recognize (Fine, 1993). The fact that the students were themselves locals brought insider knowledge that the instructor lacked, ranging from fluency in Singlish and other local languages to an intimate familiarity with Singapore’s Housing Development Board apartment buildings. As a researcher, this sustained and direct engagement with the fieldwork, with the students almost acting as an additional layer of informants, was a powerful foundation for academic study. Two article manuscripts have been written based on this fieldwork.
Citizen-led
The students were exposed to a high degree of uncertainty in this course, and they addressed it through actions based on reasoned deliberation. One set of students was forced to make major editorial decisions for their final report without the instructor being present to manage or guide the process. In addition, the instructor had to leave space for students to make decisions that the instructor did not agree with. Students came to understand citizen-leadership through their own empowered participation.
Public
Students had to engage extensively with the city they live in, the challenges facing civil society, and the concerns of disadvantaged communities. For some, this meant engaging with issues of deep personal concern, particularly for those with firsthand experience of poverty. For others, the personal encounter with poverty was eye-opening. The relationship of civil society organizations with the state was thoroughly debated. In their end of term course evaluations, 33 of 36 student respondents agreed or strongly agreed (the averages were 4.56 and 4.35 on a 5-point Likert-type scale for the two times this was asked) that they ‘feel better prepared for a lifelong commitment to civic engagement and social responsibility in Singapore’ due to this course.
Beyond the numbers, the transformative nature of the course for both the students and instructor was a palpable feature. We could not get all the access we wanted as some government offices with public-facing responsibility for social services refused to speak to us. Bringing original evidence to a classroom discussion of the state’s lack of transparency and its controls over civil society was a novel and sometimes discomfiting experience for the students. In the first task force, the students received feedback on their report from a public servant who objected to the use of a particular phrase. The instructor left the choice of wording to the students, and they voted to change it after a feisty exchange among themselves. The students made the choice to self-censor in the face of power, but stripping all pretense from the decision was a learning moment. Without a public record of its findings, however, the task force leaves a sense of unfulfilled potential. Future iterations of the task force could address this by building in public components such as blog posts, a poster exhibition, or an open-door final event into the course.
Comparable study
We look at Teo You Yenn’s research, published in her two books, This Is What Inequality Looks Like (2018) and Neoliberal Morality in Singapore (2011), as comparable studies. Teo’s ethnographic work collected data on the everyday lives of Singaporeans through in-depth interviews and used them to understand Singapore society and its inherent social structures. In her books, she describes how the data collection process took place over slightly more than a decade since 2003 and draws on interviews with over 200 and 60 respondents for each of her books, respectively. Teo’s earlier work made academic contributions, while the 2018 book was immensely successful in generating public discussion on inequality. Media coverage of her work was wide, and her research was quoted in a parliamentary discussion on income inequality (Jagdish, 2018; J. Seow, 2018). Teo achieved scale in data collection through the classic ethnographer’s path of long-term, continual individual effort. While Teo’s work has benefited enormously from her approach, one could also imagine a similar project which uses our framework (scaled, citizen-led, public), completed in a shorter period of time, mobilizing and educating ordinary citizens, and providing a pool of engaged citizens upon the public release of the study. Rather than doing so through the course of the research, Teo built an engaged community through the sales of her popular press book, op-ed pieces, and multitude of public engagements on the topic of inequality, thereby serving as an exemplar for public sociology in Singapore (Teo, 2021).
Case study 2: the Labour Court Research Project
Two of the authors led a study on low-waged migrant workers in Singapore, the Labour Court Research Project (LCRP), in 2016–2017. The study evaluated the system for processing salary and injury disputes for low-waged migrant workers in Singapore. The research aimed at policy reform through respectful engagement with the government and raising public awareness.
Motivation
The coalition of academics, students, and NGOs were drawn together by a common concern about the failures of an injury and salary claim system for migrant workers, and apparent unwillingness by authorities to acknowledge or address these problems. Our decision to scale the research was largely done for practical reasons. Experienced advocates advised that a small qualitative study was likely to be ignored by policy makers and the media. To be able to collect a large dataset, we made the study citizen-led involving more than 100 academics, students, and NGO staff and volunteers. Practically, collecting and coding 150 qualitative interviews and writing a report over a period of 6 months would not have been possible without a large volunteer research team.
Method
Scaled
We had a large research team and conducted a large number of interviews. The research team numbered more than 100 volunteers, including 7 co-authors, around 50 interviewers, approximately 10–20 interpreters, 15 qualitative analysis coders, and several dozen experts from NGOs, medical and legal professions, and universities who reviewed and commented on drafts of the report.
An important practical consequence of this scale was that we needed to organize the conceptualization, design, data collection, analysis, and writing through formal and relatively inflexible structures. In the case of data collection and coding, we developed a multi-layered management structure, with supervisors, project managers, shift supervisors, and interviewers/coders. Thus, the scaling imposed a considerable managerial burden in the form of investment in supervision and establishing consensus-oriented procedures to manage the many participants. We found this to be an unavoidable cost of scaling, though it did become easier with practice.
Citizen-led
Our research was citizen-led through the involvement of non-professionals as co-researchers. Few team members had any graduate research training. These non-professional researchers came from a wide variety of backgrounds – including NGO staff and volunteers, undergraduate and graduate students, medical practitioners, and lawyers – and played a role in all stages of the project, including conceptualization, design, ethics approval, data collection, data analysis, writing, and dissemination.
Our meetings and discussions were conducted in a classroom-like environment and in a democratic fashion. Anyone, including those with or without relevant experience, had a chance to share their inputs. This process added value, particularly to the research design and outcomes, through the involvement of people with diverse backgrounds. For example, many of our volunteers had direct experience of the long process injured migrant workers face. Firsthand knowledge of these processes and the workers involved meant that our interview questions could be tailored to ask about important, but often overlooked, steps in the process, and also incorporate jargon and slang widely used by injured workers. The research leads managed logistics prior to the actual meeting (i.e. venue, agenda setting) and also contributed during the meeting.
A trade-off of involving participants with different levels of experience in doing research is the variance in richness of the data collected. The less experienced participants often did not ask as many probing questions during the interview, thus producing fewer rich quotes. We overcame this challenge through continued training and support for the research team.
Public
The intrinsically motivating social purpose was important in helping us to recruit and mobilize a large number of volunteer non-professional researchers. The publicness of the research fueled commitment, focused work, and motivated the necessary personal sacrifice needed to complete the project. In addition, the public nature of our research topic allows us to draw on significant sympathy and support from larger civil society in Singapore.
This public focus did impose costs on the lead researchers. The research was not valued by senior colleagues with a narrow view of what counts toward evaluations for career progression. In addition, writing public reports does not expedite the academic publication process. There are some synergies in collecting and analyzing data, but not in publishing.
Outcomes
Scaled
We had hoped that scale would give us greater influence on policy and media, allow saturation of a more complex model of the problems, and provide more weighty evidence for elements of our model that might otherwise be disregarded as one-off cases. In general, we did achieve these objectives. We had a considerable number of closed-door meetings with policy makers, a launch with over 100 attendees, and prominent media coverage (Toh, 2017). In addition, we could show the repeated occurrence of problems (such as forged pay slips, or witness intimidation) that were systemic presented in our final report (Fillinger et al., 2017).
The scaling, however, did have trade-offs, and imposed costs and constraints on us as lead researchers. The investment in supervision was substantial. Creating formal hierarchical structures of volunteers which also comply with protecting participants (as required by IRB) required considerable organizational and project management skills. Similarly, we faced challenges managing large weekly meetings and negotiating with a diverse coalition of groups and people. As well as the investment of time, this scale made the research process less flexible: we could not quickly change decisions that took months to reach or change research methods which people had already been trained to implement.
To implement this scaling, our research method brought together academic social science research with volunteerism and public advocacy. Success was dependent on experienced persons who could bridge these two worlds.
Citizen-led
Through a citizen-led approach, our research could reach a large scale at low cost, democratically involve non-professionals, and avoid being skewed toward academic concerns. The research cost very little. Besides a small foundation grant (SGD$10,000) – which was spent on printing the report, managing IRB compliance (anonymization and storage of data), and editing the final report – there was no other funding. The research democratically involved volunteers and built a network of volunteer researchers which had not existed before. Many participants reported being affected by both their contact with migrant workers and their empowerment as researchers. At many points throughout the project, non-professional researchers succeeded in refocusing the research toward social injustice and helping migrant workers. Without these interventions, we feel that the research would have been skewed toward academic questions and of less help to migrant workers.
Public
The research met the conditions of being public since it was aimed at producing a public policy report, aimed at generating a dialogue among a non-professional public – the Singaporean government, NGOs, employers, and the public at large – about a value that is not automatically shared; in this case how Singapore relates to a vulnerable population, which NGOs claim are mistreated, and government and business claim are generally happy. The public aspect of the research was the primary motivation for all involved, driving committed engagement and deep participation, and enabled a project of this scale.
Our public launch of the research was a significant medium through which we publicized our research. We produced a magazine-like booklet containing the findings of our research and key takeaways which we then printed and distributed copies of at the launch. The booklet also included a foreword written by the former attorney general of Singapore who was personally invested in the cause. Our launch was attended by media, academics, and members of the public. It generated some public discussion in the days after through news articles and podcasts interviewing one of us regarding the research.
Comparable study
We look at Charanpal Bal’s (2016) research, documented in his book Production Politics and Migrant Labour Regimes: Guest Workers in Asia and the Gulf, as a comparable study. Bal’s research provided a study of temporary migrant workers in Singapore using data he collected from 45 in-depth interviews with workers and 4 months of participant observation of workers at their workplace. Bal’s work is significant in that it is the only research on male migrant workers in Singapore that has been able to collect data through participant observation at the workplace. Bal’s (2017) work was able to incorporate elements of publicness and generated public discussion on the topic via lectures and written articles.
In comparing Bal’s work with the method outlined in this article, we observe that Bal’s research could have stood to gain further breadth had it adopted elements of scale and participation. Involving volunteers from migrant worker NGOs could have, for example, enabled the research to collect participant observation data from more workplace locations. Capturing such a variety of observations might have been useful in bolstering the public impact of the research.
Challenges of doing scaled, citizen-led, and public qualitative research
Our experience with doing scaled, citizen-led, and public qualitative research revealed five areas that pose potential challenges: creating a deliberative environment, systematizing data collection, systematizing data analysis, being alert to the sensitivities of working on public issues, and conducting scaled participatory research ethically.
Creating a deliberative environment
In citizen-led research, it is essential to create an environment where participants feel confident to exercise decision-making powers over the research process (Bergold and Thomas, 2012). To do so, the professional researchers had to overcome the knowledge gap between them and participants; they also had to take on the taxing role of project manager. This entailed establishing the planning, training, and supervisory structures necessary for novice non-professional researchers to have immersive field experiences – and these needed to be arranged beforehand so that non-professional researchers could experience fieldwork as part of their introduction to the project. On an ongoing basis, we had to facilitate genuinely deliberative decision making with non-professional researchers (who may well have considerable professional accomplishment in their own fields). We do not think that meaningful collaboration between members of the research team requires perfect equality – the professional researchers will still need to lead some parts of the decision making – but rather that participants share their opinions in meetings. The professional researcher must maintain the discursive space which allows for team members to contribute, and sometimes this means reserving their own views or going along with decisions that they do not necessarily agree with.
Systematizing data collection
Collecting large volumes of qualitative data can be complex due to the need to maintain consistent quality across the large volume of data despite the rather unstructured nature of qualitative data. From our experience, we found it helpful to systematize the data collection process to allow for some structure that would help reduce inconsistencies across the data collected. We systematized data collection by doing a few things including (1) mandating training for participants who wanted to be interviewers; (2) negotiating the field sites and access to interviewees well in advance; (3) organizing interviewers by shifts such that they would gather at a single point, be briefed on the interview protocols, and disperse to conduct interviews; (4) setting a time limit for interviewers to submit fieldnotes (e.g. fieldnotes had to be submitted within twenty-four hours of interviews and/or observations); and (5) having both non-professional and professional researchers check through fieldnotes written by novice researchers to provide them with feedback in cases where fieldnotes were inadequate.
Systematizing data analysis
Similar to data collection, analyzing large volumes of qualitative data is challenging in that more labor-power is required to achieve this, and this labor-power needs to conduct the analysis in a consistent manner despite there being no fixed formula for doing the analysis. We systematized data analysis with various techniques including (1) mandating training for participants who wanted to do data analysis, (2) having regular discussions with participants to clarify differing interpretations of how thematic coding framework is applied and letting the professional researchers make the final decision on the coding framework, (3) ensuring that each interview is coded by one participant and re-coded by another to maintain consistency, and (4) having only a small team in the final analysis process where coded interviews are used to identify key findings which are then written up for the research output.
Challenge of working on public issues
Working on public issues posed difficulties. First, exposure of novice researchers to harsh social realities may be stressful for them. For example, in the task force, encounters with families in rented public housing shocked some who had not been exposed to this side of Singapore, while others had to confront childhood memories of similar circumstances. We learnt that this can be better managed by preparing researchers in advance: providing realistic examples of what to expect, sharing advice from social workers in similar situations, and practicing cognitive distancing.
A second difficulty was negotiating political environments that are not welcoming of research on public issues. Singapore’s state-dominated environment translated into students, researchers, and the civil society organizations we studied being wary speaking about politically sensitive issues. This affected both our ability to collect data and to present the research findings. We used compensating strategies to mitigate this effect, such as presenting our findings to a limited audience and involving the relevant government authorities at early stages of the research.
A third difficulty was the involvement of vulnerable populations in our research and thus the need to protect them. We addressed this through traditional methods, such as protecting confidentiality, informed consent, and ensuring the research helped to advance the interests of research subjects.
The challenge of conducting scaled citizen-led research ethically
Citizen-led research raises two further ethical questions: first, how do you ensure ethical treatment of participants by non-professional researchers; and second, how ethical is it to use uncompensated volunteer labor for research? Our studies address the first question by requiring all researcher team members to undertake ethics training and through continuous involvement by professional researchers. On the issue of uncompensated labor, participation in any specific activity was voluntary to the greatest possible extent and the participants were involved in decision making. These expectations were clear during the training, and we ensured that volunteers only contributed the number of hours that they are comfortable with. Furthermore, the professional researcher has to ensure that reasonable norms (i.e. 1–4 hours per week for working professionals, and less than 8 except in exceptional circumstances) are established so as to avoid the potential for exploitation.
Conclusion
Through our framework, we have proposed to combine scale, publicness, and citizen-leadership to support qualitative research under the constraint of limited resources. The constraints and ambitions which drew us to developing this framework include a desire to reach out to and involve ordinary citizens, to study issues of public importance, but with limited access to funding. We build on the growing body of work in conducting public sociological interventions as a collective enterprise (Collins et al., 2017) and engaging citizens as knowledge producers (Purdam, 2014) by offering practical guidance derived from our own experience. In Figure 1, we identified the possibility of a virtuous cycle being generated through the positive interaction of the three dimensions of our framework.
We found that our scaled data collection produced depth by capturing data from multiple perspectives and by sampling across a greater variety of situations without having the research coming to resemble a survey (Small, 2009). In addition, the scaling, particularly the large-scale documentation of injustices, reinforced the urgency of our research and was essential to establishing key claims.
The citizen-led aspect of our research did improve research quality through cross-checking and multiple perspectives (Lamont and White, 2008; Morrill et al., 2005). We also saw volunteers both develop their skills in research design and evidence collection. In addition, the norms of a democratic process within the project energized the broader participatory vision of the research team (Freire, 2000). Tapping the intrinsic motivation of the research team was essential in fueling the virtuous cycle described in Figure 1.
With respect to the public aspect of the research, Burawoy’s (2005) definition mapped onto our project surprisingly closely: we promoted dialogue about values and goals not automatically shared among non-academic publics. While our research did not achieve immediate policy changes, our projects had an impact through the substantial number of people engaged by our projects – particularly the people for whom it was a catalytic first engagement – and built a community to whom this work is legible (Teo, 2021).
To evaluate the success of implementing our framework, we consider whether those positive interactions did indeed emerge. In Table 1, we have summarized the main benefits and trade-offs of incorporating each of these three elements into one’s qualitative research, and summarized our two case studies against this criteria. The student taskforce conducted 157 field visits and generated 300,000 words of field notes which helped the researcher generate two article manuscripts (beyond this one). Over 50 citizen researchers conducted their work within a framework of participatory governance and subsequently credited the experience as encouraging lifelong commitment to public engagement. The Labour Court project conducted 150 in-depth interviews, published of a high-quality report with a foreword by the former Attorney General, and generated significant media attention on the numerous cases of mistreatment found through the interviews. Over 100 citizen researchers were involved, many of whom first encountered migrant worker advocacy through the project, and many of whom have stayed engaged in both research and advocacy for migrant workers.
Examples of benefits and trade-offs of scaled, citizen-led, and public aspects of each case study.
PI: Principal Investigator; IRB: Institutional Review Board.
Overall, both case studies offer benefits and trade-offs against all three dimensions in a manner that defies an easy assessment of success or failure. Nonetheless, the affective transformation of participants through their experience with public research and their continuing engagement in related endeavors are strong arguments in favor of the merits of our approach.
Scaling gives this research approach the ability to collect compelling evidence for richer theorization and convincing argumentation. Citizen-led research methods provide low-cost resources for scaling, incorporating multiple perspectives, and empowering non-professional researchers to generate change. Publicness can motivate researchers through its potential to effect important social change and ensure social relevance.
To manage the public element, we followed established practices from previous research conducted in Singapore on similar topics. These practices include maintaining a consultative relationship with the respective government agencies, keeping them informed before making our research findings public, and using closed-door events with select invitees to share our findings. For any other researchers adopting this framework, they will have to make their own informed assessments regarding how best to position their approach within their political context.
We hope that this framework can meet many diverse needs. It has significant promise in areas of public concern where it is difficult to mobilize research funds. Marginalized communities may benefit from this low-cost, high-impact approach to research and through this, generate evidence for their claims for social and economic justice. This approach to research may also have significant influence on policy makers and the public alike, through involving ordinary citizens as co-producers of research who are informed and engaged with the issues at hand. The framework serves the public mission of the university by connecting teaching and research to the university’s social environment and by incorporating citizens into the research process. The practice of this framework can cultivate a citizenry accustomed to open and reasoned deliberation on public issues.
We look forward to further application and experimentation with this framework and hope that it will provide a basis for deeper and more effective collaboration on research that brings citizens into the discussion of important public issues.
Footnotes
Acknowledgements
The authors are grateful to the Singaporean citizens and migrant workers who agreed to be participants in this research. The authors hope they have done justice to the participants’ stories, and helped these stories contribute to building a brighter future. A.A. and N.H. owe a debt of gratitude to the many talented and tireless citizen researchers of the Labour Court Research Project. I.N. would like to thank the students from whom he learned so much, and all the organizations that generously shared their time and thoughts with the authors.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Labour Court Research Project was supported by a grant from the Chen Su Lan Trust. The Public Policy Task Forces were supported by the SMU-X program.
