Sage Journals: Discover world-class research

Abstract

Pilot studies within qualitative inquiry are crucial yet often hidden aspects of research design. In this article, we argue for pilots to have greater visibility. We explore the role of a pilot in providing a foundation for enhancing ethical reflexivity, drawing on a recent pilot study within a tertiary healthcare education setting. The Scholarship of Teaching and Learning (SoTL) presents a unique environment with complex stakeholder relationships. There is a lack of consensus nationally and internationally on whether all SoTL projects require consideration by institutional ethics review bodies. A pilot study offers an opportunity for ethical steerage of a research project, reflecting ethics in practice whilst augmenting any procedural ethics review requirements. We propose that a qualitative pilot study, as a design strategy, can enhance ethical conduct by researchers. Within SoTL specifically, the pilot can provide an opportunity for researchers to demonstrate a commitment to a pedagogy of care spanning the project’s duration, signifying a commitment to enduring teacher-student relationships within the broader learning environment. Beyond tertiary settings, we believe the pilot study, as a space for ethical reflexivity, has applicability to research settings where caring for and being seen to care for the wider participant community is a critical ethical consideration.

Keywords

methodology reflexivity scholarship of teaching and learning ethical conduct research ethics pedagogy of care

Introduction

This article examines the pilot study’s role in enhancing ethical conduct in qualitative research. Extending existing purposes of pilot studies, such as feasibility and the testing of research instruments, we draw on etymological imagery to argue that a pilot also enables the researcher to consider their position within the research. The word ‘pilot’ has its origins in the Greek word pedon or steering oar. From the early 1500s, the noun described people with a steering role such as a ship’s pilot and, in more recent years, those piloting balloons and planes. A central and common factor is the pilot’s role to guide the vessel through ‘an intricate or perilous passage’ (Harper, 2021). Relating this role to that of the pilot study in qualitative research raises parallels. With the pilot study acting as a mechanism to guide the main research, opportunities arise for the researcher, as the pilot, to make adjustments to enhance the research, ensuring its safe passage to completion. Through the pilot role, the researcher demonstrates a commitment to care for the research project and its participants, reflecting an ethic of care (Gilligan, 1977; Noddings, 2013), expanding the pilot study’s focus from careful to care-filled steerage.

Positionality or reflexivity is a vital aspect of qualitative studies where the researcher is the main research instrument. Role examination is critical in the Scholarship of Teaching and Learning (SoTL) within tertiary education settings. Prosser (2008) identifies the key characteristic that distinguishes SoTL from educational research is that although both are evidence-based, SoTL focuses on the ‘systematic reflection on evidence collected about our own practice to improve the quality of our students learning’ (p. 3). In other words, ‘to make transparent how we have made learning possible’ (Trigwell et al., 2000, p. 156). Rather than being a new field, there is a sense that SoTL is now more visible and underpins good educational practice. As Boyer (1990) foresaw, the acceptance that examining the particularity of educational practices ‘brings legitimacy to the full scope of academic work’ (p. 16). Given the focus on practice, SoTL presents ethical complexity as teachers may wish to recruit their students to inform practice. As with any research, ethical conduct is essential; however, nationally and internationally, there is variation concerning whether SoTL projects require formal institutional ethics review (Lees et al., 2021). In such settings, we believe that undertaking a pilot study can provide an ethical foundation in tandem with, or the absence of, an ethics review process.

To highlight the ethical opportunities afforded by qualitative pilot studies, we first examine links between ethics and reflexivity in qualitative research, particularly informed by Guillemin and Gillam (2004) and Woods (2019). We then introduce the multiple methodological roles of pilot studies in qualitative research before exploring connections between pilot studies and an ethic of care. Illustrated with examples from a recent pilot study conducted within a larger SoTL project, we consider specific steerage points. These points will demonstrate how a pilot study can improve the care-filled ethical nature of the research and the researcher’s actions whilst also acting as a vehicle for showing an ethical commitment to the broader participant community.

Ethical Reflexivity

Reflexivity, focussing on a critical self-evaluation of one’s position as the researcher, is central in qualitative inquiry. Described by Berger (2015) as a ‘continual internal dialogue’ (p. 220), reflexive decisions on who to recruit, how to collect data and how to disseminate findings reflect a process underpinned by the researcher’s values, biases and context, requiring their immersion in the research process (Carpenter, 2018). Ultimately, reflexivity can be seen as a process to enhance the ethical nature of the research.

Guillemin and Gillam (2004) have identified two ethical dimensions within the research process, procedural ethics and ethics in practice. They categorize the former as relating to gaining approval from an institutional ethics review body and the latter to the ethical issues arising during the research. Given that reflexivity results in researchers considering their potential impact on the research, including the participants, there is a close synergy between reflexivity and institutional ethics review aims, particularly respecting persons and minimizing the risk of harm (Gillam & Guillemin, 2018). However, as a form of procedural ethics, institutional ethics review has its critics (Fletcher, 2021). Such up-front processes may potentially impact researchers' rights (Stark, 2007) and constrain the research process (Head, 2020).

Furthermore, researchers claim that ethics review processes are considered by many as burdensome ‘performance’ (Brown et al., 2020, p. 4). Of particular relevance to the dimensions of procedural ethics and ethics in practice, Takeda (2021) argues that procedural ethics insufficiently prepares researchers for the uncertain and unpredictable reality of research in the field. By contrast, ethics in practice reflects the ongoing presence of ethical issues beyond the point of approval, thus providing a rationale for sustained reflexivity throughout the project’s lifetime. At the heart of this process is the researcher’s internal dialogue, underpinned by their ethical values, determining the ethically preferred course of action (Takeda, 2021).

Reflexivity, therefore, provides an opportunity to reflect on and evaluate all aspects of the research process. This critical self-scrutiny bridges procedural ethics within an institutional review process and the reality of research ethics in practice (Guillemin & Gillam, 2004). The ongoing self-scrutiny process also serves to explore the nature of the knowledge produced, thus, reflecting ‘a broader debate about ontological, epistemological and axiological components of the self, intersubjectivity and the colonization of knowledge’ (Berger, 2015, p. 220). Not only does reflexivity play a role in knowledge construction, but it also allows the reader of research to more clearly understand the underlying perspectives and experiences embedded within the researcher’s position. This understanding includes the researcher’s interpretation of the data. Through reflexive practice, the researcher invites the reader to consider their own assumptions, experiences and practical knowledge, adding a raised self-awareness for the reader of their interpretation of the research and its findings.

Guillemin and Gillam (2004) are pivotal in connecting reflexivity and ethics within qualitative inquiry. More recently, Woods (2019) introduces the term ethical reflexivity. With imagery fitting for our exploration of pilot studies as ‘care-filled ethical steerage’, Woods deploys a metaphor of ethical reflexivity ‘as a current running through the project as it progresses’, with presence upstream, midstream and downstream of the research project (p. 462). Gillam and Guillemin (2018) claim that reflexivity is a research skill that the researcher can learn through mentoring and practice. Therefore, a pilot study is an appropriate upstream mechanism for developing this skill, just as the researcher might use a pilot to develop other research capabilities. In addition, such skills may better equip the researcher for steerage through the midstream ‘changing circumstances of fieldwork’ (Fletcher, 2021, p. 3).

The Pilot Study in Qualitative Inquiry: Multifactorial Roles

The pilot study’s importance and function within qualitative inquiry have been categorized by van Teijlingen and Hundley (2001) within the broad areas of feasibility and the trialling of research instruments. Feasibility assessment, where researchers undertake a small-scale version of the proposed larger project to ‘test the research process’ (van Teijlingen & Hundley, 2005, p. 219), can focus on research tangibles such as the research setting and associated elements such as costs (Beebe, 2007). Researchers can also trial processes such as recruitment strategies, data collection and data analysis methods (Doody & Doody, 2015). Pilots also allow researchers to trial research instruments such as surveys to ensure questions are understood and appropriately pitched for the participant population (van Teijlingen & Hundley, 2001). Lastly, researchers can address the human aspects of the study. These aspects might include the researcher ‘gaining experience with participants’ (Beebe, 2007, p. 213) and a subsequent honing of the necessary investigative skills for the project ahead (Doody & Doody, 2015; Kilanowski, 2006).

Despite being multi-purposeful, and a ‘crucial step in the research process’ (van Teijlingen et al., 2001, p. 292), pilot studies have been under-reported in the literature. Beebe (2007) argues that pilot studies are not universally valued. Often, authors only make a brief reference to having conducted a pilot. In many cases, they only refer to having piloted a research instrument such as survey or interview questions. Historically, pilot studies have been linked most closely with positivist research and, if used within qualitative approaches, often predominate within ethnographies (Sampson, 2004). As a result, some academic journals may consider pilot studies unsuitable for publication. Editors electing not to publish pilots argue that there are no results of note to report or that authors present projects as pilots when they are more accurately a small-scale study (Watson et al., 2007). By contrast, we believe pilot studies offer the reader of qualitative research important insights into ethics in practice.

Virtues and a Care Ethic

Carpenter (2018) identifies common principles institutions and professions use to guide research conduct, such as maximizing benefit, respecting rights, ensuring inclusivity and researching with integrity. To have an awareness of how these principles might apply to any proposed research requires a reflexive stance, an immersion in the research and, as Carpenter argues, a set of specific dispositions or virtues, in addition to knowledge and skills. Virtue is defined as ‘a trait of character; manifested in habitual action, that is good for anyone to have’ (Rachels & Rachels, 2018, p. 162). Several ancient thinkers focused their work on virtues, with Aristotle being perhaps the most well-known. Anscombe (1958) explains that between Aristotle and modern times, Christianity emerged and with it a shift to associate virtues with concepts of divine law. Following the Renaissance and the subsequent secularizing of society, virtues were reconceptualized as ‘moral law’ (Rachels & Rachels, 2018). Since the seminal writing of Anscombe, feminist philosophers and others have taken up the mantle with a resurgence and a challenge for virtues to be ‘“lived out” in practice’ (Macfarlane, 2010, p. 47). As a result, and given the need to interact directly with research participants, virtuous characteristics of the qualitative researcher become essential drivers for underpinning ethical research. For instance, research participants need to rely on the researcher’s integrity, so the integrous researcher commits to being trusted and trustworthy (Melia, 2018).

The field of care ethics has dominated modern virtues literature, informed predominantly from women’s perspectives in the late-twentieth-century, notably Gilligan (1977) and Noddings (1984). Feminist perspectives on caring extend the Aristotelian virtue of care, drawing from broad social science disciplines. Noddings (2012) stresses the importance of caring as relational. While one party cares and one is cared for, the relationship may not necessarily only be one way; there is a sense of care reciprocity. Applying care ethics to an educational setting, Noddings (2012) considers the role of the teacher within the teacher-student relationship is to ‘understand what the cared-for is experiencing’ (p. 772), rather than being informed by any assumed needs. A care ethic within an educational setting has clear synergies with a pedagogy of care reflecting the work of Gilligan and Freire (Bingham & Sidorkin, 2004). While most literature discusses a pedagogy of care in relation to early childhood education, there is no reason why a relational pedagogy would not also apply to tertiary learning environments. Further to this, given the dual role of the lecturer as researcher within the SoTL, it seems a logical extension for a pedagogy of care to remain a central tenet of any research project within this setting.

Our experience suggests that a pilot study within SoTL can reflect a pedagogy of care. Relationships underpinned by care exist between the researching lecturer and the pilot participants. However, a pilot study also depicts a desire by both lecturer-researcher and student-participant to care for the future student community. For the researcher, a pilot may demonstrate an act of caring through the opportunities it provides to look after the integrity of the research project in advance through ‘forward reflexivity’ (Pritchard & Whiting, 2012, p. 350). Care for the research project is reciprocated by pilot participants volunteering to participate, knowing that only future student community members may experience research benefits. Together, there is the potential to shape how the research will develop. Having established connections between pilot studies, pedagogy and care ethics, we now turn to contextualizing the unique research features within the SoTL by introducing the specifics of our pilot study.

Our Pilot: a Case Study From Scholarship of Teaching and Learning

Our pilot was part of a doctoral project on ethical conduct for research within tertiary teaching and learning. The SoTL offers a unique setting for the critical analysis of research ethics. There is a tension between educators having duties to protect students as research participants and offering beneficial learning experiences that may come about through research participation. We recognize that variation in the ethics review process and outcomes reflect a degree of contextual subjectivity within research ethics for this type of study. In addition, we were aware of the lack of studies seeking and exploring students’ perspectives on the ethics of research involving them. Thus, we wanted to increase our understanding of students’ views of ethical conduct within tertiary-based teaching and learning research and the extent to which these views might align with the literature documenting ethical issues within this setting, as identified by institutional ethics review bodies.

Ethical Issues Within SoTL

Collecting and disseminating experiences and outcomes of teaching practice raises intricate ethical issues. At the crux of these, the researcher may also be the lecturer, and the student may also be a research participant. Complexity manifests itself through these dual-role conflicts (Linder et al., 2014). The lecturer is responsible for teaching their students and providing them with positive learning experiences, yet simultaneously they may pursue a range of other research goals. Students may benefit from changed teaching practices as an outcome of their research participation. Still, it is more likely that only future students will experience curriculum improvements. By comparison, it is more likely that the lecturer-as-researcher will gain directly from the research outcomes. There are concerns that the power imbalance within the lecturer-student relationship results in student participants being a vulnerable population (Cleary et al., 2014). An imbalance might create an environment where students feel compelled to participate or feel unable to opt not to participate. Woods (2019) asks whether researchers are guests or intruders. In the case of SoTL projects, we often cohabitate the teaching, learning and research environment with our participants, creating a unique dynamic for conducting ethical research.

It is also possible that the way ‘risks’ are considered in many social science areas, notably within SoTL, may reflect a biomedical legacy of ethics review processes. For example, Leentjens and Levenson (2013) highlight their concerns for students as participants, offering strategies to mitigate the potential risks from the teacher-student relationship, the use of third-party recruitment or electing not to recruit current students. A growing number of scholars have presented alternative ethics frameworks to underpin research (Hudson et al., 2010; Stutchbury & Fox, 2009; Tangen, 2014; Vermeylen & Clark, 2017). Arguably, frameworks reflecting relational or situational ethics might assess research and risk differently to the current predominance of rule and consequentially-based ethics review protocols. Given that students are the central contributor to and benefactor of SoTL projects provides a clear rationale for our project to seek their views and for these views to inform ethical conduct in teaching and learning research settings, irrespective of whether an institutional ethics review process is required.

There are calls for vulnerability as a term within research ethics to be scrutinized in greater depth regarding how it is defined and applied (Grinnell, 2004; Kipnis, 2001; Levine et al., 2004). Dual role research illuminates the complexity of its use. For instance, a counterargument to the predominant narrative of students as vulnerable is that students are competent, intelligent adults who can make informed choices within the tertiary setting. Kipnis (2001) stresses that through consent, those giving or withholding permission hold an ‘ethical power’ (p.4). Students may equally consider themselves vulnerable if there is no evidence of the effectiveness of the teaching and learning strategies they experience. The setting, relationships and potential ethical issues provide a clear rationale for the research design to include a pilot study. Through this process, a pilot can acknowledge the potential research complexity at the project’s genesis whilst also reflecting a caring disposition for the research participants and the broader participant community of current and future students.

There is an acknowledgement that institutional ethics review bodies were not established with the SoTL in mind, stemming from a need to protect patients in biomedical research settings (Martin, 2013). Some institutions deem that projects evaluating teaching practice do not require ethics approval. For example, reporting student participation and views on a nursing workshop at an Australian university, Craft et al. (2017) cite: ‘[e]thical approval was sought but was not required as it is the policy of this university that evaluations of teaching approaches do not require ethical certification’ (p. 115). Where ethics review is mandated, there are arguments that current processes weigh too heavily towards managing risk over benefits, inhibiting educational change (Butterwick et al., 2020). Some academics lack awareness of the need for ethics review when assessing their teaching practices (Stockley & Balkwill, 2013). In some cases, a failure to engage with ethics review processes has had disciplinary consequences (Tomkowiak & Gunderson, 2004). Therefore, in the absence of a consistent approach to ethics review, it is incumbent on the researcher to demonstrate ethical conduct.

Applying reflexivity within our pilot study has provided a way to lay a solid foundation for the research ahead, especially in better understanding our position as those who have a duty and desire to care for the research project, its participants, relationships and environment. In addition, purposefully being reflexive within the pilot helped address the concerns of Browne (2013), who argues that researchers can all too easily add reflexivity to the methodology as an afterthought rather than a cornerstone.

Ethical Steerage Points

We now present the steps taken within our pilot study to embed and reflect ethical conduct as researchers and how reflexivity enhanced our pilot’s ethical nature, and, undoubtedly, the research that will follow. We consider our pilot study through three ethical steerage points, mirroring methods, modelling ethical relationships and practicing forward reflexivity (Figure 1). We do not claim that these are the only steerage points to consider, nor will all points be relevant for all studies. We have left space for additional areas of ethical consideration by other researchers utilizing pilot studies in qualitative research.

Figure 1.

Ethical steerage points: Researcher strategies to enhance ethical conduct in qualitative pilot studies.

Mirroring Methods

We planned our pilot study to mirror each part of our proposed research project. We did this as we did not want to make assumptions about which aspects of the research may need adjusting. Our institution required us to seek ethics approval before commencing our pilot. However, it did not require us to include a pilot in our project, and so in that way, we viewed the ethics review as procedural. We had already made design decisions that centred on ways to strengthen the ethical nature of our research. Therefore, comparing our design alongside the institutional review requirements helped us see the difference between Guillemin and Gillam (2004)’s procedural and practice ethics ‘in practice’. A pilot study can be one strategy to consider implementing, especially if your institution does not require ethics approval, as can be the case for SoTL projects. The pilot provides a practice-based method to help safeguard the project and the participants. The embedding of reflexivity creates an additional evaluative process to assess methodological, but importantly, ethical considerations (Woods, 2019).

Our ethics approval permitted us to recruit across the same population as we would be seeking participants from in the main study. We invited participants to join either a pilot focus group or a one-on-one interview. We offered these different data collection methods to understand better how data would be generated and how the various dynamics of the interview setting might impact the type of conversations. For example, there are suggestions that focus groups facilitate unique interpersonal interactions, yet individual interviews provide more depth (Guest et al., 2017). Alongside the literature, our experience as lecturers was that focus groups more closely emulated group work, with which students would already be familiar. Our pilot enabled our participant community to help shape the study design by us trialling both methods.

The design of our data collection method included the use of vignettes based on recently published SoTL projects. These vignettes formed the basis of conversations with participants to elicit their perspectives of what constituted ethical conduct as hypothetical participants in each vignette. We selected vignettes in readiness for their use in the main study. Still, we remained open to reviewing our selection based on the level of engagement with each vignette and any feedback about our choices. Aside from the structure of having vignettes, different interview styles were also trialled, including semi-structured indicative questions prepared beforehand, along with the trialling of more unstructured free-flowing conversation. Finally, we transcribed the recordings with varying types of data analysis, such as concept mapping (Clayton, 2006) and thematic analysis (Braun & Clarke, 2012), assessing the most effective way to make sense of the data.

We aimed to learn from the pilot what worked best and demonstrate a level of care for hearing student voices and hearing them in the best way possible. We followed Noddings (2012) guidance, who explains that within care ethics, ‘it is important not to confuse what the cared-for wants with that which we think he should want. We must listen, not just “tell”, assuming that we know what the other needs’ (p. 773). While it is common to involve participants when using participatory and action research methodologies (Brydon-Miller et al., 2011) and researchers frequently use these methodologies in educational settings, our main project was not an example of participatory action research. We piloted our study with students from the wider participant community as a way to ensure we could adequately care for our eventual study participants. We saw value in seeking viewpoints from the participant community to show respect, irrespective of the specific qualitative inquiry approach. This democratizing strategy may be mutually beneficial where there are enduring relationships with the participant community, as is the case within tertiary teaching and learning settings. Seeking, welcoming and valuing student views were ways to acknowledge and attempt to set aside our academic positionality.

Modelling Ethical Relationships

Implementing our pilot study revealed a degree of paradox in terms of dual role relationships. On the one hand, we were mindful of the literature espousing caution regarding power imbalance and coercion within educational settings. Yet, we guided our planning by the value we placed on the relational underpinnings of teaching (Noddings, 2012). We used our pilot study to establish new and grow existing relationships with academic staff and student groups across our faculty. With the course leader permission, we made a point to visit students in the classroom to inform them of the invitation to participate, rather than recruiting online within the institutional learning management system. Rather than avoiding relationships, we chose to foster them, reiterating our existing relationships with all prospective participants, in the pilot or the main study, by highlighting our shared connections as teachers and learners within the same tertiary institution. We stressed to classes we visited the importance of the pilot study and the participants’ views in helping to shape the research design. Course leaders would introduce us and our research, reflecting the trust we hoped students would reciprocate.

Through our pilot study, we also drew on the work of philosopher Levinas, whose thinking aligns well with Nodding’s position of interpersonal relationality and virtues but varies with a focus on the ethical responsibility for ‘the Other’, whose proximity maybe both close and distanced (Vermeylen & Clark, 2017). We saw our pilot participants as immediately proximal, given their direct involvement as pilot participants, while distancing spatially from the eventual main project. Their voices spoke on behalf of those whose time to be proximally involved was still to come by acting as a bridge between the pilot and the primary research.

For Levinas, as researchers, ‘the Other’ affects us, creating in us an infinite obligation, and through this sense of duty, we gain increased self-awareness (Vermeylen & Clark, 2017). Therefore, a Levinasian approach to research ethics requires a reflexive stance to ensure that participant community voices remain central. Accountability to ‘the Other’ should not be an afterthought, nor should it be limited; Levinas reminds us that we should never be finished with our responsibilities (Levinas & Robbins, 2001). We argue that applying a Levinasian lens also shows that accountability should not be limited to just the main research project; there is an onus on ensuring responsibility starts early. In this way, a pilot can reflect Guillemin and Gillam (2004)’s procedural ethics given its precursory, point in time role. At the same time, the pilot and its participants shape what will be an enduring foundation for the primary research. The onus on the researcher for sustained ethical reflexivity reflects a dynamic ethics in practice.

Practising Forward Reflexivity

A pilot study can effectively gain insights into aspects of the research that may need amending before its onset or identify design and implementation elements that might need rechecking throughout the project’s duration. In this way, Pritchard and Whiting (2012)’s ‘forward reflexivity’ (p. 350) invites the researcher to consciously and deliberatively adopt a reflexive lens across the duration of the research project instead of as a methodological afterthought. Reflexivity within our pilot study helped us get a feel for the ‘current’ upstream. Furthermore, it played a role in normalizing reflexivity, laying a foundation for us to self-evaluate our project position continually.

The pilot provided the vehicle for ethical steerage and the time and space to plan and prepare for the research journey. We created time and space for forward ethical reflexivity by using a critical friend, a strategy implemented commonly in education research and practice whereby an invited person provides an external perspective to provide support and add rigour (Smith & Bradbury, 2019). The critical friend usually shares a similar professional background to the researcher but specifically does not have experience of the specific context (Foulger, 2010). The shared foundation of the critical friendship provides a trusted platform for the researcher to gain insights into the specifics of the research setting that may not have otherwise been visible. From a reflexive perspective, a critical friend can be the catalyst for reflection and help reveal unexamined assumptions (Fletcher, 2019).

Solo researcher projects often employ a critical friend (Bullough Jr & Pinnegar, 2001). However, we believe their use also has value in multi-researcher projects. In our case, while only the primary researcher conducted the data collection, all researchers considered themselves tertiary education ‘insiders’. While there are strengths to insider research through the richness of data, there are potential limitations. And, in the case of dual relationship teaching and learning research, these limitations underpin our broader research project and rationalize our inclusion of a pilot study. Therefore, an external critical friend was invited to uphold researcher integrity as a ‘research tool’ (Appleton, 2011, p. 1).

In our case, the critical friend acted as the researcher, with the primary researcher taking on a participant role. This role play provided reflexive space for the researcher to reflect on their choice of vignettes and their perceptions of the indicative interview questions, all from a participant and self-perspective view. A further critical friend joined to create a sense of discussion to emulate the focus group. Following this session, the primary researcher reflected on their experience, the questions, how they responded and their thoughts about the broader research project. This period of reflection helped identify and explore biases and assumptions. The primary researcher noted their experiences in a reflective journal. Then they transcribed a recording of the interview, again with reflections journaled. Not only was the researcher able to think reflexively about the way they had chosen and constructed the vignettes, but the critical friend was also able to ‘pilot the pilot’, providing external feedback. Chenail (2011) invites insider researchers to consider an ‘interviewing the researcher’ strategy as there is the possibility that insiders may limit their study to areas of familiarity rather than also finding ways to focus on ‘what they don’t know they don’t know’ (p. 257). Therefore, a critical friend plays a vital role in enabling the researcher to practice forward reflexivity underpinned by integrity, rigour and care.

Conclusion

In this article, we have argued for pilot studies having greater visibility within qualitative inquiry with their purpose broadened from predominantly methodological to encompass an ethical role. Importantly, we believe the pilot study provides an optimal setting for forward-thinking, upstream ethical reflexivity. While reflexive pilot studies can add value to any qualitative inquiry project, we have specifically considered their value for projects within the SoTL where dual roles of both researcher and participant have the potential for ethically challenging relationships. The teacher has an ongoing relationship with the broader student community within a tertiary education setting. We underpinned our pilot study with a pedagogy of care, including care for those students participating in our pilot but also care for ‘the Other’ unseen students within the wider student community. In this way, a pilot study has been one way to uphold the integrity of the dual role teacher-researcher, making visible the commitment to the research, the student participant community and their learning experiences.

As a result of our pilot study, we felt more connected to the pilot participants’ voices and the wider community they represented. In addition, we felt better prepared for uncertainty. There was a sense that we had cared for the wider student community in terms of protecting them from any apparent flaws in the design, and it helped us be open to seeing future areas for refinement. We have shared three areas of ethical steerage employed in our pilot study: mirrored methods, modelling ethical relationships and practicing forward reflexivity. Whilst our pilot and the main project required institutional ethics review and approval, we are aware of the variation of ethics review body processes for SoTL projects internationally and nationally. We present our pilot study to offer ethical evaluation guidance where formal ethics review is not required. We hope that other researchers will see the applicability of our pilot reflections beyond tertiary teaching and learning environments to research settings where caring for and being seen to care for the wider participant community is a critical ethical consideration.

Footnotes

Acknowledgements

The authors would like to acknowledge the students who participated in the pilot study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Amanda B. Lees

Rosemary Godbold

References

Anscombe

G. E. M.

(1958). Modern moral philosophy. Philosophy, 33(124), 1–19. https://doi/10.1017/s0031819100037943

Appleton

(2011). Critical friends’, feminism and integrity: A reflection on the use of critical friends as a research tool to support researcher integrity and reflexivity in qualitative research studies. Women in Welfare Education, 10, 1-13.

Beebe

L. H.

(2007). What can we learn from pilot studies? Perspectives in Psychiatric Care, 43(4), 213–218. https://doi/10.1111/j.1744-6163.2007.00136.x

Berger

(2015). Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative research. Qualitative Research, 15(2), 219–234. https://doi/10.1177/1468794112468475

Bingham

Sidorkin

A. M.

(2004). The pedagogy of relation: An introduction. Counterpoints, 259, 1-4.

Boyer

E. L.

(1990). Scholarship reconsidered: Priorities of the professoriate. The Carnegie Foundation for the Advancement of Teaching.

Braun

Clarke

(2012). Thematic analysis. In Cooper

(Ed.), APA handbook of research methods in psychology (Vol. 2). Americal Psychological Association. https://doi.org/10.1037/13620-004

Browne

B. C.

(2013). Recording the personal: The benefits in maintaining research diaries for documenting the emotional and practical challenges of fieldwork in unfamiliar settings. International Journal of Qualitative Methods, 12(1), 420–435. https://doi/10.1177/160940691301200121

Brown

Spiro

Quinton

(2020). The role of research ethics committees: Friend or foe in educational research? An exploratory study. British Educational Research Journal, 46(4), 747–769. https://doi.org/10.1002/berj.3654

10.

Brydon-Miller

Kral

Maguire

Noffke

Sabhlok

(2011). Jazz and the banyan tree. In Handbook of qualitative research (pp. 387–400).

11.

Bullough

R. V.

Jr Pinnegar

(2001). Guidelines for quality in autobiographical forms of self-study research. Educational Researcher, 30(3), 13–21. https://doi/10.3102/0013189x030003013

12.

Butterwick

Head

Madalinska-Michalak

Raykov

Taylor

Taylor-Neu

Zgaga

(2020). Introduction: Ethical issues in educational research. European Educational Research Journal, 19(1), 3–9. https://doi.org/10.1177/1474904119896506

13.

Carpenter

(2018). Ethics, reflexivity and virtue. In Iphofen

Tolich

(Eds.), The Sage handbook of qualitative research ethics (pp. 35–50). Sage. https://doi/10.4135/9781526435446.n3

14.

Chenail

R. J.

(2011). Interviewing the investigator: Strategies for addressing instrumentation and researcher bias concerns in qualitative research. Qualitative Report, 16(1), 255–262. https://doi.org/10.46743/2160-3715/2011.1051

15.

Clayton

L. H.

(2006). Concept mapping: An effective, active teaching-learning method. Nursing Education Perspectives, 27(4), 197–203.

16.

Cleary

Walter

Jackson

(2014). Above all, ‘do no harm’: Key considerations when including students as research participants in higher education settings. Contemporary Nurse, 49(1), 93–95. https://doi.org/10.1080/10376178.2014.11081958

17.

Craft

Christensen

Bakon

Wirihana

(2017). Advancing student nurse knowledge of the biomedical sciences: A mixed methods study. Nurse Education in Practice, 48, 114-119. https://doi.org/10.1016/j.nedt.2016.10.003.

18.

Doody

C. M.

(2015). Conducting a pilot study: Case study of a novice researcher. British Journal of Nursing, 24(21), 1074–1078. https://doi.org/10.12968/bjon.2015.24.21.1074

19.

Fletcher

(2019). An invited outsider or an enriched insider? Challenging contextual knowledge as a critical friend researcher. In Green

Plowright

Johnson

N. F.

(Eds.), Educational researchers and the regional university (pp. 75–92). Springer. https://doi.org/10.1007/978-981-13-6378-8_5

20.

Fletcher

(2021). Navigating procedural ethics and ethics in practice in outdoor studies: An example from sail training. Journal of Adventure Education and Outdoor Learning, Advanced online publication. https://doi.org/10.1080/14729679.2021.1902826

21.

Foulger

T. S.

(2010). External Conversations:An unexpected discovery about the critical friend in action research inquiries. Action Research, 8(2), 135–152. https://doi.org/10.1177/1476750309351354

22.

Gillam

Guillemin

(2018). Reflexivity: Overcoming mistrust between research ethics committees and researchers. In Iphofen

Tolich

(Eds.), The Sage handbook of qualitative research ethics (pp. 263–274). Sage. https://doi/10.4135/9781526435446.n18

23.

Gilligan

(1977). In a different voice: Women's conceptions of self and of morality. Harvard Educational Review, 47(4), 481–517. https://doi/10.17763/haer.47.4.g6167429416hg5l0

24.

Grinnell

(2004). Subject vulnerability: The precautionary principle. The American Journal of Bioethics, 4(3), 72–74. https://doi/10.1080/15265160490497416

25.

Guest

Namey

Taylor

Eley

McKenna

(2017, 2017/11/02). Comparing focus groups and individual interviews: findings from a randomized study. International Journal of Social Research Methodology, 20(6), 693-708. https://doi.org/10.1080/13645579.2017.1281601

26.

Guillemin

Gillam

(2004). Ethics, reflexivity, and “ethically important moments” in research. Qualitative Inquiry, 10(2), 261–280. https://doi/10.1177/1077800403262360

27.

Harper

(2021). Pilot. Online etymology dictionary. Retrieved 29 March, 2021 from https://www.etymonline.com/word/pilot

28.

Head

(2020). Ethics in educational research: Review boards, ethical issues and researcher development. European Educational Research Journal, 19(1), 72–83. https://doi.org/10.1177/1474904118796315

29.

Hudson

Milne

Reynolds

Russell

Smith

(2010). Te Ara tika Guidelines for māori research ethics: A framework for researchers and ethics committee members. Health Research Council of New Zealand.

30.

Kilanowski

J. F.

(2006). Lessons learned from a pilot study on the health status of children from itinerant populations. Journal of Pediatric Health Care, 20(4), 253–260. https://doi/10.1016/j.pedhc.2005.12.018

31.

Kipnis

(2001). Vulnerability in research subjects: A bioethical taxonomy. National Bioethics Advisory Commission. https://govinfo.library.unt.edu/nbac/human/overvol2.pdf#page=209

32.

Leentjens

A. F. G.

Levenson

J. L.

(2013). 2013Ethical issues concerning the recruitment of university students as research subjects: Reply to lumley and jasinski. Journal of Psychosomatic Research, 75(4), 400. https://doi.org/10.1016/j.jpsychores.2013.07.016

33.

Lees

A. B.

Walters

Godbold

(2021). Variation in ethics review for tertiary-based educational research: An international and interdisciplinary cross-sectional review. Journal of Academic Ethics, 19(4), 517–540. https://doi/10.1007/s10805-020-09382-1

34.

Levinas

Robbins

(2001). Is it righteous to be?: Interviews with emmanuel Levinas. Stanford University Press.

35.

Levine

Faden

Grady

Hammerschmidt

Eckenwiler

Sugarman

(2004). The limitations of "vulnerability" as a protection for human research participants. The American Journal of Bioethics, 4(3), 44–49. https://doi.org/10.1080/15265160490497083

36.

Linder

K. E.

Elek

E. D.

Calderon

(2014). SoTL and the institutional review board: Considerations before navigating the application process for classroom research in higher education. Journal of the Scholarship of Teaching and Learning, 14(2), 1–14. https://doi/10.14434/josotl.v14i2.4217

37.

Macfarlane

(2010). Researching with integrity: The ethics of academic enquiry. Routledge.

38.

Martin

R. C.

(2013). Navigating the IRB: The ethics of SoTL. New Directions for Teaching and Learning, 2013(136), 59–71. https://doi.org/https://doi.org/10.1002/tl.20076

39.

Melia

(2018). The professional integrity of social science researchers-can virtue ethics help? In Emmerich

(Ed.), Virtue ethics in the conduct and governance of social science research (pp. 83–101). Emerald. https://doi/10.1108/s2398-601820180000003005

40.

Noddings

(1984). A feminine approach to ethics and moral education. University of California Press.

41.

Noddings

(2012). The caring relation in teaching. Oxford Review of Education, 38(6), 771–781. https://doi.org/10.1080/03054985.2012.745047.

42.

Noddings

(2013). Caring: A relational approach to ethics and moral education. University of California Press.

43.

Pritchard

Whiting

(2012). Autopilot? A reflexive review of the piloting process in qualitative e-research. Qualitative Research in Organizations and Management: An International Journal, 7(3), 338–353. https://doi/10.1108/17465641211279798

44.

Prosser

(2008). The scholarship of teaching and learning: What is it? A personal view. International Journal for the Scholarship of Teaching and Learning, 2(2). https://doi.org/10.20429/ijsotl.2008.020202

45.

Rachels

(2018). The elements of moral philosophy (9th ed.). McGraw-Hill.

46.

Sampson

(2004). Navigating the waves: The usefulness of a pilot in qualitative research. Qualitative Research, 4(3), 383–402. https://doi/10.1177/1468794104047236

47.

Smith

T. W.

Bradbury

L. U.

(2019). Wiser together: Sustaining teaching excellence with a self-study/critical friend. To Improve the Academy, 38(1), 18–32. https://doi.org/10.1002/tia2.20091

48.

Stark

(2007). Victims in our own minds? IRBs in myth and practice. Law and Society Review, 41(4), 777–786. https://doi/10.1111/j.1540-5893.2007.00323.x

49.

Stockley

Balkwill

L.-L.

(2013). Raising awareness of research ethics in SoTL: The role of educational developers. Canadian Journal for the Scholarship of Teaching and Learning, 4(1), 7. https://doi/10.5206/cjsotl-rcacea.2013.1.7

50.

Stutchbury

Fox

(2009). Ethics in educational research: Introducing a methodological tool for effective ethical analysis. Cambridge Journal of Education, 39(4), 489–504. https://doi.org/10.1080/03057640903354396

51.

Takeda

(2021). Uncovering ethical concerns through reflexivity—ethics in practice in fieldwork. International Journal of Social Research Methodology, 1–6, Advanced online publication. https://doi.org/10.1080/13645579.2021.1889828

52.

Tangen

(2014, 2014/11/02). Balancing ethics and quality in educational research—the Ethical Matrix method. Scandinavian Journal of Educational Research, 58(6), 678–694. https://doi.org/10.1080/00313831.2013.821089

53.

Tomkowiak

J. M.

Gunderson

A. J.

(2004). To IRB or Not to IRB? Academic Medicine, 79(7), 628–632. https://doi/10.1097/00001888-200407000-00004

54.

Trigwell

Martin

Benjamin

Prosser

(2000). Scholarship of teaching: A model. Higher Education Research & Development, 19(2), 155–168. https://doi/10.1080/072943600445628

55.

van Teijlingen

R Hundley

(2001). The importance of pilot studies. Social Science Update, 35, 1-4.

56.

van Teijlingen

Hundley

(2005). Pilot studies in family planning and reproductive health care. BMJ Sexual & Reproductive Health, 31(3), 219–221. https://doi/10.1783/1471189054483735

57.

van Teijlingen

E. R.

Rennie

A. M.

Hundley

Graham

(2001). The importance of conducting and reporting pilot studies: The example of the Scottish Births Survey. Journal of Advanced Nursing, 34(3), 289–295. https://doi/10.1046/j.1365-2648.2001.01757.x

58.

Vermeylen

Clark

(2017). An alternative ethics for research: Levinas and the unheard voices and unseen faces. International Journal of Social Research Methodology, 20(5), 499–512. https://doi.org/10.1080/13645579.2016.1220117

59.

Watson

Atkinson

Rose

(2007). Editorial: Pilot studies: To publish or not. Journal of Clinical Nursing, 16(4), 619–620. https://doi/10.1111/j.1365-2702.2006.01830.x

60.

Woods

(2019). Commentary 1: The role of ethical reflexivity in conducting ethically challenging qualitative research. Journal of Empirical Research on Human Research Ethics, 14(5), 462–465. https://doi.org/10.1177/1556264619857856a

Illuminating the Role of Reflexivity Within Qualitative Pilot Studies: Experiences From a Scholarship of Teaching and Learning Project

Abstract

Keywords

Introduction

Ethical Reflexivity

The Pilot Study in Qualitative Inquiry: Multifactorial Roles

Virtues and a Care Ethic

Our Pilot: a Case Study From Scholarship of Teaching and Learning

Ethical Issues Within SoTL

Ethical Steerage Points

Mirroring Methods

Modelling Ethical Relationships

Practising Forward Reflexivity

Conclusion

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iDs

References