Considerations of Quality in Phenomenographic Research

Abstract

In this paper the author considers issues of quality in phenomenographic research. Research rigor, which is traditionally evaluated by validity and reliability criteria, ensures that research findings reflect the object of study. Quality in research subsumes rigor and extends considerably beyond satisfying the criteria for rigor. A piece of research has to convince readers of its quality when evaluated against criteria that have been developed through contributions and agreements within the research community. This paper tackles the quality issue in phenomenographic research in three steps. First, criteria for quality in qualitative research are discussed. Second and drawing on the literature, related issues when the criteria are applied to phenomenographic studies and the ways of addressing the issues are examined. Finally, the phenomenographic process is analyzed and suggestions are made for enhancing quality at each stage of the process. New phenomenographic researchers especially will find this paper as a useful guide.

Introduction

The aim in phenomenography is to describe variations of conception that people have of a particular phenomenon. In the context of phenomenography, B. Johansson, Marton, and Svensson (1985) have explained that a conception is a way of seeing or understanding something, or the meaning of something to a person. A person's conception of something is assumed to be relational as it is internally constituted between the person and the world. Therefore, conceptions are expected to be different. Empirical phenomenography systematically explores participants' experiences and identifies their conceptual meanings of the phenomenon of interest. These are classified into categories according to their similarities and differences. Finally, phenomenographic findings describe the different categories of conceptions of the phenomenon from the perspective of participants (Marton & Pong, 2005). Thus, the collective variations of conception of the phenomenon rather than the conceptions of the individual participants (Marton, 1994) are reported.

In this paper I consider issues of quality in phenomenographic research and discuss how quality can be attained through careful and thorough procedures at each stage of the phenomenographic process. Rigor in research, which is traditionally characterized by the validity and reliability of the research, ensures that findings reflect the object of study. Quality in research, however, subsumes rigor and extends considerably beyond satisfying the validity and reliability criteria for rigor. Readers within an academic community have to be convinced of the quality in a piece of research when they evaluate it against criteria for quality that have been developed through contributions and agreements within that community over time (Larsson, 1993). ¹

Thus, the quality issue is tackled in three steps. First, criteria for both rigor and quality in qualitative research are discussed generally. Second and drawing on the literature, issues relating to the conceptual underpinnings of phenomenography when the criteria are applied to phenomenographic studies are discussed and possible ways of addressing the issues are examined. Third, the phenomenographic process is analyzed and suggestions for enhancing rigor and quality at each stage of the process are advanced. The first two steps are presented in the next section and step three is presented in the section that follows. The final section concludes the paper.

Criteria for rigor and quality in qualitative research and their applications to phenomenographic research

Qualitative research aims to explore complex social phenomena, as experienced by people, for deeper and more meaningful understanding. It usually involves collecting, organizing and interpreting textual data that are derived from talk or observation with the ambition of transferability beyond the study setting (Malterud, 2001). Due to the nature of qualitative research, the usual ways in which criteria are applied to evaluate quantitative research rigor may not be appropriately applied to evaluate qualitative research. Nonetheless, research rigor is requisite in qualitative research.

Qualitative research paradigms are diverse, and they embrace different ontological, epistemological, and methodological assumptions (Guba & Lincoln, 2003). They seek answers to questions that are different in nature, have different assumptions of data, and carry out different research processes. Consequently, there are issues associated with judging the claims that qualitative researchers make about their findings.

As a way forward, Guba and Lincoln (1981) introduce four criteria, intended as counterparts of quantitative research rigor criteria, for judging the trustworthiness of qualitative research findings. Briefly, the four criteria are (a) credibility, which addresses the aspect of truth in the findings and is concerned with testing the findings with the various sources from which the data are drawn; (b) fittingness, which is concerned with the applicability of the findings in other contexts; (c) auditability, which is concerned with the consistency of the findings if the research is replicated; and (d) confirmability, which is concerned that the findings are not a function of the biases and motives of the researcher. Guba and Lincoln (1981) have also recommended verification strategies for evaluating research findings against these criteria.

There are diverse views within the community of qualitative researchers regarding the role of criteria, how they should be applied, and whether a single set of criteria is possible or appropriate for the rich tapestry of qualitative research. Creswell and Miller (2000) commented on the plethora of modified quantitative research criteria and alternative criteria for evaluating the rigor of qualitative research that are available in the literature. They further argue that due to the diverse paradigmatic assumptions underlying the various qualitative research traditions, criteria and discussions about validation procedures in the literature offer little guidance regarding the choice or suitability of criteria for a particular study. In Morse's (2006) view, qualitative researchers' attempts to emulate quantitative researchers in applying criteria for rigor to evaluate qualitative research have contributed to the problem.

It is apparent that the focus on criteria for rigor alone to evaluate qualitative research in one form or another is limiting; over time the relevance of the notion of quality has lent itself as a more inclusive concept for evaluating qualitative research. It is, however, important to note that quality is not intended to replace the centrality of rigor in qualitative research. In the context of qualitative research, quality subsumes rigor and extends to other aspects, such as the transferability of findings and the commitment to reflexivity.

Spencer, Ritchie, Lewis, and Dillon (2003) have developed a comprehensive framework for evaluating qualitative research after reviewing 29 existing quality frameworks. They describe in their framework four research characteristics that embrace the central principles for quality in that a quality piece of research must be (a) contributory in advancing knowledge, (b) defensible in design in addressing the research questions, (c) rigorous in conduct of a systematic and transparent research process, and (d) credible in claim through well-reasoned and plausible arguments proffered by evidence derived from the study.

Some of the above principles have been previously elaborated in the literature. For instance, Larsson (1993) has identified knowledge contribution at both the collective and individual levels. He explains that as research is ultimately about producing new knowledge, the decisiveness about knowledge contribution at the collective level is a characteristic of research quality. This level of knowledge contribution is distinct from knowledge contribution at the individual level when a reader is convinced by the study to see some aspects of reality in a new way.

The principle of defensibility in design applies to the internal consistency of the research question(s), the nature of the knowledge of the object of study, the data, and the method(s) of analysis. An example phenomenographic study is given later to demonstrate how this principle is applied by explaining explicitly at the outset of a study how these elements are linked to attain internal consistency.

Consistent with the principle of rigorous conduct, Morse, Barrett, Mayan, Olson, and Spiers (2002) have advocated that careful and thorough procedures that promote validity and reliability be woven into every step of the research process to ensure rigor. In the 1980s, following the seminal work of Guba and Lincoln (1981), there was a general rejection of the use of the traditional criteria for rigor for evaluating qualitative research and a shift toward judging the trustworthiness of findings. Morse et al. have warned that the shift of focus from process to product merely evaluates but does not ensure rigor. Consequently, it risks missing serious threats till they are too late to be corrected. They also have issued a plea for a return to traditional terminology and argue that the use of alternatively termed criteria for evaluating qualitative research marginalizes the research from mainstream science and scientific legitimacy.

In keeping with Morse et al. (2002), the following discussion of the broader criteria for quality in qualitative research has retained the use of traditional terminology, such as validity and reliability.

Validity

Issues

Research validity basically means the internal consistency of the object of study, data and findings. There are, however, a number of conceptions of validity and ways of validation that exist in the qualitative research literature (see Spencer et al., 2003), which are symptomatic of their complexity and unresolved issues. The literature has identified validity issues specific to phenomenographic research pertaining to internal consistency.

Phenomenographic researchers aim at investigating conceptions, and interview is the usual access to people's conceptions. However, the reliance on interviews, which are contextual, to provide accurate accounts about self or the world has been critiqued in the literature. For example, Mishler (1991) has commented that the lack of a noncontextual and transparent relation between representation and reality in interviews, and the inherent difference between language and meaning in interview data, have presented both theoretical and methodological problems. Hammersley (2003) recognized these issues but does not advocate abandoning all uses of interview data. He advised researchers to be aware of the dangers of using interview data and to exercise great caution in interpreting, using and drawing conclusions from them.

Whereas the preceding discussion relates to the validity of interview data in general, Säljö (1996, 1997) directed his critique specifically at the research design of phenomenographic studies. He questioned the assumption of congruence among utterances in oral discourse (interview data) and conceptions (object of study) where utterances are analyzed and later reported as conceptions (findings). Säljö is dissatisfied that phenomenographic studies have not justified a valid link among these elements. He is also critical of the practice of interpreting linguistic differences and choice of words among interviewees as differences in meaning of conceptual content. ² In essence, the validity of phenomenographic studies has been questioned fundamentally.

Ways of addressing the issues

Aspects of the validity issue in phenomenographic studies have been addressed comprehensively in Svensson, Anderberg, Alvegard, and Johansson (2006b) and T. Johansson, Svensson, Anderberg, and Alvegard (2006). Their research focuses on the reciprocal relation, which they have termed interplay, between conception, meaning, and oral expression. They explain that in the system of the interplay, the conceptual meaning in an expression is constituted by the conception and the expression. Further, the choice of words in the expression to express the conceptual meaning is intentional. In phenomenographic analysis, the researcher looks for the conceptual meanings of expressions in the data. Johansson et al. (2006) have explained that the process of exploring and identifying internal relationships from the perspective of the individual's own understanding is essential for identifying this meaning.

The interplay, however, is a complex and nonexplicit relation. In a conversation, the conceptual meaning of an expression can be elucidated by encouraging the speaker to reflect on the intended meaning of the expression that has been made. Anderberg (2000) explained the intentional-expressive approach for phenomenographic interviews where interviewees' conceptual meanings are clarified and confirmed systematically to obtain valid data. An example of how the approach was used to clarify the intended meaning of an expression made by an interviewee during a phenomenographic interview is given in the example study later.

Generalizability and transferability of findings

Issues

Generalizability in research generally refers to the extent to which the findings obtained from a specific sample are representative of the target population. Larsson (2009) has argued that this monist view is not useful in qualitative research. There is extant debate in the literature as to whether generalizability is a relevant criterion for evaluating the quality of qualitative research and also whether it is possible to generalize qualitative research findings. For instance, Schwandt (1997) has argued that meanings of complex phenomena are context specific and that there are no context-free meanings. Thus, there is no intention to make generalization in qualitative research. Others, like Mason (2002) and Silverman (2000), argued that generalization in the sense of wider applicability is not only possible but also desirable. Larsson (2009) has advanced possibilities of generalization through the researcher's maximizing variation, enhancing reader identification of context similarity and recognition of patterns. Spencer et al. (2003) give examples of a number of ways in which qualitative findings can be generalized; for example, in case-by-case generalization, which can be achieved through thick description.

Ways of addressing the issues

Qualitative research pursues diversity in meanings of complex phenomena in the interpretive analysis of data obtained from talk or observation in a particular context. Generalizability may be more appropriately considered in terms of transferability, which is the extent in which findings can be used or applied in other contexts. This type of generalizability is also known as external validity (Kvale, 1989). There is, however, a distinction between external validity and transferability. While they are both concerned with the use or application of research findings in other contexts, external validity is the responsibility of the researcher. The researcher can enhance the external validity of the research by providing sufficient information for users to extrapolate and be in a better position to make their own transferability judgments (Miyata & Kai, 2009).

Specific to phenomenographic research, Johansson et al. (1985) have identified the application of phenomenographic findings in learning contexts to bring about qualitative changes in the conception of a phenomenon. Bowden (2000) described the application of developmental phenomenographic findings in formal education contexts. Finally, if the transferability of findings is the motivation of a study, it would be important that the research design considers the possible contexts and the extent in which the findings can be usefully applied at the outset of the study and also in determining the scope and adequacy of the selection of participants.

Objectivity and reflexivity

Issues

There are three commonly raised issues of objectivity that relate to qualitative research: (a) the engagement and interaction between researcher and participants in the research process, (b) the fact that the researcher is not independent of the phenomenon under study, and (c) that judgment by the researcher is required, especially in the interpretation of data.

However, it has been argued in the literature that objectivity is an issue when the researcher's influences are ignored (Haraway, 1991; Malterud, 2001). Lincoln and Guba (1985) further argue that the issue of objectivity is not whether the researcher has influenced the research process or a denial of influence, but how it is addressed. They advocate a commitment to reflexivity as a way of addressing the issue.

Ways of addressing the issues

Reflexivity is when a researcher identifies his or her own preconceptions that are being brought into the research at the outset and then systematically questions at each stage of the research process as to how to minimize the effects and whether the effects have been sufficiently dealt with. The researcher should document fully and explicitly each stage of the research process so that readers can make a judgment. Thus, the commitment to reflexivity is more than the declaration of researcher's preconceptions that have been brought into the research. In practice, the researcher recognizes his or her own preconceptions and takes deliberate measures systematically to minimize their influence on the research process and documents these clearly.

Reflexivity deals with issues of researcher objectivity. Issues of data objectivity have been explicitly dealt with in a few studies in the phenomenographic literature. The intentional-expressive approach for phenomenographic interviews encourages interviewees to reflect on and confirm the intended meanings in the expressions that they have used. Svensson et al. (2006a) explained that data obtained by the intentional-expressive approach for phenomenographic interviews are objective as the meanings therein are from the interviewees' perspectives constituted by their own understanding of the phenomenon of interest.

Reliability

Issues

A widely accepted definition for reliability is the extent in which the findings of a study can be replicated. It may be argued that this concept of reliability cannot be readily applied to qualitative research because the social world is unstable and that a particular research setting may change from the experience of being studied. Morse (2006), however, has argued that some forms of reliability checking are necessary especially where analytical accuracy is paramount, for example, in conversational analysis. She further argued that the emergent nature of qualitative evidence is expected to change over time therefore there could be value in replication. This, however, is not done for the purpose of re-producing prior findings, but to revisit or to reexamine a topic or a phenomenon after an interval with the view of making a fresh appraisal. A relevant example is the Marton, Dall'Alba, and Beaty (1997) phenomenographic study, which identified six qualitatively different conceptions of learning in which five of them were identical to those previously identified in Säljö (1979). Other approaches, such as triangulation, can also increase the understanding of complex phenomena. Inconsistencies in the outcomes from the various approaches on the same phenomenon, however, should not be seen as a failure in reliability (Malterud, 2001).

How does the concept of reliability sit within phenomenographic studies? Marton (1986) and Säljö (1988) have argued that the outcome space in phenomenographic studies is a form of discovery that has arisen from a rigorous process of transcript-reading iterations, analysis, and validation with data, and that such discoveries do not have to be replicable. Sandberg (1997) concurs and further explains why interjudge reliability and member checking are not appropriate verification methods for phenomenographic findings as they overlook researcher procedures. The rejection of commonly used verification approaches for evaluating the reliability of phenomenographic findings, however, begs the question of: How, then, can the reliability of phenomenographic results be demonstrated?

Ways of addressing the issues

In addressing the issue of reliability of phenomenographic findings, Sandberg (1997) sees relevance in the reliability of the interpretative process where the researcher exercises interpretative awareness and maximum fidelity to the data. The researcher's interpretative awareness is when the researcher acknowledges and explicitly deals with his or her own preconceptions throughout the research process. Ashworth and Lucas (2000) added that the researchers must deliberately set their presuppositions and biases aside ³ to engage fully with participants' lived experiences to understand their conceptual meanings.

Researcher interpretive awareness is thus similar to the concept of reflexivity, and the focus on the research process rather than the outcome is consistent with the view of Morse et al. (2002). In addition, it is important that researchers document and explain clearly how they have practiced interpretative awareness so that the reader can make a judgment about the research process and assess the reliability of the findings.

Research ethics and the integrity of the researcher

The ethical conduct of the researcher is an important attribute of research quality. Most institutions have a code of research ethics which requires the researcher to maintain the confidentiality of participants and institutions. Preserving the anonymity of participants and their institutions by using pseudonyms and disguising locations to prevent recognition of identities are common practices. Secure storage and authorized access to personal data are other important measures to ensure confidentiality. Typically, there are also measures to ensure that participants are not coerced and that they have agreed voluntarily to participate in the research and are free to withdraw from participation without concern of retribution. ⁴ Researchers are usually required to explain to participants the nature and purpose of the study and obtain their informed consent before participation.

Kvale (1996) recognizes good craftsmanship, in continually checking, questioning, and theoretically interpreting the data and findings, as an important element of validity. Researcher creativity, skill, and insights are also recognized as critical inputs toward the quality of the research. The researcher, however, has the obligation to ensure that the data are accurate and that the findings are presented accurately, in full, and honestly. A responsible researcher must also consider the effects and consequences that the interpretations and conclusions of the research can have on people or groups, such as minority or marginalized groups. Finally, integrity and truthfulness are paramount qualities of the researcher and the ultimate legitimization of the research.

Promoting rigor and quality in each stage of the phenomenographic process

Phenomenography is more than a method for gathering and analyzing data, and reporting findings. It is integral to the overall research. Thus, the consideration of quality in phenomenographic research begins at the outset of the study, from stating the research question(s) and justifying the appropriateness of the phenomenographic method, and at each stage of the research process through to the reporting of findings.

An example study is used here to provide a context for the discussion. The research question of the study is: What are students' conceptions of accounting work? The object of study is students' conceptions and the phenomenon of interest is accounting work. Accounting work is a complex social phenomenon characterized by the various fields of specialization and is carried out in rapidly changing business and regulatory environments. Employers and professional accounting associations would like accounting graduates to be more work-ready and be able to cope with the complexities of work and to meet the expectations of the accounting profession. The purpose of the study is to inform curriculum development especially in revising the traditional knowledge-based curriculum.

Justifying the phenomenographic method

A research method that is internally consistent ensures that the research design is defensible (Spencer et al., 2003). The example study justifies the phenomenographic method by first establishing the consistency between the object of study and phenomenography by referring to Svensson's (1997) definition of phenomenography as a research orientation that studies peoples' lived experiences and conceptions. It further reinforces the suitability of phenomenography by referring to Säljö (1996) who emphasizes the suitability of phenomenography for studying complex social phenomena. Finally, the example study recognizes the consistency between the relational assumption of conceptions and the epistemological approach of phenomenography where the researcher engages nondualistically with participants to explore their experiences.

The appropriateness of method can be further enhanced by the method's ability to satisfy the stated purpose of the study. Phenomenography describes the collective variations of participants' conceptions of the phenomenon of interest. The descriptive phenomenographic findings of the example study provide insights into how students understand their future work and the nature and scope of their understanding. The example study links the descriptive form of phenomenographic findings and the purpose of curriculum use by explaining that less developed student conceptions can be identified as targets for development, whereas, more developed conceptions can be used as target learning outcomes in the curriculum.

Selecting participants for the study

Phenomenographic results report the different conceptions of the phenomenon in categories. The categorization process entails an abstraction of data in terms of similarities and differences. Marton and Booth (1997) advocate maximizing the conceptual variations of participants to ensure adequate data for deriving an optimal set of categories. Large amount of data, however, could lead to superficial analysis or data management problems and they may not necessarily extend the application of the findings. The nature of the research question, the quality of the data and the intended application of the findings are relevant factors when considering the number of participants for a study.

The student participants in the example study have a wide range of characteristics, such as different demographic backgrounds and academic abilities, with and without work experience, and the different stages in their undergraduate program of study. In the example study I have carefully documented the characteristics of the participants and explained that the spread of characteristics is intended to maximize conceptual variations in the data. The clear description of participant characteristics also allows readers to judge the validity of the data. Finally, the example study recognizes the global nature of accounting practice and education and points out that the spread of participant characteristics has further enhanced the application of the findings in other contexts.

Collecting data by interviews

Data for phenomenographic studies are most commonly obtained by audio recorded interviews. The purpose of phenomenographic interviews is to explore the lived experiences of interviewees and their conceptual meanings of the phenomenon of interest. The intentional-expressive approach (Anderberg, 2000) is a useful and systematic interview strategy for elucidating and confirming the conceptual meanings in the expressions that interviewees have made. In the intentional-expressive approach, interviewees are first asked questions regarding the phenomenon of interest. Follow-up questions are then asked to encourage interviewees to reflect on the conceptual meanings of the terms or phrases in the expressions that they have used. The following is an example of a follow-up question from the example study:

Interviewee: Accountants work in very different industries, tax, audit, corporate accounting … If you ask me what accounting is, I'll say, it's a human science.

Interviewer: Human science, what does it mean?

Essentially, in phenomenographic interviews, the phenomenon of interest is explored jointly between the interviewer and interviewee (Marton, 1994). This process necessitates engagement and interaction between the interviewer and interviewees and the influence of the interviewer can be deemed as a weakness of the method. It is, however, important to recognize and preserve the characteristic of joint exploration between interviewer and interviewee in phenomenographic interviews. In the example study the interviewer paid practical regard in the interviews to minimize interviewer influence as follows:

1. Attention was given to the expressions used by interviewees and assumptions were not made about their meanings even if they seemed obvious but to clarify their intended meanings by asking follow-up questions.

2. The researcher avoided introducing new terms into the conversation and refrained from correcting the interviewee with more accurate expressions.

3. After asking a question, the researcher gave the interviewee the time and space to reflect and talk. The researcher consciously avoided showing facial expression of agreement or disagreement at the interviewees' responses but remained present and listened attentively and empathically.

4. The researcher also avoided asking leading questions. For instance, interviewees were not asked whether they think actual accounting work is different from what they learn at university. Instead they were asked to describe what they think accounting work is and what they think accountants do. In addition, questions about what they think or know about accounting work were asked in different ways to further mitigate revealing any researcher bias and for obtaining more elaborate descriptions and richer data.

The example study conducted three pilot interviews prior to the actual interviews. ⁵ The pilot interviews were reviewed to ensure that the phenomenon under study has been communicated clearly to interviewees. Pilot interviews also help to improve interview techniques especially for new researchers. The pilot interview data, however, were not used in the analyses.

Finally, every interview is unique in the joint context of time, place and presence. The consideration of the contextual elements in interviews is important for the subsequent stages of the research process, particularly for transcription and analysis as discussed below.

Transcribing interviews

It is common practice in phenomenographic studies to transcribe audio-recorded interviews verbatim to obtain data for analysis. Transcription is the interface between oral and written data. It is also a juncture of the research process where the reliability and validity of the data may be questioned.

The reliability of transcription depends on the clarity of instructions for the transcription. The level of details to be transcribed depends on the intended use of the transcripts. In phenomenographic analysis, the focus is on the intended conceptual meanings of interviewees in their expressions. Thus verbatim transcription, which does not allow interpretation or restatement, is acceptable.

Transcription also changes oral discourse to text. Spoken language is structured and accomplished differently from written text, and they each have their own set of rules. Kvale (1996) has warned that the linguistic complexities inherent in transforming oral language to the written form may change meanings as meanings are contextual and the context of the interview is lost in transcription. Furthermore, Barnacle (2005) is concerned that there may be aspects of experience that cannot be or are not expressed. Consequently, the researcher relying solely on transcripts for phenomenographic analysis runs the risk of misinterpretation.

A careful researcher should recognize the limitations of transcription and take steps to address them. In the example study, the primary researcher, who is also the sole interviewer, mitigated losing touch with the original interview contexts by reflecting on the interviews shortly afterward and made mental and written notes of relevant contextual features of the interviews. She also checked and completed the transcriptions from outlines that were done by an assistant. Finally, she listened to the recordings several times both before and after transcription.

Analyzing data

The aim of phenomenographic analysis is to derive conceptions of the phenomenon of interest from the data. So far, the paper has explained the meaning of conception in the phenomenographic context, but what is a conception empirically? What do conceptions look like in the data, and how does the researcher find them?

A conception has “two intertwined aspects: the referential aspect, which denotes the global meaning of the object conceptualized; and the structural aspect, which shows the specific combination of features that have been discerned and focused on” (Marton & Pong, 2005, p. 335). During analysis the researcher looks for these aspects in the data to interpret conceptions.

Phenomenographic analysis involves reading and rereading the transcripts throughout the process. The aim is to look for qualitatively different conceptions of the phenomenon of interest collectively rather than the conceptions of individual participants. Thus, the transcripts are not analyzed or interpreted individually. They collectively constitute the overall data where the meanings are interpreted in relation with the others. ⁶ In the example study, data analysis commenced after all interviews were completed. The primary researcher read and reread the whole set of transcripts initially to be closely familiar with the data. She was aware of the importance that opinions about conceptions were not formed at this stage. After the initial readings, she looked for qualitatively different global meanings that were evident in the data through a process of coding, revision, and recoding.

A global meaning was identified when there was sufficient evidence that a particular overall meaning has been expressed by interviewees. For each global meaning (putting the others on hold), the associated structural aspect or features supporting the global meaning were identified. This process presupposes quality data in that interviewees' intended meanings have been elicited and clarified during interviews to facilitate interpretation. The process was repeated for each global meaning that was evident in the data. This stage of the analysis also involved several revisits and readings of the transcripts, confirming the meanings with both the immediate context of surrounding statements and the transcript as a whole. This is a critical stage of the analysis and the primary researcher and her supervisors in the example study met several times to discuss and revise the global meanings and their structures and also to confirm that the interpretations were validly derived from the data. ⁷ The final global meanings are the categories of conceptions that are reported. The NVivo program was used in the example study to manage the data and systematically track the stages of the analysis to enhance the rigor of the process.

Finally, phenomenographic analysis and interpretation is a complex and demanding process, especially for new researchers. Therefore, the supervision from an experienced phenomenographer is both invaluable and necessary for quality assurance.

Reporting findings

Phenomenographic findings are reported in an outcome space that describes the categories of qualitatively different conceptions of the phenomenon. Marton and Booth (1997) gave three criteria for judging the quality of an outcome space: (a) There must be something distinctive about the conception in each category. (b) The categories are optimal and parsimonious. (c) The relation between the categories is clearly stated.

The categories of conceptions are second-order descriptions as they are descriptions of other people's conceptions of the phenomenon from their perspectives (Marton, 1981). However, the researcher is not indifferent to the phenomenon or the elements of the overall research. The researcher's voice in reporting the findings is, therefore, inevitable. That is why it is important that there is a commitment to reflexivity throughout the research process, including the reporting of findings.

Quotations from the interviews are usually used to support and clarify the meanings of the reported conceptions. In the example study, the researchers examined the texts surrounding the selected quotations carefully to ensure that the intended meanings of interviewees are conveyed in the selected quotations.

Finally, when reporting findings, the intended meanings of the researcher might not be clear to readers. Concerning linguistics, there is also danger in unreflected and unguarded language use (especially by new researchers) in the verbal representation of findings. Further, readers, unlike in interviews, cannot clarify with the researcher regarding the intended meanings of their expressions in the report. The researchers in the example study tried to mitigate these issues by continuing to use the intentional-expressive approach for reporting findings. This was done empathetically by consciously reflecting on the use of terms, and explaining the intended meanings as clearly as possible in the reporting and discussion of the findings.

Conclusion

Phenomenography originated as a descriptive approach to study people's conceptions of aspects of phenomena in the world around them. It has a history of more than 30 years. During these years, it has gained much popularity in educational research. However, some fundamental conceptual underpinnings of phenomenography have come under scrutiny and aspects of validity and reliability of its empirical approach have been questioned.

This paper has benefited immensely from the arguments and insights of previous researchers about methodological issues and their resolutions. Its main contribution lies in its holistic approach in discussing the quality of research specific to empirical phenomenography. This paper has integrated three related aspects of quality; namely criteria, issues and practical ways of addressing the issues in each stage of the phenomenographic research process. New phenomenographic researchers especially will find it a useful and practical guide.