Abstract
There is limited methodological guidance on systematically integrating numerical-style responses with open-ended questions in online surveys. This paper addresses this gap by presenting the Scale, Choice and Insight (SCI) approach, which applies to both qualitative and mixed-methods research contexts. The SCI provides a framework for combining researcher-constructed Scale items, where participants Choose their responses immediately followed by linked, open-ended Insight questions within a single survey. This integrated design captures initial attitudes or priorities alongside the contextual participant reasoning behind them. The paper outlines the theoretical justification, practical implementation steps and an analytical framework for the SCI approach, with illustrated examples. By systematically linking structured data with qualitative explanations, SCI enhances methodological rigour, transparency and interpretative depth in online survey research.
Keywords
Introduction
Digital technologies have impacted how and where researchers conduct research. This is particularly relevant for surveys in terms of design, distribution and analysis. Broadly defined, survey research involves the systematic collection of information from a sample of individuals through their responses to questions (Check and Schutt, 2012). Online surveys are often adopted because of their capacity to engage geographically dispersed participants, convenience and efficiency (Braun and Clarke, 2013; Braun et al., 2020; Vehovar and Manfreda, 2017). This paper argues that despite online surveys being commonly used, there is a need for clearer and more structured methodologies concerning the creation and analysis of them. It does this by presenting a systematic methodology which integrates numerical-style items with open-ended questions. Such integration benefits both qualitative and mixed-methods studies, as well as broader survey methodology, by combining structured measurement with interpretive depth. This builds on previous arguments presented by Busetto et al. (2020) and Creswell and Plano Clark (2018), that using both qualitative and quantitative aspects in the research design process will enable researchers to address a greater range of research problems. Survey methodologies frequently focus on either open-ended questions, emphasising rich textual data or largely quantitative survey designs, aiming for generalisability through standardisation (Jansen, 2010; Vehovar and Manfreda, 2017). No established methodologies explicitly guide researchers in combining numerical-style scales with qualitative insights in a single and cohesive survey design. The Scale, Choice and Insight (SCI) approach responds to this gap and is designed for use in qualitative, mixed-methods and survey research contexts alike. While qualitative researchers may find it particularly valuable for its capacity to capture contextualised reasoning, SCI also supports broader applications, such as enhancing measurement validity and reducing interpretive ambiguity in large-scale surveys.
The SCI approach integrates complementary question types that function in two key stages. First, Scale and Choice items capture initial attitudes or perceptions via ordinal or nominal responses to researcher-constructed questions. Second, Insight questions gather qualitative explanations that contextualise these initial responses:
The primary novelty of SCI lies in its explicit methodological clarity in integrating these components to capture nuanced qualitative data in both a systematic and rigorous fashion. While established mixed methods designs like explanatory sequential approaches also follow quantitative data with qualitative exploration (Creswell and Plano Clark, 2018), SCI distinguishes itself through its specific structure: Scale and Choice items are followed by the immediate placement of qualitative Insight questions within a single survey instrument for all participants. This focus on immediate contextualisation, rather than separate follow-up phases or potentially brief embedded questions, provides explicit guidance for researchers seeking integrated interpretative depth. Furthermore, the method’s structured analytical process facilitates a bi-directional linking, allowing researchers to connect trends derived from the Scale and Choice responses (specific participant selections on the scales) with themes identified in the Insight data, and vice versa.
To demonstrate the practical and methodological value of the SCI approach, this paper draws on empirical examples from two studies, both conducted in England, applying this method. Study 1 examined educators’ experiences with the Reception Baseline Assessment RBA; Meechan et al., 2022) and Study 2 explored practitioner perspectives on changes to child-to-adult ratios in nurseries (Meechan and Brabury, 2024).Both contexts involved complex policy issues and diverse participant viewpoints, making them suitable demonstrations of an SCI approach’s capacity to capture nuanced perspectives.
Theoretically, this paper is rooted in an interpretivist perspective, recognising that participants’ responses to survey items are socially constructed, context dependentand reflective of individual and collective interpretations of their lived experiences. Moreover, acknowledging the potential to extend into methodological pluralism (Williams, 2000), this paper discusses how further quantitative analysis of the Scale and Choice responses (i.e. the ordinal/nominal data reflecting participants’ selections on the scale items) could contribute to mixed-methods analysis, offering richer understandings that transcend traditional qualitative thematic analysis alone (Creswell and Plano Clark, 2018; Greene, 2007). This paper is structured as follows: First, a review of existing literature and methodologies highlights current limitations in online survey designs. Next, the theoretical justification for the Scale and Choice, and Insight components of this approach is presented. The subsequent section then provides detailed guidelines for implementing the SCI methodology, illustrated through specific examples from Study 1 and Study 2. The analytical framework is then presented with specific examples drawn from Study 1 to demonstrate the systematic integration of the Scale and Choice data with the qualitative thematic Insight data. A discussion section then explores the strengths, limitations and future potential of the SCI approach. Finally, a conclusion is offered and an invite made for other researchers to empirically test and offer further conceptualisations of the SCI approach going forward.
Literature review and justification
The interpretivist tradition, underpinning many qualitative research methods, emphasises understanding individuals’ subjective experiences and interpretations within specific social contexts (Tisdell et al., 2025; Williams, 2000). This philosophical stance positions qualitative researchers to explore the complexities and nuances of human experience, recognising that participants’ meanings are socially constructed and contextually bound (Guba and Lincoln, 1994; Schwandt, 2000). This perspective provides the rationale for adopting qualitative methods, particularly surveys designed to capture participants’ narratives and reflections. Qualitative online surveys have gained increasing attention because of the geographical reach that digital platforms now grant researchers, especially in relation to hard-to-access populations (Braun et al., 2020; Hewson, 2017). However, further literature concerning structured guidance for designing and analysing online, but qualitative-leaning, surveys is needed. There is often an assumption that qualitative surveys should rely on open-ended questions (Jansen, 2010). Although such approaches can be effective in capturing rich narrative data, they can sometimes lack analytical precision and structure (Vehovar and Manfreda, 2017). Quantitative based and structured online surveys contrastingly utilise closed-ended questions and offer precision, but sacrifice the participants’ unadulterated perspectives that are seen essential to qualitative inquiry (Busetto et al., 2020; Creswell and Plano Clark, 2018). Thus, qualitative researchers have to consider the trade-offs between capturing detailed narrative data and maintaining methodological structure and analytical clarity.
Braun et al. (2020) emphasise the value of online qualitative surveys, advocating for their flexibility and capacity to provide a ‘wide-angle lens’ on social issues. Their framework highlights the potential but provides limited guidance on systematically creating and integrating questions in such surveys. Structured quantitative-type measures, such as ordinal Likert-type items or nominal responses (e.g. Yes/No or selection from a list of categories), are not considered alongside the open-ended narrative responses. This can be viewed as a methodological gap for a researcher who is seeking to combine indicators of attitudes or opinions as well as interpretive narratives. Other prominent researchers have echoed similar concerns. Creswell and Plano-Clark (2018), leading scholars in mixed-methods research, advocate for integrating quantitative measures with qualitative narratives to maximise interpretative and analytical depth. They argue that combining structured and open-ended data enhances both depth and rigour, enabling richer insights than either approach alone. Yet, existing qualitative survey methodologies rarely guide how best to achieve this integration in online contexts (Vehovar and Manfreda, 2017).
A related method within the survey design literature is web probing (Neuert et al., 2023). Web probing typically involves following survey questions with open-ended probes that invite respondents to explain their reasoning, interpretations or experiences (e.g. ‘Why did you select this answer or answer in this way?’). It is used in pretesting to evaluate question design, identify measurement issues and refine survey instruments. While SCI, like web probing, integrates quantitative and qualitative elements within a single instrument, its purpose is distinct. Web probing is diagnostic and aims to improve the survey itself. SCI, however, is focussed on both design and data collection. In SCI, the combination of Scale and Choice items with Insight questions is central to the research aim, not a design function for instrument testing. This distinction positions SCI as complementary to web probing and expands the methodological literature on survey design.
Building on this distinction, the SCI approach provides practical guidelines for structuring and integrating Scale and Choice items and Insight questions within survey design. Whereas large-scale research syntheses, such as Hattie’s (2008, 2023) work on educational impact, focus on identifying trends by aggregating quantitative data, the SCI approach moves beyond this to contextualise individual interpretations. The theoretical justification for an SCI approach is rooted in interpretivism, but the structured nature of the Scale and Choice component provides opportunities for methodological pluralism in the analysis. Such an approach will be considered further in Section 4 of this paper and supports a mixed-method approach to analysis.
The SCI methodological components work together sequentially and represent two core functions that are theoretically justified below:
Scale and choice items: Making attitudes and priorities visible
The primary component of the SCI approach utilises structured, numerical-based questions. Both ordinal rankings of opinion (e.g. Likert-type scales) or nominal type responses (e.g. Yes/No or selection from a list of categories) can be utilised. These types of question systematically capture participants’ attitudes and perceived priorities. This often makes abstract concepts more tangible or ‘visible’ through numbers. Likert-type scales (Likert, 1932) offer a structured method for participants to express subjective attitudes numerically. From an interpretivist viewpoint, these captured attitudes are understood not as objective, fixed truths, but as valuable indicators reflecting participants’ socially constructed meanings and subjective interpretations within their specific contexts (Williams, 2000). While methodological discussions continue regarding the precise statistical treatment of numerical based data, its utility within an interpretivist framework like the SCI lies in pragmatically mapping participants’ attitudes relevant to the research questions (Clason and Dormody, 1994). Furthermore, when using ordinal ranking (Stevens, 1946) participants can express the relative significance or priority they assign to different issues or experiences. Allowing for this type of response via the Scale and Choice item aligns with interpretivist methodologies because the act of choosing is inherently subjective, context-dependent and reflective of participants’ constructed value systems. It helps researchers understand not only what participants perceive, but how they position these perceptions relative to each other. Together, the Scale and Choice items generate structured data (numerical attitudes reflecting participant priorities) that indicate participants’ lived experiences from their perspective. Such indicators provide a clear starting point to prompt deeper contextual exploration and interpretation. This is achieved through the following qualitative based and open ended Insight component.
Insight: accessing subjective meanings and explanations
Central to the SCI methodology and rooted in its interpretivist foundation (Williams, 2000), is the use of the Insight component that utilises open-ended questions. Interpretivism prioritises understanding phenomena through the subjective meanings and interpretations participants construct within their specific social contexts (Guba and Lincoln, 1994; Schwandt, 2000). Open-ended questions provide the most direct pathway to access these rich, nuanced perspectives, capturing the ‘why’ behind participants’ attitudes and priorities in their own words to ensure their voice remains central. Within the SCI approach, Insight questions are strategically designed to follow the Scale and Choice items, prompting participants to articulate the rationale, context, meanings and interpretations underpinning their previous responses (Braun and Clarke, 2013). This component provides the essential interpretive depth missing in quantitative survey design and analysis. By structuring the survey so that Insight narratives relate directly back to the Scale and Choice data, the SCI approach enhances the analytical rigour and interpretative clarity. This systematic connection allows researchers to contextualise numerical patterns and explore potential ambiguities or contradictions revealed between the different data types. It also achieves a more holistic and trustworthy understanding than either isolated quantitative data or purely unstructured qualitative responses might permit alone. It directly addresses the challenge of integrating structured data with rich participant meaning in online research by ensuring a diversity of participant voices, including minority or alternative perspectives, are captured to support or contest the numerical data.
The explicit theoretical justification provided here positions the SCI approach as uniquely structured for researchers aiming to balance interpretive depth with analytical clarity. Its explicit integration of the Scale and Choice Items, alongside Insight questions, bridges the methodological gaps identified in current qualitative and mixed method survey methodology. The SCI approach uses structured design and systematic analysis to promote analytical clarity and rigour. Simultaneously, by integrating qualitative insights, it maximises interpretative potential and contributes to methodological transparency. The next section of this paper provides practical guidance for implementing this SCI methodology.
Practical guide to implementing the SCI Approach
The overall process of the Scale, Choice and Insight (SCI) methodology, from initial research question through design, data collection, analysis and integration to interpretation, is visually summarised in Figure 1. The following subsections detail the design Steps 2 and 3. This is done by using examples from Study 1 and Study 2.
The Scale, Choice and Insight (SCI) methodological process.
Designing scale and choice items
Step 2 (Figure 1) should be considered in two parts: Part A and Part B. Step 2, Part A involves creating structured items where participants will indicate their position or viewpoint. While this approach lends itself to Likert-type items (Clason and Dormody, 1994), this component can also utilise other structured formats. For example, a basic Yes/No item or the offer of a range of predefined categorical choices. The selected format should depend on the specific information sought to answer the research question in Step 1. The key characteristic is that the researcher constructs the item (Scale), and the participant then indicates their viewpoint ordinally or nominally (Choice) in the response. Three examples of varying Scale and Choice type items are shared below: Example 1 from Study 1: 5-Point Likert Type Scale *It is acknowledged that this question stem is not perfectly aligned with the agree/disagree response scale. A statement-based stem would typically follow best practice in this example. Example 2 from Study 2: Categorical Choice Scale Example 3 from Study 2: 3-Point Likert Type Scale
In examples 1–3 above, while the researcher constructs the scales and response of the item, it is the participant who selects their subjective response. These structured responses then serve as the anchor for the subsequent Insight question.
Designing insight questions
Following the structured Scale and Choice items, designing the Insight questions (Step 2, Part B in Figure 1) is crucial for capturing qualitative depth and the participants’ perspective further. These open-ended questions prompt participants to provide narrative explanations, justifications or context for their preceding Scale and Choice response(s). This space allows participants to articulate the ‘why’ behind their viewpoint in their own words and within the context of having responded to the relevant scale and choice-based items. This provides insight into the participants’ immediate and subjective interpretations. Using online survey platforms means that routing or conditional logic features can be utilised (Toepoel, 2017) to present specific Insight questions based on the participant’s previously chosen response. Designing effective Insight questions should involve a clear correlation with the research question that the SCI block aligns to. Examples 4–6 below illustrate different strategies to achieve this: Example 4 from Study 2: Using Conditional Logic/Routing This approach targets insights based on specific Scale responses.
(This directly asks for elaboration based on a specific answer to the Scale Item, using survey routing to show the Insight question only to relevant participants).
Example 5 from Study 1: Following a Block of Related Items Insight questions can also capture broader reflections relating to a group of preceding Scale items, as shown below.
(This asks for reflection on the topic covered by the preceding block of Scale Items, shown to all participants who answered that block).
Example 6 from Study 2: Addressing the aligned Research Question Insight questions can also address a relevant topic directly linked to the aligned research question and not necessarily to the immediately preceding item.
(This seeks qualitative insight on a specific issue relevant to the aligned Research Question of the survey, shown to all participants, regardless of specific preceding answers).
Choosing the most appropriate linking strategy, whether conditional, block-focussed or research question-specific, should be guided by the specific research question and the type of qualitative depth the researcher seeks in relation to the preceding Scale and Choice items.
Integrating SCI components within the survey
The structured integration of the SCI’s components needs further consideration. The placement of the Insight question should facilitate a clear link to the relevant structured response(s). It can be placed immediately after a single Scale and Choice item (as in Example 7) or it can follow a block of related items to capture broader reflections on a topic (as illustrated in Example 8). This strategic placement creates a coherent ‘Scale and Choice + Insight’ flow, allowing participants to reflect immediately on their Scale response(s) while providing their Insight narrative. This link makes it explicit for both the participant when responding and the researcher when analysing.
Examples from the studies illustrate these integration structures: Example 7 from Study 2: Conditional Scale + Immediately Following Insight
(This shows the Insight question directly following and conditionally linked to a single preceding Scale Item).
Example 8 from Study 1: Block of Scales + Following Insight
(This shows the structure where an Insight question follows a block of related Scale items, asking for broader reflection on the overall topic covered).
This strategic sequencing ensures the qualitative data directly contextualises either the participant’’s specific selection (their ‘Choice’ response) or their overall perspective emerging from a block of related items, strengthening the link between the structured and narrative data for subsequent analysis and interpretation. The structured and explicit methodological framework that has been described so far positions SCI as a practical approach to qualitative online survey research. Each component of the Scale, Choice and Insight complements the other. This provides researchers with systematic guidance in survey design, data collection and integrated data analysis. This structured methodology is innovative in guiding qualitative researchers towards deeper interpretative insights while maintaining rigour, clarity and methodological transparency.
Mapping survey design, checking question ratios and accounting for insight item non-response
To ensure content and construct validity of the survey, the SCI methodology explicitly maps the designed survey questions to the overarching research questions that they aim to address (Step 3 in Figure 1). Such mapping aligns with principles of good survey design and ensures that the objectives of the research translate to specific survey items/questions (Fowler, 2014). Methodological transparency is enhanced by making the alignment between research questions and data collection instruments explicit for both the researcher and the reader. This ensures all research questions are covered by the corresponding Scale and Choice items and Insight questions, thus promoting both breadth and depth in data collection. Visually linking specific Scale and Choice items to an RQ, allows the initial data analysis to consider patterns and attitudes across the sample for that specific question. Furthermore, such mapping ensures qualitative depth is systematically incorporated into the design. This alignment is further represented through Table 1 as an example. It illustrates the number of each question type aligned with each research question in the Study 1 example.
Example of aligning S + C type items and insight questions with the research questions from study 1.
The mapping demonstrated in Table 1 allows researchers, when constructing an SCI based survey, to consider the balance of Scale and Choice type items compared to Insight questions and in further relation to the specific research question. In Table 1, the overall ratio is approximately 5 Scale and Choice items for every 1 Insight question, with these groupings referred to as SCI blocks. However, the ratio for individual research questions in the example ranged from 4.7:1 to 6:1. While this example should not dictate universal practice, the observed range suggests a potential middle ground that balances the need for structured data alongside qualitative explanation. The purpose of calculating the ratios in Table 1 is to demonstrate that extreme ratios of Scale and Choice items to Insight Questions are not recommended. It is suggested that a low 1:1 question type pairing would offer limited breadth. At the other end of the ratio spectrum, excessively high ratios may lead to diluted insights. It is therefore proposed that the concept of an SCI block, the use of multiple Scale and Choice items followed by a single Insight question, enables researchers to maintain breadth while still achieving qualitative depth. Calculating the ratio of Scale and Choice items to Insight Questions in relation to each Research question should provide an indication of coverage and further point of reflection for the researcher relating to the design.
Another consideration when determining these ratios is the potential cognitive burden on participants. Open-ended Insight questions require more mental effort and time than closed questions (Höhne, 2019), such as the Scale and Choice items. Engagement with Insight questions may also relate to the device being used to complete the survey. Typing longer, open-ended responses on mobile devices can be more effortful, potentially leading to shorter answers or higher nonresponse rates in mobile-first samples. Where feasible, researchers should explore alternative input modes, such as voice-to-text responses (Revilla et al., 2018), which have been found to reduce burden and improve data richness in mobile contexts.
Based on the studies included here, a practical range of between 5 and 8 SCI blocks can typically be included in a survey designed to take no more than 15–20 minutes to complete. For longer or more complex surveys, additional Scale and Choice items may be included, but the researchers should be selective in how they group these into SCI blocks. It is key to ensure that each Insight question relates to the SC Items before it and is directly aligned to the research question. Empirical evidence on the optimal number or size of SCI blocks remains limited and it is therefore recommended that an SCI survey is piloted first to gather feedback on perceived effort and completion time before being distributed more widely. Future research would be welcomed on how varying SCI block size and number influences response quality, drop-out rates and participant engagement.
A final consideration concerns nonresponse to Insight questions. Within the interpretivist tradition, the focus is on understanding the interpretations and experiences expressed by the survey participants. This also means that missing responses to open-ended questions are part of the variability in such qualitative data generation. In this context, each narrative is a situated account of reality rather than an objective data point in need of aggregation. Whilst nonresponse is part of the co-constructed nature of the data, patterns in nonresponse can still hold interpretive value. For example, if specific Insight questions consistently receive fewer responses, this may suggest that their design, placement or complexity affects participant engagement. Following Lincoln and Guba’s (1985) emphasis on credibility and reflexivity, researchers using SCI should report observed nonresponse rates, reflect on possible reasons for them and consider their implications for the interpretation of the Insight data.
Analytical framework for integrating SCI data
The SCI approach, in its current form, proposes a two-phase analytical framework (Step 5 in Figure 1). This involves combining the descriptive statistical analysis of data from the Scale and Choice items (Phase A) with the qualitative thematic analysis of Insight data (Phase B):
Phase A: Numerical analysis
Data from the Scale and Choice items (i.e. the responses reflecting participant selections on the scales) are initially analysed descriptively (e.g. examining distributions, measures of central tendency and dispersion) to establish patterns of response aligned with the associated research questions. This initial step supports familiarisation with the numerical data. The SCI approach utilises descriptive analysis as the primary approach to Phase A. However, the numerical data could permit further quantitative exploration. For example, using chi-square tests to explore associations between categorical variables, employing regression models (suitable for ordinal or nominal data) or utilising non-parametric tests (like Mann-Whitney U or Kruskal-Wallis) to compare group responses are possible. However, the feasibility of such advanced analyses depends on sample size and the potential trade-off between statistical and interpretive goals. The larger samples required for robust statistical modelling could mean that the corresponding in-depth analysis of qualitative Insight data becomes impractical.
The emergence of AI-powered tools, however, presents a potential avenue to mitigate this challenge. Kantor (2024) identified modern LLMs as having much improved capabilities when compared to older auto-coding features, and this potential is being embedded into some of the qualitative analysis software. For example, the Nvivo 15 release in 2024 included the AI-assist add- on, which leverages large language models to summarise data sets and automate descriptive coding. Mortelmans (2025) explores such new features but also recognises their limitations, highlighting the importance of continuing researcher oversight if using them. This could be a key area of development for the SCI approach, demonstrating potential methodological pluralism. Such quantitative exploration would present the Scale and Choice responses as variables, enabling broad patterns to be uncovered. This aligns with the focus of large-scale research syntheses such as Hattie (2008, 2023). Regardless of methodological pluralism, the primary analytical purpose of the SCI approach is to anchor the Scale and Choice data to the qualitative Insight data in a more immediate way than typically possible with mixed method approaches. The decision of whether to undertake this additional quantitative analysis ultimately rests with the researcher and should be guided by the specific research questions being addressed.
Phase B: Thematic analysis
Insight responses are analysed qualitatively (Braun and Clarke, 2006) to generate themes that contextualise and clarify the patterns identified in Phase A. A detailed integration step (Step 6 in Figure 1) explicitly links themes derived from qualitative narratives directly back to patterns observed in the numerical data, producing richer interpretations. This explicit linking step enhances interpretative clarity and analytical rigour by integrating structured ordinal/nominal responses with qualitative insights (Creswell and Plano Clark, 2018; Williams, 2000). Studies utilising the SCI approach to date have utilised thematic analysis of the Insight Questions in response to the aligned Research Question. It should be noted that thematic analysis, as stated by Braun et al (2020) has the potential to be applied across all Insight data created, with the potential to offer further breadth and depth in the type and number of themes found.
Illustrating the SCI approach: Study 1
The introduction of the Reception Baseline Assessment (Study 1) marked a shift in early childhood education policy across England. The assessment of children’s literacy and numeracy skills upon entry into formal schooling was now standardised. Given the range of opinions surrounding national assessments and their implications for both educators and children, this provided an ideal context to apply and illustrate the Scale, Choice and Insight (SCI) approach as a methodology. Study 1 sought to explore educators’ perspectives on three central themes: the impact of the RBA on classroom practice, the perceived challenges of implementation and educators’ recommendations for improvement. By applying the SCI method, Study 1 demonstrates how a structured survey design can capture both numerical-style indicators of educators’’ attitudes (via the Scale and Choice component) and rich qualitative insights (via the Insight component) that reflect the lived experiences and perspectives of its participants. Educational professionals from across 47 local authorities in England (N = 70) participated. Specific examples from Study 1 will now be used to demonstrate how the Scale and Choice items were integrated with Insight questions to address the research questions.
Research Question: What are educators’ perspectives on the Reception Baseline Assessment?
➜
➜
➜
➜
➜
➜
Thus, this integration of data analysis (Figure 1, Step 6) has demonstrated connecting the quantitative pattern (the ‘what’) with specific and relevant qualitative themes (the ‘why’). This demonstrates that the high level of disagreement was not monolithic but was clearly linked to recurring concerns about the assessment’s limited scope and its perceived inadequacy compared to established observational practices. This linking provides a much richer, more nuanced and actionable interpretation of why educators felt the RBA did not improve their understanding, moving beyond the descriptive statistic.
Previous sections established the theoretical justification for the SCI approach as a response to methodological limitations (Section 2) and outlined the practical guidance for its implementation (Section 3). Building on this, the current section has demonstrated the methodology in practice through an example from Study 1. SCI’s structured integration links both ordinal scale-based and nominal responses with qualitative narratives. This approach produces interpretations more nuanced than those yielded by purely quantitative or unstructured qualitative data alone. Building on this methodological foundation and practical illustration, the final section will now discuss the broader strengths, limitations and future potential of the SCI approach for enhancing online survey research.
Discussion
As demonstrated so far, the Scale, Choice and Insight (SCI) approach offers a structured yet interpretatively sensitive methodology. It addresses the lack of explicit guidance for systematically integrating numerical-style (both ordinal and nominal) responses with immediate and linked qualitative explanations within a single, cohesive framework. This integration, encompassing both design (illustrated via Study 1 and Study 2 examples) and analysis (demonstrated using an example from Study 1), enables researchers to gain deeper and more nuanced insights into complex social phenomena. This is done by correlating patterns in structured data directly with the participant meanings behind them. This methodological clarity and structured analytical framework enhance transparency and rigour compared to more traditionally aligned qualitative survey approaches (Braun et al., 2020).
It is acknowledged, however, that the introduction of structure, particularly the use of researcher-defined Scale and Choice items preceding open-ended Insight questions, warrants further exploration within an interpretivist paradigm. A core tenet of qualitative research is often seen as minimising imposed structures to allow participant perspectives to emerge organically (Williams, 2000). Therefore, a critique of the SCI approach needs to consider whether the proposed structure unduly constrains or shapes the participant’s narrative. This is an aspect that has been constantly wrestled with and reflected on in developing the approach, especially as the origins of this design began from an interpretivist angle. It is important to recognise, however, that interpretivism itself is not a single doctrine (Scauso, 2020) but represents a broad paradigm encompassing philosophical stances and associated methodological approaches that should be debated.
The SCI approach attempts to navigate this tension by representing a specific methodology within the interpretivism paradigm. First, the structured design allows researchers to maintain a clear focus on addressing specific Research Questions across the participant sample. This can be challenging in purely exploratory open-ended surveys and overtly demonstrates a researcher’s choice in how they choose to answer the research questions and their positionality. Secondly, the SCI allows for participant subjectivity and ownership in relation to the structured items themselves. This is especially present in the scale-type items that produce ordinal data where the distance between points is subjective. The interpretation of ‘Agree’ versus ‘Strongly Agree’, for instance, rests firmly with the participant, not the researcher (Bishop and Herron, 2015). Similarly, when a participant makes a nominal choice (e.g. selecting ‘Yes’ over ‘No’), the selection is also a subjective act reflecting their personal interpretation of the question and their experience. SCI therefore honours this by capturing the participant’s self-placement (‘Choice’) on the Scale.
The immediate nature of the Insight question is also designed to capture the participant’s own rationale for the ordinal/nominal responses of the Scale and Choice items. This gives voice to the participant’s reasoning in situ. As already established, this notably differs from explanatory sequential mixed methods designs, where qualitative follow-up often occurs later and potentially with a subsample of the original participants. Such designs typically aim more broadly to explain aggregate quantitative findings (Creswell and Plano Clark, 2018). By allowing individual reasoning to be captured in this way, the diverse interpretations behind Scale and Choice selections offer a granular level of insight not possible in typical mixed method designs (Adhikari and Timisina, 2024).
The design focus of the SCI approach also differs from large-scale evidence synthesis approaches, such as Hattie’s (2008, 2023) Visible Learning, which prioritises identifying generalisable patterns of impact across broad populations through the aggregation of quantitative findings. The explicit linking in the SCI approach supports richer theoretical interpretations and demonstrates methodological pluralism by integrating structured ordinal and nominal responses with qualitative insights (Greene, 2007). The systematic process enhances analytical rigour and transparency. Furthermore, its flexibility allows potential adaptation across diverse research contexts beyond education. Limitations remain inherent in this balance, though and will be discussed next.
The SCI approach is dependent on participants providing rich Insight responses, but response rates for such open-ended style questions are often lower than for the structured Scale and Choice items. It is well-established that open ended questions, which require participants to formulate and articulate responses in their own words, are more cognitively demanding (Neuert et al., 2021). Using the Scale and Choice style items first will focus the participants’ attention on the specific area or subject being explored before they are asked to articulate their reasoning. This structure aims to set the context and prime relevant concepts for the Insight question that follows. Researchers must be mindful, however, that preceding questions inevitably shape subsequent responses (Tourangeau et al., 2000). It is proposed that the excessive use of Insight-style questions could impact participant engagement, completion rates and the depth or quality of the data obtained, meaning researchers should be conscious of this in the design process. This reinforces the importance of the mapping, ratio checking and piloting outlined Section 3.4 of this paper, where the balance between sufficient insight and maintaining participant engagement can be considered. A further concern involves the potential risk of oversimplifying the qualitative data during the integration analysis (Figure 1, Step 6.) Researchers must reflect on and aim to balance interpretative depth with analytical clarity, acknowledging the documented challenges in achieving deep and meaningful integration in mixed methods research (Bryman, 2016).
A final concern recognises that the current validation of the SCI approach rests on limited studies of application. This paper is, therefore, also a call to other social science researchers to assist in broadening the application of this approach to allow for further identification of its strengths and areas for development. In essence, the SCI approach offers a pragmatic pathway for researchers using online surveys that allow for either a purely qualitative analysis or a mixed methods analysis depending on how the researcher(s) intend to answer their research questions. It does this by acknowledging the value of structure, focus and clarity whilst simultaneously capturing participants’ voice to explain their structured responses. It represents a specific balance point on the continuum between purely exploratory qualitative work and highly structured quantitative surveys.
The recognised limitations and concerns point towards areas for future refinement of the SCI approach. Future research utilising this approach should explore strategies that increase the richness and completion rates of qualitative Insight responses. Specific consideration should be given to how these questions are framed and linked to Scale and Choice responses. For example, is there an ideal number of Scale and Choice type items that should proceed an Insight question? With ideal here referring to capturing meaningful insight from the participant whilst balancing the need for context against participant fatigue. Applying the SCI method across diverse contexts and fields beyond education is also needed to establish its broader transferability and identify necessary adaptations for different research populations and topics. Additionally, investigating more advanced mixed-methods analytical techniques could further demonstrate the relationship of qualitative narratives and the ordinal/nominal data, moving beyond descriptive summaries and thematic linking (Creswell and Plano Clark, 2018; Greene, 2007).
Conclusion
Online surveys have become a staple in qualitative research, yet a clear methodological gap has persisted regarding the systematic integration of structured, numerical-style responses with immediate, contextualising qualitative data within a single instrument. This paper has addressed this gap by introducing and detailing the Scale, Choice and Insight (SCI) methodology. The SCI approach provides a novel, structured framework designed to enhance the rigour and transparency of qualitative online survey research. It achieves this through a two-part process: gauging initial participant viewpoints via Scale and Choice items, followed immediately by open-ended Insight questions designed to capture the participant’s subjective interpretation behind their chosen response(s). The core contribution of the SCI methodology lies in its explicit guidance for both survey design and data analysis. By advocating for systematic design steps, including mapping questions to research questions and considering the ratio of Scale and Choice to Insight items, SCI promotes methodological clarity. Its two-phase analytical framework, integrating descriptive analysis of Scale and Choice data with thematic analysis of Insight narratives, facilitates a robust connection between quantitative patterns (‘what’) and qualitative explanations (‘why’). This immediate, integrated approach within a single survey distinguishes SCI from purely exploratory qualitative surveys and traditional sequential mixed-methods designs. The SCI approach offers a pathway to nuanced understanding while remaining committed to participant meaning. Additionally, the structure allows for potential methodological pluralism through more advanced quantitative analysis if desired by the researcher.
As demonstrated through examples from studies exploring educational policy contexts, the SCI approach proves practical for capturing complex perspectives. However, inherent limitations in its approach are also acknowledged. The introduction of structure via Scale and Choice items requires careful consideration within interpretivist paradigms regarding potential influence on subsequent narratives. The methodology’s success also depends significantly on participants providing sufficiently detailed Insight responses, highlighting the need for mindful survey design to balance depth with participant engagement. Furthermore, the potential for oversimplification during data integration requires researcher reflexivity, and the current validation rests on the initial studies presented here. These limitations point towards important avenues for future research. Further investigation is needed to refine strategies for capturing rich Insight data, explore optimal question ratios and examine the impact of question sequencing. Crucially, the application and testing of the SCI methodology across diverse research fields and populations is essential to establish its broader application and identify necessary adaptations. Finally, exploring more advanced analytical techniques to further integrate the ordinal and nominal data with the narrative data presents a promising area for development.
In conclusion, the Scale, Choice and Insight methodology offers a valuable and pragmatic contribution to the field of online qualitative research. By providing a clear, systematic, yet flexible framework, SCI enables researchers to design and analyse online surveys that balance structured data collection with deep interpretative insight. The research community is invited to utilise, critique and further develop the SCI approach to contribute to its ongoing refinement and application in social research.
Footnotes
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
