Abstract
Interpreting practitioners, pedagogues, and evaluation specialists consistently encounter the methodological challenge of calibrating and managing source texts that exhibit diverse degrees of linguistic and cognitive difficulty. Although the difficulty of source materials has significant implications for multiple facets of interpreting studies, including pedagogical resource development (Yuan, 2022), cognitive processing research (Ehrensberger-Dow et al., 2020; Kuang & Zheng, 2023), and quality assessment (Chen, 2017; Korpal & Stachowiak-Szymczak, 2020), systematic investigations into the determinants of source text difficulty and their subsequent impact on interpreting performance remain notably scarce. Tianyuan Zhao’s An Empirical Study on Constructing the Difficulty Assessment System of Source Materials for E-C Consecutive Interpreting, published in December 2023, represents a significant advancement in addressing this research lacuna. This pioneering work offers a comprehensive framework for evaluating source material difficulty, with particular emphasis on English–Chinese consecutive interpreting (E-C CI), thereby making a substantial contribution to this underexplored domain of interpreting studies.
Employing a multiphased, mixed empirical approach, this work, as implied by its title, aims to develop a quantitative assessment system that captures the difficulty of source materials in E-C CI. The investigative trajectory unfolds through sequential analytical stages, initially delineating the taxonomic dimensions and constituent factors that contribute to source text difficulty, followed by the construction of a calibrated weighting mechanism to reflect the hierarchical significance of these identified parameters. The analytical framework then advances to elucidate the underlying mechanisms through which these components interact and influence interpreting difficulty, culminating in a rigorous validation of the proposed assessment model. This empirically grounded investigation is systematically structured across seven chapters, complemented by comprehensive appendices that provide detailed methodological documentation and supporting analytical data.
In Chapter 1, the author clarifies the research background, objectives, significance, research scope, and methodology, laying the foundation for subsequent chapters. Zhao further explains the critical nature of source material difficulty assessment in interpreting, with particular attention to its applications in professional certification contexts. Of methodological value is the inclusion of a research roadmap (p. 16), which effectively illustrates the study’s structural organisation and methodological design.
Chapter 2, “Literature Review,” establishes the theoretical framework by examining difficulty-related research within and beyond translation and interpreting studies. The unique nature of CI as a task processing both linguistic and paralinguistic information, coupled with its implications for interpreting pedagogy and assessment, necessitates incorporating broader scholarship on textual difficulty, paralanguage, and task complexity into source material difficulty evaluation research. The review is then narrowed in scope to examine difficulty-related studies, specifically within translation and interpreting, with particular attention to consecutive interpreting. This comprehensive review establishes the theoretical foundations for the investigation while identifying applicable methodological approaches. The theoretical framework is anchored in two fundamental concepts: the dual objective–subjective nature of the difficulty and Gile’s (2009) CI effort model, which explicates how source material features impact the core processes of consecutive interpreting, namely, listening comprehension, note-taking, and target language production. Through this analytical review, the author identifies several research gaps her study aims to address, such as the absence of students’ perspectives, the fragmented approach to analysing source material difficulty, and the lack of rigour in experiment design.
Chapter 3 introduces key concept definitions, theoretical frameworks, research questions, design, methods, and data analysis. Zhao provides clear definitions of essential concepts, such as CI, source language, CI source language, difficulty, assessment system, and paralanguage, situating each within the specific context of the study. The author integrates six different theories that encompass various dimensions of the study, such as source language features, subjective perceptions, and CI performance. The methodological architecture demonstrates triangulation through an array of qualitative and quantitative methods, such as documentary analysis, content analysis, and analytic hierarchy process, supported by robust statistical procedures, including coding protocols and multivariate analyses. However, although the author employs Weir’s (2005) socio-cognitive framework for validity assessment, despite the acknowledged time and resource constraints of the study, future research might build upon this foundation by exploring additional dimensions of consequential validity. Furthermore, even though context validity is mentioned in the previous section, the author does not include it in the later theoretical diagram, which may benefit from further clarification to strengthen the theoretical coherence of the chapter.
Chapters 4, 5, and 6 are designed and developed to respond to the three research questions in Zhao’s study, focusing on the construction, exploration, and validation of the (hypothetical) difficulty assessment model for E-C CI source materials. In Chapter 4, the author conducted questionnaire surveys and interviews to understand the state of the art of the topic under study and later extracted the dimensions and factors influencing the difficulty of E-C CI source language through qualitative coding of relevant literature from practice, teaching, and research levels. The author then identifies four common themes, that is, content, lexical, syntactic, and verbal paralinguistic, and uses them as the basis to conduct a quantitative analysis of interpreting materials with quantitative linguistics tools such as Coh-Metrix. With all 14 dimensions and 105 factors extracted, Zhao constructs the hypothetical model of the assessment system and then further refines it to comprise seven dimensions and 26 factors.
Chapter 5 delineates the methodological refinement and empirical validation of the hypothetical assessment model. The statistical analysis proceeds through two phases: exploratory factor analysis to identify underlying structures, followed by confirmatory factor analysis to validate the emergent patterns. This systematic approach yields a four-dimensional assessment framework, encompassing content and structural elements, lexical-syntactic features, paralinguistic and medium-related components, and pragmatic considerations, with 15 constituent factors. The analytic hierarchy process is employed to assign and determine the weights of each dimension and factor, creating a robust foundation for subsequent exploration of difficulty mechanisms and validity assessment.
Chapter 6 explicates the operational mechanisms of individual difficulty factors through methodological triangulation, establishing detailed scoring criteria and descriptions for each assessment component. The scientific validity of the framework is substantiated through the systematic application of Weir’s (2005) socio-cognitive framework. These three chapters demonstrate methodological coherence in addressing the study’s research questions and progress from initial conceptualisation through empirical validation to produce a theoretically grounded and practically applicable assessment model for the source materials in E-C CI.
While these three chapters offer significant scholarly contributions, several aspects warrant further consideration. First, the discussion of student participants’ field practice situations and subsequent recommendations for increased practice opportunities (p. 115), although valuable in a broader pedagogical context, may not be directly relevant to the study’s primary focus. Second, the initial factor extraction process could benefit from greater consolidation to address potential conceptual overlaps, particularly evident in the relationship between “topics,” “high requirements of background knowledge,” and “plenty of terminologies” (p. 116). Third, the comparison between student and teacher respondents (Tables 4-1, p. 116, and 4-2, p. 119) would benefit from more consistent terminological treatment, particularly in the coding of factors such as huati (topics) versus huati shengpi (unfamiliar topics), which affects the accessibility of the synthesised data in Table 4–4 (p. 126). Furthermore, while the study highlights a significant discrepancy between interpreting quality assessment experts and university teachers regarding the necessity of understanding students’ perceived difficulty with teaching or test materials (p. 123), the representational imbalance in the sample—three teachers versus one expert—suggests potential limitations in the generalisability of these findings. Last, the analysis of difficulties arising from national standards and industrial regulations (p. 135) presents opportunities for stronger theoretical grounding through engagement with existing scholarship, particularly in areas such as topic familiarity (Chen et al., 2021) and culture-related items (Hale, 2014). Such theoretical reinforcement would enhance the scholarly rigour of these observations.
The concluding chapter synthesises the study’s findings, innovations, and limitations. While acknowledging methodological constraints, including the absence of authentic interpreting tasks and limited analysis of student perceptions, this research makes substantial contributions across multiple domains. The study bridges a crucial gap in E-C CI assessment by developing a systematic framework for evaluating source material difficulty. Its implications extend beyond theoretical advancement to practical applications in interpreter training, assessment protocols, and the language service market. The research’s particular significance lies in its potential application to automated interpreting assessment systems. By integrating both objective difficulty metrics and subjective learner perceptions, the study establishes a comprehensive foundation for developing more sophisticated automated evaluation tools in interpreting studies.
The study presents several limitations that merit further consideration. First, this study focuses on first-year interpreting students of the Master of Translation and Interpreting (MTI) programmes who are in the second-semester study of consecutive interpreting. Such a scope may limit the generalisability of the research findings, and therefore, the conclusions drawn from this study may not be entirely applicable to interpreting learners at other proficiency levels or stages of training, and even the professional certification examinations. The second limitation concerns the potential operational complexity of the proposed difficulty assessment system. While methodologically robust, its implementation in practical settings, including real-time interpreting tasks, classroom environments, standardised testing, and autonomous learning contexts, may prove challenging due to its reliance on multiple quantitative indicators that require sophisticated calculation procedures. Moreover, the multidimensional nature of the assessment framework demands high levels of professional expertise from assessors, potentially limiting its practical applicability. Third, the assessment framework might benefit from expanded consideration of emerging technological factors affecting source language difficulty. Contemporary developments in interpreting technology introduce additional variables that are worthy of further investigation, including automatic speech recognition accuracy (Li & Chmiel, 2024) and the impact of machine translation quality on interpreting processes (Chen & Kruger, 2024).
Despite these limitations, this book holds significant importance for enhancing the scientific nature and effectiveness of interpreting teaching, assessment, and research, thereby promoting the development of the interpreting discipline.
