Validation of an Instrument for Chinese Engineering Students’ Epistemological Development

Abstract

Perry’s theory has demonstrated usefulness in understanding students’ epistemological development and the implications for educational practices. It depicts the development of adults’ thinking in four stages: Dualism, Multiplicity, Relativism, and Commitment within Relativism. Challenges exist in large sample studies due to the lack of appropriate instruments that can represent the stages, in particular Multiplicity and Relativism. This challenge is of unique importance because the transition from Multiplicity to Relativism signifies the most dramatic change in thinking development. To map the epistemological development profile of Chinese engineering students on a large scale, this work re-constructed and validated a survey using Perry’s theory. We tested the survey through content and structural validation. Content validation was performed by collecting ratings from eight external content experts. Structural validation was accomplished through an exploratory factor analysis using responses from 621 Chinese engineering students. This survey can be useful for understanding Chinese engineering students’ epistemological development.

Introduction

Within the broader territory of cognition, epistemological thinking, or Epistemic Cognition, is defined as the third level in Kitchener’s (1983) 3-Level Model of Cognitive Processing. The first and second levels, i.e., Cognition and Metacognition, relate to an individual’s ability to perform and monitor learning tasks, such as memorization, reading, etc. The third level of Epistemic Cognition relates to one’s ability to discern knowledge’s limits, certainty, or “the criteria for knowing” Epistemic Cognition, 1 which often starts forming from late teenage years and further develops in adult years, allows an adult to reason through conflicting issues in areas like ethical problems or life and career choice. Considering the impact that one’s epistemic cognition has on other levels of cognitive process, many researchers have studied epistemic belief of young adults.

Multiple studies of the epistemological development of young adults over the past several decades have produced a series of different theoretical frameworks, models, and hypotheses. Among these theories, models, and hypotheses, William Perry’s intellectual and ethical developmental model has been recognized as the pioneering effort in understanding young adults’ cognitive development. It was first developed among Harvard elite white male undergraduate students in the late 1960s. According to the common theme within the literature, Perry’s original nine-position theory can be summarized into four major stages, i.e., Dualism, Multiplicity, Relativism, and Commitment (within Relativism). 2 These four stages depict an individual’s development in thinking from a dualistic way to a relativistic way. Since the first publication of Perry’s pioneering work, Perry’s theory has been tested and expanded at both the theoretical level and the practical applications level.

Investigations using theories and models related to students’ epistemological development, such as Perry’s theory, have influenced current educational practices, in particular studies in engineering education. Researchers have pointed out that expert engineers tended to demonstrate more mature epistemological thinking than novices. 3 Corresponding pedagogical strategies have been proposed or tested to modify engineering educational practices, such as the design of engineering curricula, 4 and the interactions between instructors and students. 5

In the past decade, some researchers have tried to apply Perry’s theory to the study of Chinese college students. 6 , 7 , 8 , 9 , 10 , 11 In order to understand these students’ epistemological development on a large scale, it is necessary to have a valid tool to measure their epistemological development in the context of Perry’s theory. Different challenges exist for measuring epistemological development in the context of Perry’s theory, especially trying to differentiate the stages of Multiplicity and Relativism. 12 This challenge is of particular interest because the transition from Multiplicity to Relativism for a student signifies the most dramatic change in the development of their thinking, according to Perry’s theory.

Considering the benefits of using pen-and-paper approaches to measure Perry’s positions on a large scale and the current difficulty of doing so, the goal of this research is to develop a survey that reflects the stages of Multiplicity and Relativism respectively. We constructed a survey for educators and practitioners based on extensive literature. To test the content validity of this instrument, we collected external experts’ ratings focusing on these two stages. Moreover, with the goal of mapping engineering students’ epistemological development, we tested the survey among 621 engineering students. In particular, we explored the structure of the survey through exploratory factor analysis. This study is a part of a large study on mapping the epistemological development profile of Chinese doctoral students in engineering using both quantitative and qualitative methods. This report conveys our current effort to establish a valid measure in order to map the epistemological developmental profiles of engineering students.

Literature Review

Theoretical Framework

Perry’s theory (1970) depicts nine positions along which college students’ epistemological development takes place. 13 The positions can be grouped into four major categories: Dualism (Positions 1 and 2), Multiplicity (Positions 3 and 4), Relativism (Positions 5 and 6), and Commitment (Positions 7, 8, and 9). 14 The following section is a brief description for each position from Perry’s theory. 15

Dualism

Position 1: Basic Duality

A person in this position holds “black and white” certainty for all values and ideas. Living in an absolute world, an authority is deemed as the source of knowledge and holds the role of giving knowledge to students.

Position 2: Multiplicity Pre-Legitimate

A person in this position perceives diversity in opinion and uncertainty as unnecessary confusion by poorly qualified authorities or a narrow area of freedom set by the authority for student’s own exploration.

Multiplicity

Position 3: Multiplicity Subordinate

Uncertainty and diversity are deemed as common phenomenon and unavoidable in life but only in domains where authority has not attained the right answers. Evaluation of one’s work has become puzzling in areas where authorities have no consensus. One may suppose that an authority grades based only on how well the answer is expressed.

Position 4. In this Position, There are Two Alternative Views

4a. Multiplicity Correlate. In this position, Multiplicity is raised from a subordinate to a legitimate status, which is separate and equal to that of the authority. Equal absolutism, i.e., “Everyone has a right to his own opinion” dominates the Multiplicity realm, 16 while in the authority’s domain there remains a right-wrong dualism.

4b. Relativism Subordinate. A person in this position perceives relativistic reasoning as what the authority wants. The student’s weighing of different approaches to one problem and developing of his/her own thoughts occur in the context of an authority’s realm out of a student’s desire to conform to authority’s demand.

Relativism

Position 5. Relativism

In this position, all knowledge and values, including the ones of authorities, are perceived as relativistic. A person here will analyze, weigh, or evaluate different evidence, factors, or solutions to develop his/her own opinion, argument, or solution.

Position 6. Commitment Foreseen

A person in this position realizes a necessity to commit oneself in a relativistic world. Acknowledging that reason itself has a limit and cannot fully justify or assure one person, individuals must commit themselves through their own faith, assuming the responsibilities that are associated with their choices.

Commitment within Relativism

Positions 7, 8 and 9. These three positions each describe different levels or scopes of commitment to a relativistic view. A person undertakes responsibilities in different major areas of life and experiences the implications of the commitment.

According to Perry’s description, the revolution from Position 4 (Multiplicity Correlate or Relativism Subordinate) to Position 5 (Relativism) is “both the most violent accommodation of structure in the entire development, and at the same time the most quiet”. 17 Relativistic thinking, which once was a special case in the context of Dualism, then became the context within which dualistic thinking formed a special case. It is a “complete transposition between part and whole, figure and ground”. 18 Therefore, it is of particular interest to researchers to be able to dissect these two positions in measurement. However, as Perry noted, almost no student in their sample “referred to it as a conscious event, a discrete experience, a ‘realization’ ”. 19 This characteristic of the transition can add to the difficulty in measurement.

Perry’s theory describes the epistemological development in students’ thinking from a dualistic to a relativistic view for college students. 20 Later reports point out that Perry’s samples were elite male college students from Harvard; most students in other colleges did not show epistemological developmental stages as high as that of Perry’s original observations. 21 Further applications of Perry’s theory have been expanded to different genders, 22 majors, 23 and age groups. 24 , 25 , 26 , 27 These applications have further validated and expanded Perry’s theory in different ways.

The theories or models following Perry’s theory include: women’s way’s of knowing, 28 epistemological reflection model, 29 reflective judgment model, 30 and argumentative reasoning model. 31 These models each have their own emphases in the proposed theory or their testing process. For example, Belenky et al. (1986) focused on female participants from diverse backgrounds, as opposed to Perry’s male elite participants. 32 King and Kitchener (1994) focused on understanding an individual’s epistemological development in terms of their reflective thinking, i.e., the characteristics and features, or essence of these different stages of reflective thinking. 33 Kuhn focused on the abilities of students to build a sound argument and the characteristics of the reasoning process. 34 Although each of these theories and models provide unique contributions to the original Perry model and expand the epistemological development model in different ways, these models or theories all validated the developmental pattern first proposed in Perry’s model, that is, the transition of thinking from a dualistic manner to a relativistic manner that traces an individual’s epistemological development. 35 , 36

Current Instruments for Measuring Epistemological Development

In order to understand students’ epistemological development, it is useful to have valid quantitative measures for providing an overall understanding of students’ epistemological development on a larger scale as compared to qualitative measures, which often demand hour-long interviews and intensive qualitative data analysis process. 37 , 38

So far, quite a few different pen-and-paper measurement methods have been proposed since the first issue of Perry’s intellectual and ethical model. These quantitative measures include Epistemological Reflection (mer), 39 , 40 , 41 , 42 Learning Environment Preferences, 43 Reasoning about Current Issues Test (rci), 44 survey of epistemological beliefs, 45 and Zhang’s Cognitive Development Inventory (zcdi). 46 , 47 A comparison of these multiple quantitative measures was conducted to identify an appropriate quantitative measure for mapping the profiles of Chinese engineering doctoral students’ epistemological development. 48 According to the comparison of these different quantitative measures, Zhang’s Cognitive Developmental Inventory was identified to be most relevant to the purpose of applying Perry’s theory among the Chinese student population. 49

Zhang and her colleagues performed a series of five consecutive studies on the cognitive development of u.s. and Chinese college students over the past decade. 50 , 51 , 52 , 53 , 54 , 55 Zhang’s Cognitive Development Inventory was developed based on Perry’s theory. It includes 75 short statements and has five subscales assessing three of the four stages in Perry’s model as mapped in Culver and Hackos. 56 , 57 These three stages are Dualism, Relativism, and Commitment within Relativism. Dualism and Relativism were examined in two content areas, education and interpersonal relationship. Commitment, or named Life Responsibility in zcdi, reflects the last position in Perry’s model, Commitment within Relativism.

Zhang’s Cognitive Development Inventory provides a good measurement tool to understand the epistemological development profile, especially among Chinese students. However, zcdi only reflects three out of the four stages, i.e., Dualism, Relativism, and Commitment. That is, it cannot differentiate Multiplicity and Relativism due to the difficulties in item construction and the possible overlap between these two stages of thinking according to Zhang. 58 This similar difficulty in differentiating Multiplicity and Relativism was also observed in Fago’s study. 59 In Fago’s study, he constructed a 45-item survey from Perry’s original description. He then applied the survey among 751 college students. Through a factor analysis of the survey results, he observed only three factors, which reflect three of four stages in Perry’s model, i.e., Dualism, Relativism, and Commitment.

The difficulty in these studies in differentiating Multiplicity and Relativism can be attributed to several possible reasons. In the original Perry discussions, Position 3-Multiplicity Subordinate denotes an incomplete state of Multiplicity. 60 The two alternative pathways within Position 4, i.e., Position 4a-Multiplicity Correlate and Position 4b-Relativism Subordinate, each represent an alternative pathway to true relativistic thinking. Position 4a-Multiplicity Correlate is cognitively closer to Position 3-Multiplicity Subordinate in that people in these two positions exhibit different levels of multiplistic thinking. It is therefore reasonable to group them in the measurement of the stage of Multiplicity. However, a person in Position 4b-Relativism Subordinate may show some similar thinking or behavioral patterns to a person in Position 5-Relativism, that is, in these two positions, people would analyze, weigh, and evaluate different factors, causes, and arguments (Position 5-Relativism) or try to mimic this way of reasoning because this is “the way they want you to think” (Position 4b-Relativism Subordinate). 61 Therefore, grouping Position 3 and Positions 4a and 4b all together under Multiplicity can be potentially complicated for measurement.

Here, in order to differentiate Multiplicity and Relativism, a detailed content validity test was conducted. The survey was also piloted among more than 600 Chinese engineering students. Its structural validity was examined through an exploratory factor analysis. By doing so, the goal is to design a valid survey that differentiates Multiplicity and Relativism in order to map Chinese engineering students’ epistemological development on a large scale.

Method

Survey Construction

With the aim of differentiating Multiplicity and Relativism via a pen-and-paper measurement tool, we conducted an extensive search over current literature on the pen-and-paper measurement tools for adult epistemological development in the context of Perry’s theory. On the basis of the literature search, especially on current available measurement instruments, a survey was constructed to reflect the four specific positions within Perry’s theory, i.e., Position 3-Multiplicity Subordinate, Position 4a-Multiplicity Correlate, Position 4b-Relativism Subordinate, and Position 5-Relativism, which are the positions that are in the stages of Multiplicity and Relativism.

Because of the subtle differences across the stages and the positions within, it is challenging to design items that can be specific to the two stages. Therefore, we focused on compiling, revising, or constructing the items that reflect these positions across current literature. As we have discussed in the literature review section, there have been quite a few studies trying to measure adult’s epistemological development in the context of Perry’s theory. Among the current literature, we compiled and revised items from several instruments, 62 , 63 , 64 and also constructed some new items based on the description of the thinking at different positions according to Perry’s narration that are most relevant to these two stages. 65

To make sure the meaning of each item is clear to Chinese students, these items were beta-tested among six Chinese graduate students from different majors in a large Midwestern university in the u.s. Editorial changes were made to the items so that the items are understandable to Chinese students. Based upon the authors’ original design along the Perry scale and the research team’s rating of each item after some editorial change, the research team grouped each item within the context of the Perry theory, thereby creating a mapping of each potential survey item to Perry’s positions. The list of survey items along with its original literature source and their Perry positions is shown in Table 1. The Perry positions shown in Table 1 were identified based on both the original literature and the authors’ understanding of Perry’s theory. The ones that have the citation as Perry are items newly written according to Perry’s original descriptions within specific positions.

Content Validity Check

A content validity check is commonly used to examine the degree to which an instrument reflects the guiding theoretical framework in its design of survey items. 66 Here, the goal is to differentiate the stages of thinking in between Multiplicity and Relativism. As we mentioned above, challenges exist in designing items specific to the two stages because of the subtle differences across the stages and the positions within each stage. Moreover, the differences between the two stages can also be very difficult to detect via large-scale testing. 67 Therefore, by inviting experts that are familiar with this framework, one can potentially differentiate these two stages, i.e., Multiplicity and Relativism.

Moreover, as we discussed in the literature review, the subtle differences across the positions within the two stages could potentially complicate the process of differentiating the two stages. Therefore, it is reasonable to perform a content validity check at the individual position level, i.e., Position 3-Multiplicity Subordinate; Position 4a-Multiplicity Correlate; Position 4b-Relativism Subordinate; and Position 5-Relativism, in order to parse out the subtle differences across the positions and obtain meaningful data.

Eight experts who are familiar with Perry’s theory were invited to rate how well the survey items reflected the specific positions of Perry’s theory. These external experts that were involved in the process of the content validity check represent a variety of perspectives and experiences with Perry’s theory. According to their self-reports, they were engaged in research efforts related to Perry’s theory, taught classes related to Perry’s theory, or attended training sessions (i.e., classes, workshops, etc.) related to Perry’s theory. One of the experts self-indicated that he “led numerous workshops on the Perry scheme and [was] active in research on the scheme for many years”.

table 1

Survey items compiled and constructed from an extensive literature review (after beta-testing and editorial changes)

† Perry Positions: P3-Multiplicity Subordinate; P4a-Multiplicity Correlate; P4b-Relativism Subordinate; P5-Relativism.

* This is an item developed based on clev (Checklist of Educational Views), the instrument that Perry (1970) used in his sampling procedure.

table 2

Sample item in the expert rating form

A sample item, as presented to content experts on the Expert Rating Form, is displayed in Table 2. An open-ended question was also added to collect any additional comments on the survey items. An information page was provided to the experts with brief definitions of each position in Perry’s theory. After the experts rated all of the items, researchers then compiled their ratings and calculated the average ratings across content experts for each item. If the sum of external experts’ ratings for a certain position of an item is higher than 24, i.e., the average score among all eight experts is equal to or higher than 3, then the item would be rated as in the corresponding position. For the purpose of measurement, we need items that only reflect one particular position or possibly its neighboring positions, but not items that reflect all positions. Therefore, some items that reflect two unrelated positions will be deemed as not useful. For example, if an item had high scores on both Multiplicity Subordinate and Relativism Subordinate, it would be regarded as not acceptable.

Exploratory Factor Analysis ( efa )

Exploratory factor analysis was used to explore the structure of this survey. It is an important tool to examine structural validity in the early stages of instrument development. Its goal is to “identify the underlying dimensions of a domain of functioning, as assessed by a particular measuring instrument”. 68 Here, we further explore the potential construct(s) within the survey and compare the constructs with results from experts’ ratings. Principal component analysis (pca) was used to extract factors. Finally, as relevance exists among these factors, an oblique rotation (Promax) was employed to explain factors.

Specifically, the survey obtained from the content validity check was administered after a few minor wording changes to engineering students in a leading research-intensive Chinese university. This effort was part of a larger study to explore Chinese students’ epistemological development. A five-point Likert scale was used (Strongly Disagree, Disagree, Not sure, Agree, Strongly Agree) to record students’ responses to these short statements.

Six hundred and twenty-one responses were collected from engineering undergraduate and graduate students who majored in 17 engineering disciplines such as Mechanical Engineering, Electrical Engineering, and Materials Science and Engineering. These survey responses were then used for exploratory factor analysis.

Results

Content Validity

Using the content validity check, we were able to identify the degree to which each survey item reflected the four positions within the stages of Multiplicity and Relativism, i.e., Position 3-Multiplicity Subordinate; Position 4a-Multiplicity Correlate; Position 4b-Relativism Subordinate; and Position 5-Relativism. A compilation of the grand total of ratings for each survey item across experts is shown in Table 3. From the content validation results, we found that the classifications of these items from external ratings suggested the closeness of Position 3-Multiplicity Subordinate and Position 4a-Multiplicity Correlate in their reflections of the stage of Multiplicity. Also, the classification suggested the closeness of Position 4b-Relativism Subordinate and Position 5-Relativism in their reflections of the stage of Relativism.

Overall, from the content validation results of external content experts, among the twenty-four survey items, twenty items were identified to represent at least one of the four positions of Perry’s theory (see the shaded numbers in Table 3). Across these twenty items, the classifications of Perry’s positions for eighteen items matched the original classifications based on review of the literature and the authors’ understanding, which demonstrates a high level of design validity of the survey items to accomplish their original purposes of reflecting certain positions of Perry’s theory.

Through closer examination of the individual items, we found that the items that reflect the Position 3-Multiplicity Subordinate and/or the Position 4a-Multiplicity Correlate, i.e., Item 1 to Item 10 are very close in meaning and therefore have quite a lot of overlaps. For example, Item 2 and 7 reflect both Position 3-Multiplicity Subordinate and Position 4a-Multiplicity Correlate. The ratings of some other items also suggest this similarity, for example, Items 1 and 3 both have quite high ratings in both Position 3 and Position 4a (higher than 20). Items within Position 3-Multiplicity Subordinate and Position 4a-Multiplicity Correlate reflect the students’ realization or awareness of the diversity of views and their dilemma to determine the criteria with which these views can be evaluated when “no correct answer is available”. Together, the classifications of these items from external ratings suggested the closeness of Position 3-Multiplicity Subordinate and Position 4a-Multiplicity Correlate in their reflections of the stage of Multiplicity.

Based on these discussions, in our final classifications we grouped the items that reflect either P3 or P4a or both as items of Multiplicity. It should be noted that this classification does validate the conventional way of classifying Multiplicity, i.e., Position 3-Multiplicity Subordinate and Position 4a-Multiplicity Correlate were both grouped in the stage of Multiplicity. Although, in the conventional way of classifying Multiplicity, Position 4b-Relativism Subordinate was also included in the category of Multiplicity.

In contrast to the conventional way of classifying Position 4b-Relativism Subordinate in the category of Multiplicity, the external experts rated items designed for Position 4b-Relativism Subordinate and Position 5-Relativism to be closer in their representations of Perry positions. Items 11, 13, 14, 20, 21, 22, and 23 all have quite high ratings in both Position 4b and Position 5 (equal to or higher than 20). This observation raised the concern about the conventional way of categorizing Position 4, including both 4a and 4b into the stage of Multiplicity. That is, the conventional way of grouping both 4a and 4b within Multiplicity could possibly complicate the differentiation between Multiplicity and Relativism and therefore cause difficulties in measurement.

In addition to the above-mentioned overall patterns and observations, we would like to include a few notes about several items. The following did not satisfy the criteria of having the grand total larger than 24: Items 6, 11, 12, and 22.

For Item 6, “When experts in a particular field disagree with one another, no one really knows the answer”, the rating for Position 3-Multiplicity Subordinate and the Position 4a-Multiplicity Correlate were 22 and 23 respectively, which strongly suggests that the item reflects the stage of Multiplicity. Therefore, it is still included in the final version of survey.

For Item 11, “It seems to me that some instructors try to get you to look at something in a complex way by weighing multiple factors at once”, the rating for Position 4b, Relativism Subordinate and Position 5-Relativism were both 22, which suggests that the item reflects the stage of Relativism. Therefore, it is also included in the final version of survey.

For Item 12, “I try to think in an independent manner, because thoughts that appear to be independent get good grades,” the ratings for Position 3, Position-4a, Position-4b and Position-5 are 18, 17, 21, and 17 respectively. No single position can be particularly claimed as the reflection of this Item. Therefore, it is excluded from the final version.

For Item 22, “It is not difficult for me to give up ideas and opinions I hold if I find that my classmates’ ideas sound more reasonable,” the rating for Position 4b-Relativism Subordinate and Position 5-Relativism were both 22, which suggests that the item reflects the stage of Relativism. Therefore, it is also included in the final version of survey.

Finally, for Item 9, “I am certain of one thing—even if there is an absolute truth, we will never know about it and, therefore, there is no correct answer to most questions,” it also has a grand total larger than 24 in Position 5-Relativism. However, it was originally classified as Position 4a-Multiplicity Correlate. With the rating of 21 in Position 4a-Multiplicity Correlate and 24 in Position 5-Relativism from experts’ ratings, we determined that this item is not suitable for the purpose of differentiating Multiplicity and Relativism.

In summary, a 22-item survey was constructed and validated via the survey construction based on an extensive literature review and external content experts’ rating process (see the Classification column in Table 3).

Structural Validity

We adopted the survey that was obtained through the content validity check. An exploratory factor analysis was conducted to explore the structure of the current survey. Considering the research goal of differentiating Multiplicity and Relativism, a two-factor model was identified as the most suitable model with the Kaiser-Meyer-Olkin (kmo) value (.786) indicating acceptable sampling adequacy. 69 Factor loadings of all the items in this model are listed in Table 4. Factor loadings that were larger than .35 were considered as loaded to a certain factor. 70

table 3

Grand total of rating for each survey item across experts (eight experts in all)

Eight items were loaded to factor one. All of these items represent the dimension of Multiplicity. Meanwhile, twelve items were loaded to factor two. All of these items represent the dimension of Relativism except for one. Therefore, it is reasonable to conclude that the structure of the survey has reflected the two dimensions of Perry’s theory. Two items were not loaded to any factors.

Items 2, 3, and 8 in factor one, whose original Perry positions were P3/P4a, were found to load onto the same factor representing Multiplicity as items whose original Perry positions were P4a. This finding confirmed the similarity between P3-Multiplicity Subordinate and Position 4a-Multiplicity Correlate. Item 2 in factor two, whose original Perry position was P4b, was found to load onto the same factor representing Relativism as items whose original Perry positions were P4b/P5 or P5. This observation confirmed the similarity between Position 4b-Relativism Subordinate and P5-Relativism.

In summary, through an external content validity check and exploratory factor analysis, we confirmed that this survey can reflect the two dimensions of Multiplicity and Relativism. Using this survey, one can potentially differentiate students’ thinking along these two stages.

Discussion

The results from the external content experts’ rating and exploratory factor analysis suggest that because of the similarity between Position 3-Multiplicity Subordinate and Position 4a-Multiplicity Correlate, the items that reflect these two positions will tend to overlap with each other. Meanwhile, because of the similarities of the behavioral patterns between Position 4b-Relativism Subordinate and Position 5-Relativism, the items under these two positions will tend to overlap with each other.

Our results suggest modifications for the measurement process using Perry’s theory. Instead of using the current way of grouping Position 3-Multiplicity Subordinate, Position 4a-Multiplicity Correlate and Position 4b-Relativism Subordinate together as in the literature, it is more reasonable to separate Position 4a and Position 4b because of the similarity between Position 4b-Relativism Subordinate and Position 5-Relativism. By so doing, it is then more likely to produce an instrument that can differentiate the thinking between Multiplicity and Relativism.

table 4

Item mapping for a two-factor model

Abbreviations: M= Multiplicity; R= Relativism.

Factor loadings > .35 appear in bold.

As we have mentioned in the literature review, multiple surveys have successfully identified the other two stages of Perry’s theory, i.e., Dualism and Commitment within Relativism but not the differentiation between Multiplicity and Relativism. Therefore, the design of this instrument fills the current gap of literature in measuring the critical transition in Perry’s developmental stages using quantitative instruments. By combining this survey with current available surveys, which have ready-to-use items for measurement along the other two stages of Perry’s theory, i.e., Dualism and Commitment within Relativism, 71 one will be able to conduct large scale study for the understanding of the epistemological development of Chinese students in the context of Perry’s theory.

Moreover, this survey tool is compiled and constructed with the purpose of applying it among the Chinese population. Therefore, in the compilation, construction, and testing processes, the applicability among Chinese students was taken into consideration when the authors were deciding which items to include and during the construction and editing of different items. The survey has also been validated among the Chinese population for clarity and understanding. Thus, the survey tool will be applicable for measuring the epistemological development of Chinese students. Nonetheless, we do not think the content of the statements within the survey will require significant modifications before its administration among a different ethnic group. All the items were constructed and/or compiled based on prior research studies that were performed among different ethnic groups. 72 , 73 , 74 , 75 Still, careful review and additional testing will be required to apply it among different ethnic groups.

One potential limitation for this instrument is that it seems like the survey cannot help differentiate someone with the view of Relativism Subordinate from individuals with the view of Relativism. Potential problems could arise regarding the individuals who score high in Relativism with this instrument as to whether or not these individuals indeed demonstrate the relativistic thinking or simply have these behavioral patterns which imitate truly relativistic thinking. This limitation can potentially be addressed by further qualitative measures particularly in understanding the motivations of the behavioral patterns that are similar to relativistic thinking.

Conclusions

This paper reports the construction and validity testing of a survey developed in the context of Perry’s theory for its application among Chinese students through external content validity check and exploratory factor analysis. Using this survey, one can measure Chinese students’ epistemological development in the context of Perry’s theory. We have not tested the applicability of this tool among other ethnic groups. Nonetheless, with careful review of cultural applicability and additional revisions of items, we expect that this survey can be useful for large scale measurement of the epistemological development of students from other ethnic groups or countries. Future work will include mapping Chinese engineering students’ epistemological developmental profile. Within the broad context of active engineering educational reform in China, future research will also include investigating factors such as varied pedagogical practices that are potentially associated with students’ epistemological development.