Sage Journals: Discover world-class research

Abstract

The ability to think critically and creatively is essential for students to help them thrive in the 21st century. Creative and critical thinking can be measured through problem solving because the assessment contains tasks that require students to find problems, analyze and evaluate problems, and work out the solutions. Therefore, this study was designed to develop a critical and creative reading assessment model based on problem solving and test the validity and reliability of the assessment. The assessment contained problem-solving essay tests that were presented in the form of stories. The material stimulus was news item texts that focused on social problems. The data collected were in the form of mean scores of expert validity testing and practitioner inter-rater reliability testing. The assessment was tried out to 36 students. The validity of the instrument was analyzed descriptively, while the reliability of the instrument was analyzed using analysis of variance (ANOVA) and intraclass correlation coefficients (ICCs) with SPSS 26. The results of the study indicated that the mean score of the expert validity test was categorized high (3.74), and the result of the reliability test showed that there was no difference in the inter-rater assessment because p > .05 and the ICC obtained was excellent (r_xx = .716). It can be concluded that the assessment developed in this study was valid and reliable, and thus can be used as an assessment to improve the critical–creative reading skills which are the essential skills needed to survive and thrive in the 21st century.

Keywords

critical thinking critical reading creative reading problem solving reading assessment

Introduction

Critical–creative reading is a reading activity that involves high-level thinking skills, that is, critical and creative thinking (Wallace, 2010). Both thinking activities have a different focus; critical reading is focused on assessing, while creative reading is focused on a process of making or producing (Beghetto & Sriraman, 2017; Paul & Elder, 2008). The critical–creative reading ability is required by everyone in today’s global age to respond critically and creatively to new, irregular situations that are rapidly changing and developing (World Economic Forum, 2015).

The critical–creative reading ability can be measured through problem-solving-based assessments because the assessment contains tasks that require students to find problems, analyze and evaluate them, and then work out the solutions (Jonassen, 2010). Problem-solving-based assessments involve a cognitive operation directed to discover the unknown (Anderson, 2010). The problem-solving skills signified by two important skills, namely, problem finding skill and its problem context and the skills describing the problems’ structure, knowledge about the problem, as well as activities for problem solving (Jonassen, 2010). In the problem-solving activities, learners are also demanded to actively develop various alternative models of problem solving and test the accuracy of the developed model (Jonassen, 2010).

Why is a problem-based assessment needed? In daily life, we are never free from big or small, simple or complex problems. The success of our lives is actually marked by our success in solving various problems in this life. Therefore, learners need to be equipped with problem-solving skills. In professional context, people are paid to solve problems, not complete exams (Organisation for Economic Co-Operation and Development [OECD], 2014). Therefore, the problem-solving skill needs to be trained so that learners can be actively engaged in solving their own problem and the problems faced by society (Griffin & Care, 2015). In general, the problem-solving skill is oftentimes neglected by educators in the teaching and learning as well as in the evaluation (Beghetto & Sriraman, 2017; Jonassen, 2010). Indonesia is one example of a country whose national examination test for Indonesian subjects is all in the form of multiple choice (Priyatni & Martutik, 2019). The level of thinking measured in Indonesian national test is dominated by basic reading skills (Harsiati, 2017b).

Multiple-choice assessment cannot be used to measure higher order thinking skill needed to live in the era of the 21st century, particularly critical–creative reading skills because the competence needed to answer the multiple-choice objective questions is to choose one answer from the provided options (OECD, 2014). Assessment practiced in Indonesia is likely focused on what is known instead of what can be performed by the students (Harsiati, 2017a). The teachers mostly concentrate on evaluating lower order thinking skills rather than emphasizing on developing the skills to think at a higher order level, such as critical thinking, reasoning, and problem solving, which in fact constitute skills and abilities needed in the 21st century (Priyatni & Martutik, 2019). Therefore, educators, especially teachers, need to transform the way they teach and evaluate the students.

The problem-solving-based critical and creative reading assessment proposed in this study requires students to respond critically as well as simultaneously construct a product rather than simply select from a set of predefined alternatives or answers. Therefore, this assessment is important to be developed to strengthen or complement multiple-choice assessments that only select the already provided answers.

Literature Review

There are two essential topics discussed in this section, namely, the element of critical–creative reading and the elements of the proposed assessment.

The Element of Critical–Creative Reading

Critical reading is an activity to express ideas, draw conclusions, predict goals, analyze arguments, find different points of view, and evaluate ideas from written texts (Priyatni & Nurhadi, 2017; Zintz & Maggart, 1984). Critical reading contains not only the activity of understanding a text but also the activity to determine whether the information in the text is true or false and to assess the knowledge and opinions contained in the text (Sultan et al., 2017b, 2018). The main objectives of critical reading activities are to evaluate information from various points of view, including logical, rhetorical, historical, ethical, social, and personal perspectives (Wheeler, 2013), and discover new interpretations of texts (Asilioglu, 2008).

Critical reading is a reading activity that provides the reader with the skills to analyze, synthesize, and evaluate what is being read (Hudson, 2010). Some experts argue that critical reading is characterized by reasoning and reflecting activities (Costa, 1991). Being reasonable means showing a reason or sound judgment, and being reflective means to think about what we have read or heard seriously and involves any activity to assess the validity of the information and opinions we encounter (Sultan et al., 2017a). Based on the aforementioned information, it can be concluded that the essential elements of critical reading include analysis and evaluation of what is being read, examination of what is good and what is not, and “why.”

Unlike critical reading, creative reading is a reading activity that involves creative thinking skills. Creative thinking skill is identified with the emerging of new ideas or the combination of several ideas in the form of new ideas (Marin & Halpern, 2011). Creative thinking also involves divergent thinking, a cognitive aspect that is characterized by fluency, flexibility, elaboration, and originality (Nakano & Wechsler, 2018).

Creative reading is strongly associated with imagination, innovation, originality, and excellence (Jonassen, 2010; Swann et al., 2011). As the aforementioned definitions, it can be summarized that creative reading is reading to produce creative writing based on the response to what has been read. Creative reading is the key to productive thinking because creative thinking is defined as a mental activity used to build ideas (Syahrin et al., 2019). In conclusion, critical–creative reading consists of two essential elements that are the skills to make a decision and the ability to develop a new idea based on the result of the decision-making process.

The Existing Critical–Creative Reading Assessment and the Proposed Critical–Creative Reading Assessment

Program for International Student Assessment (PISA) included the assessment based on problem solving as an additional assessment. The problem-solving assessment was first carried out separately in PISA 2003 which focused on individual problem solving capabilities and static assignments. Students must choose several alternatives, and student also need to analyze systems and identify relationships between parts of a system. The stimulus used for problem-solving-based assessment in PISA 2003 is a robot cleaner unit (OECD, 2014).

In addition, a problem-solving assessment has also become a domain in PISA 2012, which focused on complex and computer-based issues (OECD, 2014). PISA 2012 also used a creative problem-solving assessment to assess an individual’s ability to solve real-life problems of a general nature. PISA 2012 also utilized an assessment based on problem solving as an additional assessment in general and based on the interactive computer (OECD, 2014).

Problem-based assessment developed by OECD (2013) is designed to solve general problems that are problems related to technology. This article proposes an assessment model to evaluate students’ abilities to solve social issues that emerge in their daily lives. This assessment was also designed to measure the students’ critical–creative reading skills in learning the Indonesian language (Bahasa Indonesia). This assessment model is modified from the Story Problem Solving Model developed by Jonassen (2010), with three focused activities, namely, (a) finding the problem and the root of the problem, (b) planning and executing solutions, and (c) expressing the best solution in the form of creative writing. Jonassen’s model assessment is very simple and in line with the problem-solving critical–creative reading framework.

The validity and reliability of the proposed assessment model were tested. The result was then submitted to complement objective tests that had been national domination assessment in Indonesia. The problem-solving critical–creative reading assessment requires students to respond critically as well as simultaneously construct a product rather than simply select from a set of predefined alternatives or answers. Therefore, this critical–creative reading with a problem-solving-based assessment is important to be developed to strengthen multiple-choice assessments that only select the already provided answers.

The assessment proposed in this study was adapted from Jonassen (2010). Jonassen (2010) suggests four steps in solving a problem, namely, (a) problem presentation, (b) problem articulation, (c) alternative solution finding, and (d) argumentation/justification. The problem presented is the analysis of the problem and its context, problem articulation is to formulate the problem and find the root of the problem, alternative solution finding includes considering several alternative solutions, and argumentation/justification is to determine the best solution along with the reason. The four steps were adapted into three steps, namely, (a) finding the problem and determining the root of the problem, (b) planning a solution, and (c) expressing the solution in the form of creative writing.

Problems that are to be solved by the students are those that are found in their everyday lives. These problems are problems with an unknown solution. If someone has set a goal and does not know how to reach that goal, that is what is called an unknown solution (Jonassen, 2010). Problems solved in the assessment are simple problems because the assessment is going to be used to measure senior high school students’ problem-solving skills. Problems can be sorted into two types, namely, simple problems and complicated problems (Jonassen, 2010). A simple problem is in small-scale, disconnected from other problems, has no great consequences, the solution is not too complicated, and can be solved individually.

Problems developed in the assessment are well-structured problems. Problems can be categorized into two, namely, a well-structured problem and ill-structured problems. A problem can be categorized as a well-structured when all elements of the problem are clearly described, requiring convergent answers and prescribing solution process. A problem is defined as an ill-structured when all elements of the problem are not clearly described and require the learners to secrete an opinion and personal belief to determine the solution of the problem.

The proposed assessment was in the form of problem-solving essays that allow students to discover problems and determine the root of the problems, plan solutions to the problems, and express the solutions in the form of creative writing. This assessment targets various aspects, including creative and critical thinking.

Research Methods

Research Design

The current study aimed to develop a product in the form of a problem-solving-based critical–creative assessment. The design of the study was adopted from O’Malley and Pierce (1996), who suggests eight stages of product development, namely, (a) build a team, (b) determine the purpose, (c) specify objectives, (d) conduct professional development, (e) collect sample assessments, (f) adapt existing assessments, (g) try out the assessments, and (h) review of the assessments.

The first step was to build a team. The team consisted of researchers, experts, and practitioners (teachers) who collaborated in developing the assessment. The second step was to determine the aim of the assessment, which was to improve students’ problem-solving, critical thinking, and creative thinking skills. Next, the third step involved deciding which basic competence to be measured. Students’ critical–creative thinking skills were finally determined as the basic competence to be measured in the assessment.

The fourth step, conducting professional development, was to inform all team members about the design of the prepared assessment. The fifth step was finding appropriate examples or models of assessment. The assessment model was adopted from Jonassen (2010). Jonassen (2010) proposes three steps in solving problems, namely, (a) choosing the problem to be solved; (b) understanding the structure of the problem, including recognizing the structure that underlies the problem, producing an appropriate presentation to find a solution, and drawing a conclusion on the problem presentation; and (c) determining the solution.

The sixth step was to adapt the assessment developed by Jonassen (2010). The three problem-solving steps proposed were adapted into the following steps: adding solutions (a minimum of three solutions), selecting the best solution and the reason why choosing the solution, and developing the solution in the form of creative writing. The assessment contains three main tasks, namely, (a) finding the problem and the root of the problem, (b) planning and executing at least three alternative solutions, and (c) expressing the best solution in the form of creative texts. The main features of the critical–creative reading assessment based on problem solving can be seen in Table 1.

Table 1.

The Main Features of the Critical–Creative Reading Assessment Based on Problem Solving.

Main features	Test model
Test objectives	• Ability to find problems• Ability to find the root of the problem• Ability to design at least three alternative problem-solving solutions• Ability to determine the best solution for reasons• Ability to arrange text framework• Ability to develop a text framework into creative writing text
Test form	Essay story problem solving
Number of items	Six items of complete overall ability test
Time allocation	120 min
Source of stimulus	News texts
Content of stimulus	Social problem
Complexity	Simple problems
Writing style	Narrative
Length of the text	450–600 words

The seventh step was to carry out validity and reliability tests. Validity was measured from two aspects, namely, construct validity and content validity. A construct validity test was done to see the compatibility of the items with the theoretical foundation of the problem-solving-based critical–creative reading assessment. A content validity test was carried out to see the appropriateness of the test items with the direction or objectives and aspects measured in the assessment pack. The validity of the assessment was tested by a team of reading and writing experts.

In addition, the reliability of the assessment tool was also tested. Inter-rater reliability was conducted because the assessment contained essay questions. The reliability test was performed using the following procedures: (a) the assessment was tested on 36 students; (b) subsequently, four practitioners were invited to rate student work using the compiled assessment rubric, with the highest score of 4 and the lowest score of 1; (c) the assessment data obtained from four assessors were inserted into a table; and (d) the data were analyzed using SPSS 26. The eighth step, product revising, was conducted to make adjustments or changes in the assessment according to the validity and reliability test results.

Data and Data Sources

The data of this study consisted of two categories, namely, quantitative and qualitative data. The quantitative data were collected in the form of validity test scores from a group of experts and practitioners. The qualitative data were in the form of responses, suggestions, or criticisms on the validity of the contents and constructs of instruments from expert teams. The sources of research data were four experts, divided into two experts in the evaluation and two experts in reading and writing learning, who hold a doctoral degree qualification. The sources of data for reliability testing were four practitioners and 36 students.

Instrument

An assessment review guide was used to collect the data on assessment validity. The guide contained an assessment table which measured three aspects, namely, (a) construct and content validity, (b) aspect description, and (c) scores scaled from 1 to 4 (1 = not appropriate, 2 = less appropriate, 3 = appropriate, and 4 = highly appropriate) (for complete instrument, see Supplemental Appendix 1).

The instrument used to collect the inter-rater reliability data was the problem-solving-based critical–creative reading assessment and its scoring guidelines. The reliability test was conducted by (a) instructing the students, as the tryout subjects, to read the test guideline, stimulus, and the essay questions; (b) asking the students to understand the scoring guidelines and telling them that their test answers would be scored using that guideline; (c) asking the students to complete the essay tests; (d) asking the students to review their work based on the guideline; and (e) evaluating the students’ work and providing feedback to their work.

The full scoring guidelines are listed in Table 2.

Table 2.

Scoring Guidelines of the Critical–Creative Reading Assessment Based on Problem Solving.

Aspect/score	Finding problems	Finding the root of the problems	Designing three solutions	Deciding the best solution	Designing text framework	Creating full text
4	Focusing on one problem contained in the scenario	Having a clear root problem as shown using a diagram	Consisting of three actions that match the problem, all of which differ significantly	Explaining one best solution accompanied with reasons	Creating a framework of a text-based on the content structure of the text	• Writing four/five complete paragraphs containing details of original ideas, suitable, and valuable to solve problems• Using a complex sentence structure with an effective grammatical structure• Using appropriate and varying vocabulary and conjunction• Having no, or only has one or two, spelling errorNote: Original ideas are new or fresh ideas that do not resemble the existing ideas. Original ideas can be a combination of some ideas obtained from reading activity and reactions toward what is being read
3	Focusing on one problem contained in the scenario	Having unclear root problem	Consisting of three actions that match the problem, but the actions are not significantly different	Explaining one best solution but not accompanied by reasons, or the reasons are illogical	Creating a framework of a text-based on the content structure of the text	• Writing four/five complete paragraphs containing details of original ideas to solve problems, but there are parts of the paragraph that is insignificant to solve the problems• Using simple/complex sentence structure, but there are some ineffective grammatical structure• Using appropriate vocabulary and conjunction, but with less variant• Having three to five spelling errors
2.	Focusing on one problem contained in the scenario	Having unclear root problem	Consisting of less than three actions that match the problem, but the actions are not significantly different	Offering solution without reasons or the reasons are illogical	Having a framework with an unclear organization of ideas	• Writing less than four complete paragraphs with unclear idea organization/content structure• Using simple sentence structure with an ineffective grammatical structure• Having less variant on the vocabulary and conjunction used• There are five to 10 spelling errors
1	Containing problems, but it is mismatched with the given scenario	Having unclear root problem	Containing actions unsuited to the problems	Giving ill-fitted solutions to the problems	Having a framework with unclear idea and organization	• Writing one to two complete paragraphs with unclear idea organization/content structure• Using simple sentence structure with an ineffective grammatical structure• Having less variant on the vocabulary and conjunction used• Having more than 10 spelling errors

Data Analysis

The result of the validity test was analyzed descriptively and interpreted based on the criteria of assessment validity, such as presented in Table 3.

Table 3.

Criteria of Assessment Validity.

Range	Criteria
3.26–4.00	High
2.51–3.25	Fairly high
1.76–2.50	Medium
1.00–1.75	Low

The reliability test result was analyzed using analysis of variance (ANOVA) and intraclass correlation coefficients (ICCs).

Results

The Main Features of the Critical–Creative Reading Assessments Based on Problem Solving

The product developed in this research is a problem-solving assessment model used to measure the ability of high school students in critical–creative reading. The assessments contain test models and scoring guidelines. The objectives of the test focus on three skills, namely, (a) finding the problem and the root of the problem, (b) planning and executing solutions, and (c) expressing the best solution in the form of creative texts. The first skill consists of two sub-skills, namely, the skill to find problems and root problems based on information in story problem solving. The second skill contains two sub-skills, namely, the ability to design at least three different alternative problem-solving solutions and determine the best solution completed with reasons. The third skill contains two sub-skills, namely, the ability to create a text framework and develop the text framework into a creative writing text. The type of test was an essay-story problem solving. The test package contained six items that constituted a complete overall ability which consisted of the ability to find the problem, the root of the problem, finding three different solutions, finding the best solution with the reasons, designing the text framework to explain the best solution, and developing the text framework into a full text. The time allocation for the essay test was 120 min.

The material stimulus used to develop essay tests originated from news texts that address social issues so that students can provide solutions to social problems faced by the environment/nation. The examples of social problems are the rise of hate speech in social media, the clash of supporters, rampant fraud, early marriage, or drug abuse. The writing style of the material stimulus is narrative. The length of the text is 450 to 600 words. The main features of the critical–creative reading assessment based on problem solving were used to develop the blueprint of the test and also as a reference for developing the test. In addition to the tests, the scoring guideline is also developed. The score is ranged from 1 to 4, the best score is 4 and the worst score is 1. There are six major sub-skills assessed, which are finding the problem, determining the root of the problem, finding three significantly different solutions, determining the best solution with the reasons, planning and executing solutions, and expressing the best solution in the form of creative texts.

The Validity of the Test

The mean of the validity test score was 3.74. This suggests that the validity of the assessment can be categorized as high. The expert judgment on construct and content test validity also showed high validity of the assessment because the four experts stated that all items were in line with the theoretical critical–creative reading assessment and the framework of problem-solving skills. The result of the validity test can be seen in Table 4.

Table 4.

The Results of the Test Validity of Critical–Creative Reading Assessment Based on Problem Solving.

No.	Aspect	Description	Result (average)	Criteria
1.	Construct validity
	The appropriateness of the test items and the rational/theoretical of the critical–creative reading assessment based on problem solving	The item is appropriate to measure critical reading skills, that is, the ability to assess or judge	4.00	High
		The item is suitable to measure creative reading skills, which are the skill to make or produce texts	3.60	High
		The item is appropriate to measure problem-solving ability	4.00	High
		The item is appropriate to measure students’ sensitivity to the social issues in their surroundings	3.40	High
2.	Content validity
	The appropriateness of the test items with the framework of critical–creative reading assessment based on problem solving	The test item is appropriate to measure the ability to find the problem	4.00	High
		The test item is appropriate to measure the ability to find the root of the problem	3.60	High
		The item is appropriate to measure the ability to plan and execute solutions	3.60	High
		The item is appropriate to measure the ability to express the best solution in the form of creative text	3.70	High
Average			3.74	High

There were some important suggestions from the expert team that serves as a reference for refining these developed assessments. The suggestions from the expert team are presented in Table 5.

Table 5.

Suggestions From the Expert Team.

No.	Aspects	Suggestions
1.	Construct validity	Problems to be solved should focus on social issues that are close to the students but have national/global impacts, such as drug abuse, speech in social media, rampant fraud, or early marriage
2.	Content validity	Phrase: The root of the problem should be added to the synonym to avoid misinterpretationFor students to distinguish between the problem and the root of the problem, the item needs to be elaborated with the task of creating a diagram showing the root due to the discovery of the problem
		The task of finding the root of this problem is important so that students are accustomed to solving problems based on the root of the problem rather than on surface-level issues
3.	Scoring guideline/rubric	The scoring guideline needs to be weighted for each item because the level of difficulty is different

Feedback from the experts has been considered to improve the assessment (see Supplemental Appendix 2).

The Reliability of the Test

The inter-rater reliability test was performed by asking four practitioners to score the work results of the 36 students, using the scoring guideline (see Table 2). The result of the reliability test was analyzed using ANOVA and ICC (see Tables 6 and 7).

Table 6.

The Result of Reliability Test Analysis Using ANOVA.

ANOVA
		Sum of squares	df	Mean square	F	Significance
Between people		60.722	35	1.735
Within people	Between items	0.56	3	.019	.118	.949
	Residual	16.444	105	.157
	Total	16.500	108	.153
Total		77.222	143	.540
Grand mean = 3.1389

Note. ANOVA = analysis of variance.

Table 7.

The Result of Reliability Test Analysis Using ICC.

	Intraclass Correlation^b	95% Confidence Interval		F test with true value 0
	Intraclass Correlation^b	Lower Bound	Upper Bound	Value	df1	df2	Significance
Single measures	.716^a	.586	.825	.11.078	35	105	.000
Average measures	.910^c	.850	.950	.11.078	35	105	.000

Note. ICC = intraclass correlation coefficient.

The result of the ANOVA showed that there was no difference in the inter-rater scores because p > .05 and the result of the ICC analysis showed a satisfactory score, which is r_xx = .716.

Discussion

Based on the result of the test validity, proved by studying the construct and content validity, it can be concluded that the assessment developed in this study had high validity to be used to measure critically creative reading skills. This is because the assessment involves three cognitive operations, which are as follows: (a) finding the problem and the root of the problem, (b) planning and executing the solution, and (c) expressing the best solution in the form of creative texts. Finding the problem and the root of the problem is a critical reading activity done first by understanding the story problem solving, identifying the problem contained in it, analyzing it, and then determining the root of the problem. This is in harmony with the critical thinking feature that involves bringing the outside knowledge and values to deliver and evaluate the presentation and decide what to ultimately accept as true (Facione, 2015). The essence of critical reading activity is an activity to analyze and evaluate what we read, working out what is good and what is not, and why.

Finding three significantly different solutions is one form of creative reading skills. This is because students not only understand the text but also respond to the text content. Students are also asked to express the best solution in essay form. The essay contains at least four or five full paragraphs containing one introductory paragraph, two or three paragraphs detailing the original ideas offered for problem solving, and a closing paragraph. The creativity in reading involves an output which is the result of a process, and all processes require some form of input. The input of the creative result is an idea derived from critical reading; the idea is then processed into a creative work. The basic information processing model of creative reading is usually represented as Input → Process → Output. Creative skills are also characterized by original ideas relating to solutions to solve problems, the use of complex sentence structures with effective grammatical structures, appropriate and varied use of vocabulary and conjunctions, and minimal spelling errors. This is in line with the characteristics of creativity, which is associated with imagination, innovation, originality, and invention (Pope, 2005).

The assessment tools in the form of story problem-solving essay tests were taken from the news texts. From the news text, it can be found real problems, such as social problems faced by the community around the students, so that students can play an active role in helping solve the problems faced by society or themselves. This is in line with the opinion of Harrington (2018), which states that the characteristics of creativity are new, useful, or valuable. Useful refers to the fact that creative “thing” has to make sense or be useful for a particular context. This is also in line with the opinion which states that the problem-solving-based assessment requires students to respond critically and simultaneously construct a product rather than simply select from a set of predefined alternatives or answers.

The test results of the reliability test, which were performed using an inter-rater reliability test to measure the reliability of essay tests and analyzed using SPSS 26, indicated that the developed essay test proved to be reliable for measuring critical–creative reading ability. This means that this assessment can produce a constant, consistent, and unchangeable or reliable measurement result (O’Malley & Pierce, 1996).

The assessment developed in this study can be categorized as an authentic assessment seen from the context, students, and assignments given (Herrington & Herrington, 2006). In relation to the context, this assessment uses a real-world context. The next criterion is the students. An authentic assessment must contain problem-solving and higher order thinking skills. In the context of the task, what is required is to stimulate various active responses which involve complexity in decision making and require assessment that is integrated with student activities (Mueller, 2005).

Conclusions and Recommendations

From the above description, it can be concluded that the critical–creative reading assessments based on problem solving had high validity, as seen from the construct and content validity. This means that this assessment can be used to measure students’ critical–creative reading skills, especially for high school students. In addition to being valid, this assessment is also reliable, which means the critical–creative reading assessments based on problem solving can produce trusted or reliable measurement results. Furthermore, this assessment was also comprehended that students could measure the usefulness of the Indonesian language in daily life and play an active role in society.

Based on the results, it is recommended to implement this problem-solving-based critical–creative reading assessment to complement the multiple-choice tests that have been used in national examinations, school exams, or formative examinations. In summary, this assessment proves to be valid and reliable to be used as an assessment to improve critical–creative reading skills, which are the essential skills needed to survive and thrive in the 21st century.

Supplemental Material

Appendix – Supplemental material for The Development of a Critical–Creative Reading Assessment Based on Problem Solving

Supplemental material, Appendix for The Development of a Critical–Creative Reading Assessment Based on Problem Solving by Endah Tri Priyatni and Martutik in SAGE Open

Footnotes

Acknowledgements

Our gratitude to the Directorate of Research and Community Service, the Directorate of Higher Education, the Ministry of Research, Technology, and Higher Education of the Republic of Indonesia for providing a Research Grant in 2018.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the Directorate of Research and Community Service, the Directorate of Higher Education, the Ministry of Research, Technology, and Higher Education of the Republic of Indonesia under contract number: 21.8.23/UN32.14/LT/2017.

ORCID iD

Endah Tri Priyatni

Supplemental Material

Supplemental material for this article is available online.

References

Anderson

J. R.

(2010). Cognitive psychology and its implications (7th ed). Worth Publishers.

Asilioglu

(2008). The importance of critical reading for cognitive learning and the ways to improve it. Dicle University Journal of Ziya Gökalp Education Faculty, 10, 1–11.

Beghetto

R. A.

Sriraman

(Eds.). (2017). Creative contradictions in education: Cross disciplinary paradoxes and perspectives. Springer International Publishing.

Costa

A. L.

(Ed.). (1991). Developing minds. Association for Supervision and Curriculum Development.

Facione

P. A.

(2015). Critical thinking: What it is and why it counts. Measured Reasons.

Griffin

Care

(Eds.). (2015). Assessment and teaching of 21st century skills methods and approach. Springer Dordrecht Heidelberg.

Harrington

D. M.

(2018). On the usefulness of “value” in the definition of creativity: A commentary. Creativity Research Journal, 30(1), 118–121. https://doi.org/10.1080/10400419.2018.1411432

Harsiati

(2017a). Feedback and self regulation in writing learning assessment in junior high school. Journal of Intensive Studies on Language, Literature, Art, and Culture, 1(1), 71–85.

Harsiati

(2017b). Karakteristik tes literasi membaca pada Programme for International Student Asessment (PISA) [Characteristics of reading literacy tests in the Program for International Student Assessment]. Litera: Jurnal Bahasa, Sastra, dan Pengajarannya, 17(1), 90–106. http://doi.org/10.17977/um008v1i22017p001

10.

Herrington

(2006, July 10–12). Authentic conditions for authentic assessment: Aligning task and assessment [Conference session]. Proceedings of the 29th HERDSA Annual Conference, Perth, WA, Australia. http://www.herdsa.org.au

11.

Hudson

(2010). Teaching second language reading (3rd print.). Oxford University Press.

12.

Jonassen

D. H.

(2010). Learning to solve problems: A handbook for designing problem-solving learning environments. Taylor & Francis.

13.

Marin

L. M.

Halpern

D. F.

(2011). Pedagogy for developing critical thinking in adolescents: Explicit instruction produces greatest gains. Thinking Skills and Creativity, 6, 1–13. https://doi.org/10.1016/j.tsc.2010.08.002

14.

Mueller

(2005). The authentic assessment toolbox: Enhancing student learning through online. Journal of Online Learning and Teaching, 1(1), 1–7.

15.

Nakano

T. C.

Wechsler

S. M.

(2018). Creativity and innovation: Skills for the 21st. Estudos de Psicologia (Campinas), 35(3), 237–246. http://doi.org/10.1590/1982-02752018000300002

16.

O’Malley

J. H.

Pierce

L. V.

(1996). Authentic assessment for English language learners: Practical approaches for teachers. Addison-Wesley.

17.

OECD. (2013). PISA 2012 assessment and analytical framework. OECD.

18.

OECD. (Ed.). (2014). Creative problem solving: Students’ skills in tackling real-life problems. OECD.

19.

Paul

Elder

(2008). The thinker’s guide to the nature and functions of critical & creative thinking. Foundation for Critical Thinking Press.

20.

Pope

(2005). Creativity: Theory, history and practice. Taylor & Francis.

21.

Priyatni

E. T.

Martutik

(2019, April). Has the Indonesian national test been measuring students’ literacy and critical literacy? (Study in National Examination in Indonesia) [Paper presentation]. LSCAC International Conference, Hue City, Vietnam.

22.

Priyatni

E. T.

Nurhadi. (2017). Membaca kritis dan literasi kritis [Critical reading and critical literacy]. TSmart.

23.

Sultan

A. R.

Nurhadi

E. T. P.

(2017a). The development of a critical reading learning model to promote university students’ critical awareness. The New Educational Review, 48(2), 76–86. https://doi.org/10.15804/tner.2017.48.2.06

24.

Sultan

A. R.

Nurhadi

E. T. P.

(2017b). The effect of the critical literacy approach on pre-service language teachers’ critical reading skills. Eurasian Journal of Educational Research, 17(71), 159–174. https://doi.org/10.14689/ejer.2017.71.9

25.

Sultan

A. R.

Nurhadi

E. T. P.

(2018). Development of mass media text-based instructional materials to improve critical reading skills of university students. Pedagogika, 132(4), 26–47. https://doi.org/10.15823/p.2018.32

26.

Swann

Pope

Carter

(Eds.). (2011). Creativity in language: The state of the art. Palgrave Macmillan.

27.

Syahrin

A. H.

Priyatni

E. T.

(2019). Creative thinking patterns in student’s scientific works. Eurasian Journal of Educational Research, 81, 21–36. https://doi.org/10.14689/ejer.2019.81.2

28.

Wallace

(2010). Critical reading in language education. Palgrave Macmillan.

29.

Wheeler

L. K.

(2013, April 20). Critical reading of an essay’s argument. http://Web.Cn.Edu/Kwheeler/Reading_basic.Html

30.

World Economic Forum. (2015). New vision for education: Fostering social and emotional learning through technology. World Economic Forum.

31.

Zintz

M. V.

Maggart

Z. R.

(1984). The reading process: The teacher and the learner. Wm. C. Brown.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.18 MB