Sage Journals: Discover world-class research

Abstract

Higher education in Germany is characterized by specific historicalcultural traditions stemming from von Humboldt’s humanistic understanding of education. In this context, higher education focuses on a variety of multi-perspective goals. Accordingly, higher education focuses on both the acquisition of knowledge and skills in specific domains (e.g., medicine, economics) and on the development and promotion of generic (interdisciplinary) skills (e.g., problem solving, critical thinking), whereby the latter aspect is becoming increasingly more important in the 21st century. This paper analyses ten years of competence research in higher education in germany, and considers advances and perspectives.

Keywords

german education higher education competence research

Introduction and background

Higher education in Germany is characterized by specific historical-cultural traditions stemming from von Humboldt’s humanistic understanding of education (for a comparison with US traditions in higher education, see Beck, 2020). In this context, higher education focuses on a variety of multi-perspective goals. According to the German Council of Science and Humanities (2015), modern academic education in Germany comprises three central pillars: competence in scientific disciplines, preparation for the labor market, and personal development. From a research perspective, this understanding of education is in line with a holistic notion of competence (Weinert, 2001). Accordingly, higher education focuses on both the acquisition of knowledge and skills in specific domains (e.g., medicine, economics) and on the development and promotion of generic (interdisciplinary) skills (e.g., problem solving, critical thinking), whereby the latter aspect is becoming increasingly more important in the 21st century (Pellegrino & Hilton, 2012), as emphasized by current surveys amongst employers (e.g., German Chambers of Industry and Commerce, 2015). A competence portfolio of this kind, acquired over the course of academic studies, is required for all professionals and engaged citizens and allows for lifelong learning, which is necessary in an increasingly changing age of information. One of the main ideas and goals in the area of competence orientation in Germany and abroad – in contrast to the focus on knowledge – is therefore to promote the so-termed transferrable skills that can be applied to solve problems in various professional and real-life situations, and to avoid the acquisition of so-termed “inert knowledge” (Pellegrino & Hilton, 2012).

Literature review on the state of research in 2010

Exactly ten years ago, the state of international research on student competencies and learning outcomes acquired in higher education was systematically analyzed (Zlatkin-Troitschanskaia & Kuhn, 2010). This comprehensive literature review, funded by the German Federal Ministry of Education and Research, illustrated how deficient the state of research in Germany, and also internationally, actually was ( for further overviews of the state of competence research, see also, e.g. Coates 2014; for further meta-studies, see Zlatkin-Troitschanskaia et al., 2017). This report, based on the analysis of several dozens of international projects in the field, showed that while there were indeed individual research initiatives in various countries, there was hardly any systematic and consolidated research on the valid assessment of student competencies in higher education.

Despite the sociopolitical consensus and the corresponding growing competence orientation – not least in the context of the ongoing Bologna reform in Europe – there has only been limited empirical research until the last decade, particularly regarding the competence level and development of higher education students and graduates. This practically and politically controversial deficit was primarily due to the lack of theoretical-conceptual models that would allow for an empirically precise description and operationalization of skills and knowledge to be acquired during the course of academic studies. Corresponding test instruments and measurement methods for the objective, valid, and reliable assessment of knowledge and skills acquisition and development were also scarcely available in Germany.

Against the background of deficient research, the conceptual heterogeneity and ambiguity in the field can also be explained in that several terms such as ‘knowledge’, ‘skills’, ‘competence’, ‘learning outcomes’ have been used in part inconsistently. At the same time, however, competence research has developed from the integration of many different disciplines such as education, psychology, sociology, where different terminologies have been established depending on the research tradition. In this paper, an attempt is made to use these terms according to their original and established usage in the corresponding linguistic-cultural as well as scholarly contexts (in the US and Germany). In this context, ‘learning outcomes’ refers primarily to formally measurable indicators such as grades or test scores, while ‘competencies’ refers primarily to psychometrically measurable personal states and abilities of students.

Internationally as well, relatively little competence research was being conducted in higher education. For instance, the U.S. Department for Education (2009) stated that “[there is a] remarkable absence of accountability mechanisms to ensure that colleges succeed in educating students”. At both the national and international level, hardly any research has been conducted regarding the questions what do graduates know, and how long will they know it? According to the internationally established “Curriculum-Instruction-Assessment Triad” model (Pellegrino et al., 2001), a common problem in higher education has been that the program designs, instructional activities, and assessment strategies used often do not lead to the desired student learning outcomes (Pellegrino, 2017).

Overall, this 2010 review of the national and international state of research on the assessment of student competencies documented considerable systematic deficits in both assessment practice and competence research in higher education, particularly with a view to the ongoing challenges of competence-oriented education. These developments call for new research-based approaches and methods for validly measuring students’ competencies over a course of academic studies within and across various disciplines and for effectively fostering these competencies among students in higher education.

Political and practical developments and challenges in higher education

In Europe and Germany, this situation has been made particularly challenging by the Bologna reform, which has been ongoing since 2000 (e.g., Adelman, 2009). In the first decade of the 21st century, there was a substantial structural shift in higher education from well-established diploma study programs to the newly implemented bachelor's and master's degrees. These comprehensive political reforms over the past decades have included, in particular, a trend toward modularizing degree courses, attract more international students, and award internationally transferable credit points. These trends, in turn, have contributed to the already existing and ever-increasing diversity of students, degree courses, and institution profiles in higher education. These harmonizations and diversifications have both had a substantial impact on all participants in and stakeholders of higher education. Educators and administrators have been challenged to provide education that is both of high quality and tailored to the individual needs of the highly heterogeneous student body.

After the implementation of this new academic degree system in Germany, first surveys among the most important stakeholders, such as businesses, criticized substantial deficits of both bachelor and master graduates. For instance, a business survey from the year 2015 by the Association of German Chambers of Industry and Commerce, “Competent and practically relevant – the economy’s expectations of higher education graduates”, reported low satisfaction levels with bachelor graduates: Less than half of businesses saw their expectations met and only 16% considered bachelor graduates well prepared for the labor market (German Chambers of Industry and Commerce, 2015). Surveys of this kind indicated significant deficits in central professional and generic competencies, for instance domain-specific expertise and methodological skills. In particular, the businesses criticized that essential 21st century skills are not sufficiently fostered in university studies. These skills have been considered particularly important for professional and everyday decisions and actions, for being engaged and globally oriented citizens, and for ensuring lifelong learning (Pellegrino & Hilton, 2012; Rotherham & Willingham, 2001). Overall, the business surveys indicated that the three central pillars of higher education (competence in scientific disciplines, preparation for the labor market, and personal development) had not been effectively promoted in higher education, and they called for a stronger focus on competence orientation.

Notably, the surveys of university students indicated similar deficits: Graduates of the newly established bachelor and master study programs also strongly criticized that central facets of professional competence such as critical thinking, problem-solving, communication, and practical skills are not promoted sufficiently in their degree courses (German Centre for Higher Education Research and Science Studies, 2013).

An additional challenge that concerned practitioners and politicians with regard to the implementation of the new, increasingly divergent study structure in Germany was the lack of validity and comparability of grades and certificates in higher education. For instance, a study conducted by the German Council of Science and Humanities (2012) reported that established (examination) practices to measure student learning outcomes over the course of study in the form of grades and certificates can only offer vague indications of the competencies actually acquired by students in higher education and are only marginally comparable across Germany or (even less so) between countries. In Germany, the final grade of a study degree depends not only on the students’ performance in class or exams but also on the subject they study and the university they study at.

At the international level, many OECD reports have also illustrated that student learning outcomes in higher education are hardly comparable internationally (Organisation for Economic Co-operation and Development (OECD), 2012, 2013b). The AHELO study highlighted this challenge as well (OECD, 2013a). Shortcomings in terms of the comparability of higher education have been also indicated by other studies. For instance, in another OECD study, the PIACC study, Japanese high school students showed the same level of literacy skills as university graduates from Italy (Zabal et al., 2013). Significant deficits among university students in terms of basic knowledge (e.g., in subjects such as economics) and skills, were reported in several comparative studies (e.g., Zlatkin-Troitschanskaia & Kühling-Thees, 2020).

This deficient situation at the national and international level has been also highlighted in numerous reports in the press and media. In particular, the question has been discussed how this challenging situation can be improved effectively, especially in the context of ever-increasing globalization and student mobility (van Damme, 2015). The substantial increase in students with migration or flight backgrounds in higher education has made this situation even more dramatic, and has demonstrated that innovative instruments to measure both students’ levels of competence at the beginning of their degree courses and their learning progress and outcomes during their studies in higher education in a valid and reliable way are urgently required (Zlatkin-Troitschanskaia et al., 2018) to be able to effectively develop and improve curricula and instruction in the sense of the Assessment Triad (Pellegrino et al., 2001) and, consequently, to foster the targeted learning outcomes in higher education.

Modeling and measuring competencies in higher education: The KoKoHs program

Preparation phase of the research program

To address these challenging trends in higher education practice and to substantially promote research on the valid assessment of student competences and their learning outcomes, the German Ministry of Education and Research (BMBF) funded the above-mentioned meta-study on the state of national and international research and best practices on this topic (Zlatkin-Troitschanskaia & Kuhn, 2010). This study illustrated some significant related work in many countries including several assessments, for instance performance-based assessments such as the College Learning Assessments (CLA, Shavelson, 2010) as well as other types of assessments developed by the Education Testing Service (ETS, 2010) in the U.S., and other testing institutes such as CENEVAL in Mexico (2013), FUNDAÇÃO CESGRANRIO in Brazil (FUNDAÇÃO CESGRANRIO, 2016) or the ACF in Australia (Wheelahan, 2011). As this meta-study also demonstrated, at the national level, approaches to model and measure student learning outcomes in higher education were found in several countries. These approaches constitute a valuable basis for further research (for an overview, see also Coates, 2014). However, this study also revealed that there only little research existed on this topic, especially at the international level, including a lack of large-scale programs and comparative studies in higher education.

Development of competence models and test instruments (1st research phase)

Following the report of this meta-study, the German Federal Ministry of Education and Research launched the first nationwide program for modeling and measuring competencies and student learning outcomes in higher education in Germany. The 1st funding phase ran from 2011-2015 (Zlatkin-Troitschanskaia et al., 2017). At this 1st working stage, more than 200 researchers from all across Germany and from various fields of expertise such as subject-specific didactics in higher education, learning psychology, and psychometrics developed initial modeling and assessment frameworks and the corresponding test instruments for the valid assessment of student competencies in close interdisciplinary collaboration. In the context of 24 collaborative projects, the KoKoHs researchers focused on five major study domains in higher education, including business and economics, engineering, and teacher education. The developed assessments focused on both domain-specific (e.g., domain-specific knowledge) and generic competencies (e.g., scientific reasoning) that students and graduates are expected to acquire in their course of study.

The KoKoHs projects followed the research phases as recommended in Mislevy’s Evidence-Centered Design (Mislevy & Haertel, 2006). When developing models that described targeted student learning outcomes, the KoKoHs researches considered two perspectives: the learning goals declared in study curricula and regulations, and the labor market’s professional and social requirements, i.e. the current expectations of employers and other stakeholders from higher education graduates (e.g. in terms of transferable skills and lifelong learning). With the test instruments developed in KoKoHs, the knowledge and skill levels of students and graduates as described in the competence models can be assessed in an objective and reliable manner, as indicated by the initial validation results (Zlatkin-Troitschanskaia et al., 2017).

These developed competence models and assessment tools were one of the key outcomes of the 1st funding phase. The findings on the competence levels of students across the different phases of their university education were another equally important result of this working stage. These findings indicated substantial deficits in student competencies and, in this way, demonstrated the practical potential of using valid diagnostic instruments in higher education. The valid diagnosis of students’ study-related preconditions is widely recognized as a necessary prerequisite for identifying specific student needs and for developing approaches and training tools to effectively promote student competencies in higher education (for an overview of the key results of this working stage incl. an overview of assessments developed in KoKoHs, see Zlatkin-Troitschanskaia et al., 2017).

The results from this 1st funding phase of the KoKoHs program were critically evaluated in the external international audit. This examination revealed two shortcomings at the end of the 1st working stage of KoKoHs: First, many of the developed test instruments had not yet been comprehensively validated. Rather, evidence on the five validation criteria (e.g. response processes, relation to other variables) according to the Standards for Educational and Psychological Testing (see American Educational Research Association, American Psychological Association & National Council on Measurement in Education (AERA, APA, NCME), 2014) was provided for only a few of the KoKoHs projects. Second, most test instruments developed in KoKoHs were paper-and-pencil-based, and only a few innovative assessments, for instance with technology-based formats, had been developed in the 1st phase.

Validation and methodological innovations (2nd research phase)

Following the recommendations from the international audit and the results of the 1st working stage, the 2nd funding phase of the KoKoHs research program was launched by the BMBF in 2015. This working stage, which ran until 2020, focused on comprehensive validation (as recommended by the AERA, APA, NCME, 2014 standards) and on methodological evaluations.

In the 2nd funding phase, the research program focused on three large clusters, which consisted of 16 cross-university collaborative projects:

Four projects focused on domain-specific competencies in economics and medicine,

Five projects focused on generic competencies such as scientific reasoning and self-regulation, and

Seven projects focused on teacher education in numerous subjects such as mathematics, physics, or economics

An overview of all 16 KoKoHs projects from the 2nd funding phase is illustrated in Zlatkin-Troitschanskaia, Pant, Toepper et al. (2020).

A common research objective of all of these projects in the 2nd funding phase was the comprehensive validation of the KoKoHs assessments following the five criteria of AERA, APA, NCME (2014). About half of the projects in this working stage focused in particular on the development and validation of complex technology-based assessments, which were mostly performance-oriented (for an overview, see Zlatkin-Troitschanskaia & Shavelson, 2019). Innovative technology-based formats were implemented in these projects, for instance, for assessing students’ multiple document comprehension while they are reading several academic texts (Hahnel et al., 2019), or for assessing students’ self-regulation skills using computer-based learning diaries and mobile assessment apps (Steuer et al., 2019). Computer-based assessments such as video-based tests and simulations have been used in the majority of KoKoHs projects. In addition, in about one third of the projects, assessments from one domain were adapted and/or transferred to another domain, for example from mathematics to economics (e.g., Jeschke et al., 2019; König et al., 2020).

In this working phase, several projects used a longitudinal design to make valid statements about the development of competencies over the course of academic studies and about the main influencing factors on competence acquisition in higher education (for teacher education, see, e.g., Vogelsang et al., 2019, for economics, Zlatkin-Troitschanskaia et al., 2019). In a few projects, competence development was measured longitudinally over the entire course of study in higher education, for example, to examine the learning trajectories of economics students over their course of study (Zlatkin-Troitschanskaia et al., 2019).

Comparative analyses were also carried out in some of the projects ( for teacher education, see, e.g., Kaiser & König, 2020; for economics, Zlatkin-Troitschanskaia & Kühling-Thees, 2019). In these international studies in collaboration with KoKoHs partners, many test instruments developed and validated in KoKoHs were tested and used in several other countries (e.g., in Japan, the US, and China).

Overall, in the 2nd phase, the KoKoHs projects focused on numerous working areas, using KoKoHs test instruments and validating them with regard to the five criteria, which include relationships with external variables and consequences of testing (for an overview, see Zlatkin-Troitschanskaia, Pant, Toepper et al., 2020). The comprehensive validation analyses include various innovative approaches such as log file analyses, eye tracking studies, etc., and numerous qualitative, quantitative, and integrative methods.

The scientific transfer project scientifically coordinated the research work performed in the 16 collaborative projects. As a survey among the 16 KoKoHs projects conducted by the transfer project showed, more than 80,000 students from the five major study domains in higher education were assessed across Germany in the KoKoHs program (Zlatkin-Troitschanskaia, Pant, Toepper et al., 2020). The data collected and analyzed in the KoKoHs projects provide valuable evidence-based insights into competence acquisition in higher education and the main influencing factors thereon. These findings offer also indications on how different competence facets can be effectively promoted over the course of study.

Key outcomes of the KoKoHs program

In the context of the two funding phases and, consequently, over a time period of ten years in total, the KoKoHs program assessed and researched competencies across the entire continuum of students' dispositions, situation-specific skills, and performance (for the competence continuum model, see Blömeke et al., 2015). Therefore, the competence assessment conducted in KoKoHs included:

students’ domain-specific and generic cognitive and motivational-affective (study-related) dispositions (e.g., prior education and subject-related knowledge),

their situation-specific skills, assessed, for instance, by examining their decision-making skills in certain situations (e.g., computer-based simulations), and

their observed performance (e.g., by examining how (prospective) teachers actually act in class using video-based observation)

as well as relationships or dependences between these three areas i-iii.

In addition, information on numerous central contextual factors, which might influence competence acquisition and development over a course of study, was also collected and included in the modeling and analyses (e.g., at the institutional level by conducting multi-level analyses).

With a view to the well-established taxonomy for learning that distinguishes between different levels of cognitive processes and knowledge (Anderson & Krathwohl, 2001), the competence models and corresponding test instruments developed in KoKoHs cover all of these (meta)cognitive levels. They range from the understanding of basic domain-specific concepts (e.g., drafting financial plans, see Förster et al., 2015) to the solving of complex performance-based tasks focusing on sustainability (Seeber et al., 2019), which requires a combination of higher-order metacognitive skills.

The KoKoHs test instruments were tested for numerous areas of application within higher education (i.e., from the introductory study phase to the end of master’s degree courses) as well as in the subsequent practical training phase (e.g., in schools or hospitals). Accordingly, some assessments were used in the practical training year in teacher education and medicine or among graduates who had already entered professional employment (e.g., Kuhn et al., 2020). Some of the assessments were used to assess the competence development of students over their entire course of study as well as during their subsequent training phase, for instance, in teacher education (e.g., Vogelsang et al., 2019).

Over the course of the two research phases in the KoKoHs program, the developed test instruments were tested both in terms of their suitability for process diagnostics/formative assessment over the course of studies and summative evaluation at the end of a study stage, e.g., at the end of bachelor’s or master’s degree course. Many KoKoHs projects also demonstrated the predictive or prognostic validity of the developed tests. For instance, several KoKoHs tests contribute to the prediction of students’ grades during their studies as well as of their academic success, changing their subjects, or dropping out of their degree courses in a statistically significant way (e.g., Kühling-Thees et al., 2020).

The test instruments developed in KoKoHs and the results of the comprehensive and in-depth validation analyses were documented by the KoKoHs projects in the so-termed “Portfolio of KoKoHs Assessments” in German and English (see Zlatkin-Troitschanskaia, Pant, Nagel et al., 2020). This assessment portfolio includes comprehensive information on most of the test instruments developed in KoKoHs, such as the test design, assessed content, test structure as well as their applicability in research and practice, including the corresponding validity analyses and the necessary information to use these assessments in both higher education research and practice.

After ten year of conducting research in the KoKoHs program, the main outcomes can be summarized as follows: The KoKoHs projects addressed practically (and socially) important and scientifically innovative questions focusing on the effective acquisition and the development of students’ competencies in higher education and the main influences thereon. In this research context, innovative analytical approaches were used. (Quasi-)experimental, longitudinal, and multi-level analytical studies were performed using various assessment formats, including innovative computer-based performance assessments.

The measurement methods and assessment tools developed in KoKoHs were comprehensively validated as recommended by the AERA, APA, NCME standards (2014). In the in-depth validation studies, the suitability of the test instruments for the valid and reliable assessment of domain-specific and generic competencies over a course of studies for different practical purposes (e.g., formative or summative evaluation) was examined and established in higher education as well as during subsequent practical training. In this context, innovative methods to increase the practical applicability of the assessments (e.g., using adaptive testing approaches) were developed (for an overview, see Zlatkin-Troitschanskaia, Pant, Nagel et al., 2020).

The results obtained in Germany-wide KoKoHs studies assessing a total of more than 80,000 students at more than 300 higher education institutions have demonstrated that the competence models and test instruments developed in KoKoHs for various application areas constitute a well-founded basis for the valid assessment of competence development in higher education. These findings also indicate a valuable potential in terms of fostering students’ competencies in higher education.

Frontiers and perspectives in competence research and praxis in higher education

Now that suitable competence models and valid test instruments have been developed and a strong collaboration network with numerous higher education institutions has been established, the KoKoHs researchers aim to make it possible that the assessments continue to be used in higher education research, and they also support their wide-spread implementation in practice.

Since the focus of the 1st and 2nd phase of the KoKoHs program was on the development and validation of assessment frameworks and tools, the test instruments have not been systematically implemented into higher education teaching and learning practice to date. Half of the KoKoHs projects are addressing this focus in the currently ongoing funding phase, which is scheduled to run for two years (2020-2022). In this (third) working stage, seven KoKoHs projects from the domains of teacher education and generic competencies are focusing on the Curriculum-Instruction-Assessment Triad (Pellegrino et al., 2001) and aiming to transfer the assessments developed and validated in KoKoHs as a central pillar in this triad into higher education teaching and learning practice. To achieve this goal, these transfer projects are building on established collaborations with higher education practitioners (lecturers) and focusing on developing practical approaches together as to how teaching and learning at the respective universities can be effectively supported by using the KoKoHs assessments. In this way, the implementation of innovative competence-oriented concepts in teaching and examination practice (teaching-learning-assessment tools) is being systematically promoted. This includes increasing the suitability of competence assessments for university practice, for instance by developing and implementing feedback systems and practical guidelines for the interpretation of the assessment results.

Throughout this ongoing transfer phase, KoKoHs is not only implementing assessments for entry diagnostics, formative and summative evaluation of competence levels, and their development over the course of studies but also aiming to strengthen collaborations with the stakeholders at higher education institutions. In the transfer studies, the KoKoHs researchers are offering continuous advice and support to ensure that education practitioners learn how to appropriately interpret the assessment results and how to identify differences within and across their classes and institutions. Instruments that appropriately address the specific needs of the individual institutions are being implemented into practice, including targeted feedback platforms to allow the practitioners to make the best possible use of these assessment tools.

For instance, study entry diagnostics conducted at the beginning of studies to validly measure students’ competencies in certain domains can be used to redesign and optimize the teaching and learning opportunities offered in the study entry phase. In this way, the students’ specific needs and deficits can be addressed directly and their progress can be assessed during this phase through repeated measurement to provide students and their teachers and university administrators with valid information on the overall performance of the group. The results from competence diagnostics can be compared to students’ learning requirements and can be used to answer practical questions such as how many of the students are likely to need targeted tutoring and support programs or how many of the students are at risk of discontinuing their studies due to individual competence deficits (e.g., language or mathematical skills).

Now that KoKoHs has been running for 10 years, a meta-study is currently being conducted and scheduled to run until spring 2021, with the aim of systematically reviewing and mapping the competence assessment landscape in Germany and of identifying further potentials for competence research and practice in higher education at the national and international level. The results of this meta-analysis will be combined with feedback from higher education practice from ongoing transfer studies.

Although this meta-study has not yet been completed, it can already be stated at this stage that from a research perspective, the KoKoHs models and test instruments are laying the groundwork for future analyses of the complex relationships between so-termed input (e.g., learning preconditions), processes (e.g., attended learning opportunities), and higher education learning outcomes (acquired competencies) for Germany as well as for international comparative analyses in certain domains (e.g., economics, teacher education). However, to explore these relationships, more longitudinal studies and multilevel analyses need to be performed in higher education using the KoKoHs assessments. To increase the external validity of the results, more randomized experimental studies that simulate various actual (digital and non-digital) teaching and learning contexts in higher education are required as well.

To specifically address the abovementioned trends and challenges in the context of the increasing internationalization in higher education, the collaborative research on international adaptations should also be continued, and international validation and implementation studies need to be conducted, for instance by launching additional comparative studies. The ongoing digital approaches offer a new promising prospective for international collaborative research in the field of competence assessment and development in higher education.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research in this article was part of the KoKoHs program funded by the German Ministry for Education and Research.

References

Adelman

(2009). The Bologna process for U.S. eyes: Re-learning higher education in the age of convergence. Institute for Higher Education Policy.

American Educational Research Association, American Psychological Association & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

Anderson

L. W.

Krathwohl

D. R.

(2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's taxonomy of educational objectives. Longman.

Beck

(2020). On the relationship between “Education” and “Critical Thinking”. In Zlatkin-Troitschanskaia

(Ed.), Frontiers and advances in positive learning in the age of information (PLATO) (pp. 73–87). Springer.

Blömeke

Gustafsson

J.-E.

Shavelson

R. J.

(2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für Psychologie, 223(1), 3–13. https://doi.org/10.1027/2151-2604/a000194

Coates

(Ed.). (2014). Higher education learning outcomes assessment. Lang.

Educational Testing Service. (2010). ETS proficiency profile user’s guide.

Förster

Brückner

Zlatkin-Troitschanskaia

(2015). Assessing the financial knowledge of university students in Germany. Empirical Research in Vocational Education and Training, 7, 1–20. https://doi.org/10.1186/s40461-015-0017-5

FUNDAÇÃO CESGRANRIO. (2016). Exame Nacional de Cursos (ENC or Provão).

10.

German Centre for Higher Education Research and Science Studies. (2013). Studienqualitätsmonitor. 2012. Studienqualität und Studienbedingungen an deutschen Hochschulen, HIS: Projektbericht, Juli 2013 [Monitor for Quality of Study 2012. Quality and Conditions of Study in German Higher Education, HIS: Project Report July 2013]. DZHW.

11.

German Chambers of Industry and Commerce. (2015). Kompetent und praxisnah – Erwartungen der Wirtschaft an Hochschulabsolventen. Ergebnisse einer DIHK Online-Unternehmensbefragung [Skillful and experienced: Economy's expectations towards university graduates. Results of a survey by the Chamber of Industry and Commerce].

12.

German Council of Science and Humanities. (2012). Prüfungsnoten an Hochschulen im Prüfungsjahr 2010 [Examination Grades at Higher Education Institutions in 2010]. Report and scientific-political commentary by the Scientific Council from November 9, 2012 (publication No. 2627-12). Wissenschaftsrat.

13.

German Council of Science and Humanities. (2015). Empfehlungen zum Verhältnis von Hochschulbildung und Arbeitsmarkt. Zweiter Teil der Empfehlungen zur Qualifizierung von Fachkräften vor dem Hintergrund des demographischen Wandels [Recommendations on the relationship between higher education and the labour market. Second part of the recommendations on the qualification of skilled workers against the backdrop of demographic change]. WR Publikationen.

14.

Hahnel

Kroehne

Goldhammer

Schoor

Mahlow

Artelt

(2019). Validating process variables of sourcing in an assessment of multiple document comprehension. British Journal of Educational Psychology, 89(3), 524–537. https://doi.org/10.1111/bjep.12278

15.

Jeschke

Kuhn

Lindmeier

Zlatkin-Troitschanskaia

Saas

Heinze

(2019). What is the relationship between knowledge in mathematics and knowledge in economics? Investigating the professional knowledge of (pre-service) teachers trained in two subjects. Zeitschrift für Pädagogik, 4, 511–524. https://doi.org/10.3262/ZP1904511

16.

Kaiser

König

(2020). Analyses and validation of central assessment instruments of the research program TEDS-M. In Zlatkin-Troitschanskaia

Pant

H. A.

Toepper

Lautenbach

(Eds.), Student learning in German higher education: Innovative modelling and measurement approaches and research results. Springer.

17.

König

Bremerich-Vos

Buchholtz

Fladung

Glutsch

(2020). Planning competence of pre-service German language teachers: Conceptualization, measurement, and validation. In Zlatkin-Troitschanskaia

Pant

H. A.

Toepper

Lautenbach

(Eds.), Student learning in German higher education: Innovative modelling and measurement approaches and research results. Springer.

18.

Kuhn

Zlatkin-Troitschanskaia

Lindmeier

Jeschke

Saas

Heinze

(2020). Relationships between domain-specific knowledge, generic attributes, and instructional skills – Results from a comparative study with pre- and in-service teachers of mathematics and economics. In Zlatkin

- Troitschanskaia Pant

H. A.

Toepper

Lautenbach

(Eds.), Student learning in German higher education: Innovative modelling and measurement approaches and research results. Springer.

19.

Kühling-Thees

Happ

Zlatkin-Troitschanskaia

Pant

H. A.

(2020). The impact of entry preconditions on student dropout and subject change in business and economics. In Zlatkin-Troitschanskaia

Pant

H. A.

Toepper

Lautenbach

(Eds.), Student learning in German higher education: Innovative measurement approaches and research results (pp. 351–370). Springer VS.

20.

Mislevy

R. J.

Haertel

G. D.

(2006). Implications of evidence-centered design for educational testing. Educational Measurement: Issues and Practice 25, 6–20. https://doi.org/10.1111/j.1745-3992.2006.00075.x

21.

Organisation for Economic Co-operation and Development. (2012). Assessment of higher education learning outcomes. Feasibility study report: Volume 1. Design and Implementation.

22.

Organisation for Economic Co-operation and Development. (2013a). Assessment of higher education learning outcomes. AHELO feasibility study report – Volume 2. Data analysis and national experiences.

23.

Organisation for Economic Co-operation and Development. (2013b). Assessment of higher education learning outcomes. AHELO feasibility study report – Volume 3. Further insights.

24.

Pellegrino

(2017). Teaching, learning and assessing 21st century skills. In Guerriero

(Ed.) Pedagogical knowledge and the changing nature of the teaching profession. OECD Publishing.

25.

Pellegrino

J. W.

Chudowsky

Glaser

(2001). Knowing what students know: The science and design of educational assessment. National Academies Press.

26.

Pellegrino

J. W.

Hilton

M. L.

(2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. National Academies Press.

27.

Rotherham

A. J.

Willingham

(2001). “21st century” skills. Educational Leadership, 67(1), 16–21.

28.

Shavelson

R. J.

(2010). On the measurement of competency. Empirical Research in Vocational Education and Training, 2(1), 41–63.

29.

Seeber

Michaelis

Repp

Hartig

Aichele

Schumann

Siepelmeyer

, (2019). Assessment of competences in sustainability management: Analyses to the construct dimensionality. Zeitschrift für Pädagogische Psychologie, 33(2), 148–158.

30.

Steuer

Engelschalk

Eckerlein

Dresel

(2019). Assessment and relationships of conditional motivational regulation strategy knowledge as an aspect of undergraduates' self-regulated learning competencies. Zeitschrift für Pädagogische Psychologie, 33(2), 95–104.

31.

Van Damme

(2015). Global higher education in need of more and better learning metrics. Why OECD’s AHELO project might help fill the gap. European Journal of Higher Education, 5(4), 425–436.

32.

Vogelsang

Bowoski

Buschhüter

Enkrott

Kempin

Kulgemeyer

Reinhold

Riese

Schecker

Schröder

(2019). Entwicklung von Professionswissen und Unterrichtsperformanz im Lehramtsstudium Physik – Analysen zu valider Testwertinterpretation [Development of professional knowledge and teaching skills in academic physics teacher education – Analyses regarding the valid interpretation of test scores]. Zeitschrift für Pädagogik, 65(4), 473–491.

33.

Weinert

F. E.

(2001). Competencies and key competencies: Educational perspective. In Smelser

N. J.

Baltes

P. B.

(Eds.), International encyclopedia of the social and behavioral sciences (Vol. 4, pp. 2433–2436). Elsevier.

34.

Wheelahan

(2011). From old to new: the Australian qualifications framework. http://www.academia.edu/1476390/From_old_to_new_the_Australian_qualifications_framework

35.

Zabal

Martin

Klaukien

Rammstedt

Baumert

Klieme

(2013). Grundlegende Kompetenzen der erwachsenen Bevölkerung in Deutschland im internationalen Vergleich [Basic competences of the adult population in Germany in an international comparison]. In Rammstedt

(Ed.), Grundlegende Kompetenzen Erwachsener im internationalen Vergleich – Ergebnisse von PIAAC 2012 [Basic competences of adults in an international comparison – Findings by PIAAC 2012] (pp. 31–76). Waxmann.

36.

Zlatkin-Troitschanskaia

Happ

Nell-Müller

Deribo

Reinhardt

Toepper

(2018). Successful integration of refugee students in higher education: In-sights from entry diagnostics in an online study program. Global Education Review, 5(4), 158–181.

37.

Zlatkin-Troitschanskaia

Jitomirski

Happ

Molerov

Schlax

Kühling-Thees

Förster

Brückner

(2019). Validating a test for measuring knowledge and understanding of economics among university students. Zeitschrift für Pädagogische Psychologie, 33(2), 119–133.

38.

Zlatkin-Troitschanskaia

Kühling-Thees

(2019). Study-related domain-specific and generic competencies of economics students – Insights from a German-Japanese study. Journal of Asia-Pacific Studies, 35, 285–297.

39.

Zlatkin-Troitschanskaia

Kuhn

(2010). Messung akademisch vermittelter Fertigkeiten und Kenntnisse von Studierenden bzw. Hochschulabsolventen – Analyse zum Forschungsstand (Arbeitspapiere Lehrstuhl für Wirtschaftspädagogik Ausgabe Nr. 56) [Measurement of academically imparted skills and knowledge of students and graduates – State of research analysis (Working Papers, Chair of Business Education, Issue No. 56).]. Johannes Gutenberg-Universität.

40.

Zlatkin-Troitschanskaia

Pant

H. A.

Lautenbach

Molerov

Toepper

Brückner

(2017). Modeling and measuring competencies in higher education: Approaches to challenges in higher education policy and practice. Springer VS.

41.

Zlatkin-Troitschanskaia

Pant

H. A.

Toepper

Lautenbach

(Eds.). (2020). Student learning in german higher education: Innovative modelling and measurement approaches and research results. Springer.

42.

Zlatkin-Troitschanskaia

Pant

H. A.

Nagel

M.-T.

Molerov

Lautenbach

Toepper

(2020). KoKoHs assessment-portfolio – Testverfahren zur Modellierung und Messung generischer und domänenspezifischer Kompetenzen bei Studierenden und Hochschulabsolventen. pfalzdruck.

43.

Zlatkin-Troitschanskaia

Shavelson

R. J.

(2019). Performance assessment of student learning in higher education (special issue). British Journal of Educational Psychology, 89(3), 413–563.

Advances and perspectives of competence research in higher education – Report on the German KoKoHs program

Abstract

Keywords

Introduction and background

Literature review on the state of research in 2010

Political and practical developments and challenges in higher education

Modeling and measuring competencies in higher education: The KoKoHs program

Preparation phase of the research program

Development of competence models and test instruments (1st research phase)

Validation and methodological innovations (2nd research phase)

Key outcomes of the KoKoHs program

Frontiers and perspectives in competence research and praxis in higher education

Footnotes

Declaration of conflicting interests

Funding

References