Abstract
Recent research by KU Leuven showed that 33% of the engineering graduates in Flanders changed jobs in the first year, with 60% of those citing job content as a reason. Also, industry often reports that recent graduate hires lack the right skills for the job. It appears that students seem to enter the labour market less prepared both in perception and skill level. This study investigates the perceptions of first-year students on their future role and the competencies they need by developing an engineering role model on the business model of Tracey and Wiersema. The premise of the PREFER-model is that most vacancies for junior engineers fall into one of three roles: Product Leadership (i.e., focus on radical innovation), Operational Excellence (i.e., focus on process optimization), and Customer Intimacy (i.e., focus on client-tailored solutions). A survey was administered to first-year students from the three largest engineering degrees in Belgium, Ireland, and the Netherlands. A total of 197 students in Belgium (KU Leuven – Engineering Technology), 89 students in Ireland (TU Dublin – Engineering), and 372 students in the Netherlands (TU Delft – Aerospace Engineering) participated. In this survey, students were also asked to express their preference for three fictional job vacancies reflecting the three different roles. The results showed that first-year students do not have a clear view of the future and have an idealized perception of the engineering profession centred around the Product Leadership role. Students were also found to overestimate their level of preparedness when it comes to their mastery of competencies. It is suggested that having a discerning professional roles model as well as instruments that allow students to assess their role alignment and associated role competencies will help mitigate these issues.
Introduction
In every engineering curriculum, preparing students for their engineering profession is an established learning outcome. Many universities do so in the shape of internships, company visits, and guest lectures. Over the last 20 years, it has become clear that triggering engineering students to reflect on their professional future is equally important and forms a new challenge for engineering institutions. Prior research showed that explicitly articulating student social identity and career goals has beneficial consequences for student learning, 1 motivation,2,3 student retention, 4 and students practicing as engineers. 5
Van den Bogaard, in her PhD-thesis on first-year engineering students' study success, highlighted that students who drop out during their first year have more trouble with the career perspectives of engineering than students who stay. 4 She argues that first-year students should be stimulated to reflect on their identity within the engineering professional world from the start of their educational career. However, Karatas et al. observed that first-year science and engineering students’ beliefs about science and engineering are often flawed and unsophisticated. 6 On the one hand, this is not completely surprising given that the engineering domain is very broad and there are endless career directions. On the other hand, to date, there is no overarching internationally validated framework to let engineering students reflect on their professional future to aid educators in helping students develop their engineering identity.
Next to preventing unnecessary dropout during their studies, it is also important to avoid engineering graduates not taking up an engineering career after they have graduated. Numbers in the Netherlands consistently show that as little as 60% of the engineering graduates take up a career in engineering.7,8 Recent research using a life history approach by Van Hattum-Janssen and Endedijk shows that when entering an engineering degree, a distinction can be made between three types of students: (1) students who have always had an inclination for engineering and who are looking at a career in engineering, (2) students who select engineering as their degree as they are primarily motivated by looking to do something very challenging and generally intend to look to work outside of engineering and (3) students who choose engineering after a careful selection but are ambivalent about whether to also carve a career for themselves in engineering. 5 If educators and industry combined are to motivate more students to select a career in engineering, it is particularly this third group that they should focus on. As reported by van Hattum-Janssen 4 and van den Bogaard, 5 students have not developed a professional identity yet as an engineer and it is of vital importance to aid them in developing a professional engineering identity during their time at university if they are to be retained by the engineering profession.
A third issue is that there is also evidence that as many as 33% of recently graduated engineers change jobs within their first year of working with 60% citing job content as the reason. As was reported in Flanders, 9 this is not only costly for employers but also for the graduates themselves due to possible relocation costs, reimbursement of training costs, loss of a company car, as well as effects on pension, insurance and benefits depending on the local situation. In addition to that, industry European wide is reporting that many engineering graduates do not possess the right non-technical competencies for the position they were hired to fill. 10
In an attempt to create such a professional identity role model, Hofland and colleagues developed a first version of a professional roles framework for engineers based on the value disciplines of Treacy and Wiersema.11,12 The study of De Norre and colleagues describes some attempts to raise awareness for professional roles in the bachelor and master curriculum. 13 Based on the results of these two papers and encouraged by industry, the central focus of the European Erasmus+ Knowledge Alliance PREFER project (Professional Roles and Employability of Future EngineeRs) was defined: the validation of a framework of professional roles for engineers and the implementation of dedicated tests and skills education in engineering curricula to train students for those roles. 14 In this project, the professional roles model is further optimized and validated in close collaboration with industry and the Engineering professional bodies in the three partner countries of this project: Belgium, Ireland, and the Netherlands.15,16 As part of that, based on mixed methods research at KU Leuven, detailed key competencies were identified for the three different roles. 17 The study of the design and implementation of dedicated skills into the curriculum has been performed at TU Delft, and as a result, new curriculum elements have been implemented at several engineering universities across Europe.18,19 In parallel with this, TU Dublin, KU Leuven, and the company BDO have developed two tests to measure the interests and the alignment with the different professional roles and competency profiles.20,21
The prime objective of the present study is to corroborate the research findings observed by De Norre and colleagues and to compare the outcomes of three large representative samples of first-year students in leading engineering institutions in Belgium, Ireland, and The Netherlands. 13 Additionally, we will evaluate the discriminatory power of some general learning outcomes to discriminate between the three professional roles identified by Hofland et al. 11 Part of the results of the surveys in Belgium and the Netherlands has previously been presented at a conference and are now being presented in a larger context within the three countries participating in the project. 22
In this study, we investigated which similarities and differences exist between the three populations on their view of their future and their preferred professional role. We also looked at which competencies students feel they are already most developed at as well as the competencies they feel they needed to develop most, in light of their preferred professional role.
Prefer professional roles framework (PREFER)
The development of an overarching framework to encase student perceptions regarding a complex engineering labour market is of paramount importance. Although conceptual frameworks often are a reduction of a complex reality, they offer very concrete opportunities to grasp particular aspects of this reality that goes beyond the engineering specialisation (e.g., electrical engineer, chemical engineer).
Within engineering education research, several attempts at overarching models have been made. In her PhD-thesis studying aerospace engineering alumni, Saunders-Smits (2008) makes a distinction between the engineering manager and engineering specialist, but other studies go beyond the manager-specialist dichotomy. 7 For example, Gerwel et al. also describe an entrepreneurial role, 23 whereas Kinoshita et al. include consultants. 24 However, the typologies of engineers are mostly set up for research purposes than developing a role model, such as the identification of skills to evaluate the curriculum, the exploration of engineering students' preferences and expectations of the engineering profession, or the alumni perceptions on skill development. Another approach was found in the typology of Endedijk et al., describing five personality types (the nerd, the status seeker, the hipster, the security seeker, and the loner) based on stereotypical characteristics. 25 Their findings suggested that more stereotypical (male) students are still more likely to aim for a career within the technical field and highlighted the need to enhance diversity in engineering education and industry.
Few studies focus on the start of the career and rather focus on career trajectories or engineering education. In a systematic literature review, Craps and colleagues found similarities between the models of Hofland et al., Kamp and Klaassen, and Spinks et al., identifying engineers focusing on innovation, optimisation, and customisation.11,26–28 Kamp and Klaassen described a fourth role, the contextual engineer, emphasising the diversity and different cultures or contexts. However, Spinks et al. suggested incorporating future-oriented aspects due to globalisation and rapid change within all roles. As the framework of Hofland and colleagues was the only model that involved stakeholders from industry and education, it was decided to further develop this model based on the work of Treacy and Wiersema. 12
In strategic business management, Treacy and Wiersema have put forward three different value disciplines: Operational Excellence, Product Leadership, and Customer Intimacy. 12 The main hypothesis of the authors is that companies who manage to focus their strategic vision on one of these value disciplines are more profitable than their competitors. The Treacy and Wiersema model proved to be a valuable framework to look at the variety of engineering functions. Hofland et al. re-engineered the model and tailored it to the engineering profession: Operational Excellence (process optimization & increasing efficiency); Product Leadership (radical innovation & research and development); Customer Intimacy (tailored solutions for individual clients). Using an extensive industry questionnaire, the authors found that 91% of the respondents were able to recognize these different roles in their company. 11
This then brought on the next logical step before developing the role model further within the context of the PREFER project and our main research question: If industry recognizes these roles, would the same apply to students across the three universities, each different in set-up and in different European countries? This led to the formulation of the following sub research questions:
Do first-year engineering students from different countries have a clear view of their professional future? Can first-year students recognize and select a future professional role based on vacancy adverts? Are first-year students able to self-reflect on the competencies they need for their favoured vacancy? When asking students to self-reflect on their mastery of competencies, can any trends be observed? What are the consequences of these findings for engineering curricula in view of professional identity and competency development?
Methodology
Sample
To see if engineering students were able to recognize these roles as a start towards developing a professional engineering identity and developing a relevant set of competencies, we decided to survey first-year students at all three institutions who had only just started their bachelor's degree in engineering. Ethical permission was thought and granted at all three institutions. The extensive paper-based questionnaire, detailed in Pinxten et al., 22 was administered among 197 first-year students at two campuses of the Faculty of Engineering Technology of KU Leuven in mid-2015 (response rate 41% - based on total number of first-year students on all 7 KU Leuven campuses) and 342 first-year students at the Faculty of Aerospace Engineering at TU Delft (response rate 83%) in mid-2017 and 89 first-year students at TU Dublin in mid-2018 (response rate 82%). The students at KU Leuven were in their 8th week of lectures, the students of TU Delft were in their 3rd week of lectures, as were the students of TU Dublin. All participating students were enrolled for the first time at university and generally did not have any industry experience (in all three institutions, internships, company visits, etcetera are incorporated in later stages of the engineering curriculum). The first-year students in Leuven and Dublin enrol in a common engineering programme who do not specialise until later. The TU Delft students are all first-year aerospace engineering students. For all three countries, their respective accreditation bodies have uniform competency requirements for engineering programmes irrespective of the engineering discipline allowing for generalisation of the results.29–31
Measurement of professional roles
A questionnaire was used to gauge students’ perceptions of their professional future. Rather than just present students with the definition of each professional role, we used fictional job vacancies to measure first-year students’ preference for the three different professional roles. Each job vacancy was similar in set-up and consisted of a brief description of the job content and a profile sketch listing required competencies for the advertised position. For the Operational Excellence role, we opted for a team lead in production methods and industrialization (core tasks: analyse production process and implement optimization ideas). For the Product Leadership role, a stereotypical research and development vacancy was defined (core tasks: develop new concepts for industrial innovation & explore new market segments). Finally, the Customer Intimacy role was operationalized by a vacancy of a technical-commercial representative (core tasks: tailored advice to new and existing clients & client portfolio). Students were asked to rank the vacancies in order of preference.
To investigate the required competencies and student recognition of them, we asked students whether they felt they already possessed the right competencies for their most preferred job vacancy, based on their (subjective) perception. Additionally, we presented the students with 11 competencies, namely the 11 official learning outcomes of the Faculty of Engineering Technology of KU Leuven, together with a brief definition of each learning outcome (see Appendix 1 for the learning outcome definitions). Students were then asked to indicate in which of these competencies they considered themselves to be most, second-most, and third most competent, as well as select the top three competencies they felt they still needed to develop.
Data analysis
All questionnaires were coded and put in MS Excel and SPPS™ 26. As the majority of the data concerns categorical data, any relationships were investigated using Pearson's Chi-Square as recommended by Field. 32 All non-descriptive statistical analysis of the data was carried out using SPSS™ 26 and all descriptive analysis in MS Excel.
Results & discussion
First-year students view of their professional future
Regarding students' views about their professional future and with reference to Figure 1, the results indicate that only a small proportion of the first-year students have a clear view of what the future utility of their engineering degrees may be. Approximately 9% of students at KU Leuven and TU Dublin and 12% at TU Delft indicated that they have a clear view of their professional future. In contrast, about 20–30% of students indicated that they do not have a clear vision of their future, with the majority of students, between 58–66%, falling somewhere in the middle with a view of their future that is not entirely clear as of yet. To check if the distribution did not vary between the universities, especially given that the students from TU Delft have already selected a specialisation, as opposed to TU Dublin and KU Leuven, a Pearson’s Chi-square test was carried out. There was no significant variation found between the three universities, (χ2(4) = 9.045, p > 0.001).
First-year students’ view of their professional future.
Students were also given 16 possible non-exclusive domains for which they could indicate if they would be interested in working in, ranging from Construction to Legal and from Research and Development to Human Resources. More than 50% of all students named between three to five areas they would be interested to work in and a further 21% named more than five areas. It was verified using a second Pearson's Chi-square test, (χ2(8) = 3.064, p > 0.001) that there were no significant differences between the number of areas students indicated they would be interested in working in and their view of their professional future.
These findings, although not entirely surprising given that the survey was of first-year students, does present a number of benefits and challenges to engineering institutions. As shown in previous research, students who chose to study engineering are generally unsure of their career trajectories, despite choosing to study engineering. 5 Engineering institutions find themselves in a position of great responsibility in that they can help to shape students’ professional futures. This can be achieved by introducing targeted interventions in the engineering curriculum such as company visits, guest lectures and projects, to name just a few.
Preferred professional role
To encourage students to reflect on their professional future, three fictional job vacancies were developed based on the three professional roles presented in the PREFER role model. Students were asked to prioritise their preferences for the three vacancies. In Figure 2, the findings across the three institutions are presented. As can be seen from the figure, a clear preference for product leadership emerged, with this preference most dominant at TU Delft where the preference was 66% compared to only 12% preferring the customer intimacy role. This may be for a number of reasons, one of which possibly is that TU Delft’s aerospace engineering curriculum is driven by design and the results may represent a conscious awareness of this on the part of the students.
First-year students’ professional role preference at the three universities.
A Pearson’s Chi-Square test confirmed that there are significant differences between the three universities (χ2(4) = 25.651, p < 0.001), with an effect size ϕ = 0.33 which is a medium effect size. 32 A closer look at the corresponding contingency table (Appendix 2) revealed that for TU Delft and KU Leuven significant differences exist between Product Leadership and Operational Excellence versus Customer Intimacy, and for TU Dublin significant differences exist only between the Customer Intimacy and the Product Leadership roles.
At TU Delft and KU Leuven, the Customer Intimacy role (i.e., technical-commercial representative) was the least preferred vacancy, whereas at TU Dublin the Operational Excellence role was the least preferred. There appears to be no clear reason for this difference. These results suggest that at an early stage of their engineering education, the majority of students have idealised conceptions of what engineering is, i.e., that it is about product innovation, and while a proportion of engineers do work in this arena, it is important to illustrate to students that engineering is also about people and processes.
The majority of the students in each university (64%, N = 195 - KU Leuven; 65%, N = 312 -TU Delft; and 68%, N = 82 - TU Dublin) indicated that they would apply for all of the vacancies. These outcomes give third-level institutions scope for championing the other roles which students did not wish to apply for and making students more aware of the choices available to them. A possible aid in this could be the PREFER EXPLORE instrument that allows students to explore which role is the best fit based on their personal preferences. 33 This personal preference test was developed as part of the PREFER project to (1) create awareness of the different professional roles an early career engineer can take on and (2) give insight into the students’ personal preferences.
Self-perceived levels of competency mastery
When asked about their self-perceived mastery of the competencies of their most preferred vacancy, first-year students generally display high confidence levels. Especially at TU Delft and TU Dublin, 48% and 56% of the respective respondents indicate that they already had the required competencies (Figure 3). Only a small proportion of students (5.5%) stated that they do not yet possess the right skills. To see if there were significant differences between the responses of the three universities and their perceived readiness, a Pearson's Chi-square test was carried out. Although the expected count in one cell (see Appendix 2 for contingency table) is below 5, it is still permitted to use a Chi-square test as the table is larger than 2x2 and amounts to less than 20% of all expected counts. 32
Self-perceived mastery levels of listed competencies.
Significant differences between the three universities were found (χ2(4) = 66.287, p < 0.001) with an effect size ϕ = 0.20 which is a small to medium effect size. 32 A closer look at the contingency table results reveals that for KU Leuven and TU Delft, significant differences exist between the three groups of students in terms of their level of preparedness. However, for TU Dublin, significant differences only exist between those students who feel they have partially mastered and fully mastered all of the competencies. This may in part be caused by the low number of responses for the “No”-category.
Regardless of any significant differences between institutions, these results taken as a whole present yet another challenge for engineering educators in dealing with students’ lack of awareness of their incompetence. This phenomenon is well studied and has been coined the Dunning-Kruger effect in which low achieving students often overestimate their ability, while high achieving students often underestimate their ability. 34 If first-year students are already very confident in their own competencies, they may perceive certain forms of learning as superfluous and thus uninteresting, which may, in turn, affect student motivation. This means that the approach to teaching competencies may need to be adjusted to take into account the self-perception of students. It may therefore be good to have students experience whether or not they actually have mastered all of the competencies they need for their preferred job vacancy. This will lead them to make better decisions about their educational choices.
Most competent and least competent competencies
Each student was also asked to indicate the three competencies they considered themselves most competent in, selected from the list of competencies defined in Appendix 1. Based on a weighting scheme (1st competency - 3 points; 2nd competency - 2 points; 3rd competency - 1 point) sum scores were calculated for each of the 11 competencies. These scores were then ranked within the professional role that each student expressed their first preference for. The top five competencies for each professional role are illustrated in Table 1.
Top five competencies students felt they were most competent in grouped by preference for each professional role.
Note: Ranking of competencies based on the sum scores for each competency (most competent – 3; second most competent – 2; third most competent – 1).
At KU Leuven, the competencies students identified themselves as most competent in for both the Operational Excellence and Product Leadership roles were almost identical, except for two competencies: the design competency (Product Leadership) and communication (Operational Excellence). Students with preferences in one of these two roles estimated their mastery of problem-solving and teamwork at a very high level. Interestingly, students with a preference for the Customer Intimacy role considered themselves better at communication than peers who chose either Operational Excellence or Product Leadership.
For TU Delft, no difference was found between the Operational Excellence and the Product Leadership roles based on the answers given. For both roles, the competencies problem-solving, teamwork, design, critical reflection, and professionalism were observed as the most listed. The teamwork competency was more pronounced in the Operational Excellence role. Students with a preference for the Customer Intimacy role considered themselves more competent in entrepreneurship and communication when compared to the other two roles.
If we look at the results for TU Dublin, we again see great similarities in the competencies listed for the Operational Excellence and the Product Leadership role in the top three competencies. The same three competencies are also listed for the Customer Intimacy role, which was not the case at KU Leuven and TU Delft. Also, in Product Leadership and Operational Excellence teamwork is not listed as a competency that they feel they have already developed sufficiently compared to KU Leuven and TU Delft. They list communication as a competency that they feel most competent in.
Comparing the results of all three universities, we noticed that the design competency appeared higher in the list in the Product Leadership role for both TU Dublin and TU Delft than in KU Leuven. In the Operational Excellence list, KU Leuven students do not list design as a competency that they feel they are already competent in, whereas students of TU Delft and TU Dublin do. It is unclear what the cause of this difference in perception of competence in design may be. Most of the competencies listed by the entire sample of first-year students as being competent in, such as problem-solving, teamwork, professionalism, critical reflection, communication, and design show a high degree of consistency. Several competencies were rated significantly fewer times by the students (e.g., application-oriented research, to make operational, and information processing). Perhaps this is due to a lack of awareness of the operational definition of these competencies, even though they were provided with a description of these competencies (see also Appendix 1).
Finally, students were asked which three competencies they felt they needed to develop the most. Once again, a sum score was calculated per role using the same weighting scheme as before (1st competency - 3 points; 2nd competency - 2 points; 3rd competency - 1 point) and ranked within the professional role the student expressed their first preference for, with results given in Table 2.
The top five competencies students feel they still need to develop further grouped per preference for each professional role.
Note: Ranking of competencies based on the sum scores for each competency (to be developed most – 3; to be developed second most – 2; to be developed third most – 1).
Similar to the results presented in Table 1 regarding the competencies students viewed as having mastery in, a distinction between choices in every profile was difficult to discern in most cases. Problem-solving remains the most identified competency still to be developed for the majority of the categories. If we also take into account the outcomes reported in Table 1, students, on the one hand, feel most competent in problem-solving but on the other hand also prioritize this competency when asked which competency they feel they need to develop further. This finding suggests that students view problem-solving as a competence that requires constant attention, but also as a competence that they have mastered, which may be construed as contradictory. This is also the case for at least two more competencies in each role and these contrasting results may require further attention.
The results also represent a desire by students to learn about entrepreneurship and application-oriented research. Although included in the KU Leuven curriculum, entrepreneurship is not a mandatory part of the TU Delft or TU Dublin curriculum. If one values negotiated learning as part of their organisational strategy, this may be something for both institutions to consider integrating into existing curricula. Another interesting finding is that the competency the students felt they need to develop the least was ethical behaviour, which is in stark contrast with current universities', governments', and public opinion. Perhaps this is also indicative of their (in)ability to critically reflect as shown by their contradictory answers to the question of whether they have mastered the competencies required for their preferred future role.
To aid students in developing self-reflection skills and overcome the risk of incorrectly estimating their own abilities, the PREFER MATCH test may be able to contribute towards a solution. The PREFER MATCH test was developed as part of the PREFER project to help address the issue of incorrectly assessing one's competency levels. In this Situational Judgment Test, a comparison is made between the students’ perception and the perception of engineers in the field. 35 Note, however, that ethical behaviour and entrepreneurship are not included in this test.
Another worthwhile observation is that a clear need has been established for students to learn transversal skills (including entrepreneurship). These types of skills are generally not taught in lecture rooms but in more hands-on settings. Hence within the PREFER project hands-on opportunities were developed for students18,19 which are also available as OpenCourseWare through the TUDelft OpenCourseWare website (ocw.tudelft.nl). In addition, as a result of the PREFER project, KU Leuven has transformed their curriculum based on the Professional Roles Framework and has embedded these transversal competencies throughout their degree. 36
Conclusions and recommendations
This study examined the perceptions of first-year engineering students of their professional future at three leading engineering institutions in three different European countries. The findings indicate that first-year students across these universities feel they do not have a clear view of their professional future. As this lack of a clear view is one of the contributing factors in student dropout during the first year, as well as a potential cause for the loss of engineering graduates to the engineering industry, it may be very worthwhile for engineering education institutions to increase the attention spent on the future disciplinary self and professional identity, starting from the moment the students begin their degree and maintaining this during their entire degree.
Using fictional job vacancies, based on a professional role model, it was discovered that although students were able to express their preference for a vacancy associated with the role, they also have idealised conceptions of what engineering is. Only minor differences were found between the three institutions regarding the preference for the Product Leadership role, seemingly the most attractive professional role, which indicates an idealized but incomplete view of the engineering profession. Care therefore must be taken that students are provided with opportunities to develop a more realistic view of the engineering profession.
The self-assessed level of preparedness for student future roles is high, especially at TU Delft and TU Dublin. This may lead to a dangerous form of students who overestimate themselves and therefore deny themselves the acquisition of required competencies. It may be worthwhile to have students reassess their actual competency level against a set standard so that they may verify their perception and adjust their learning strategies accordingly.
Within the PREFER project, two instruments were developed: PREFER MATCH and PREFER EXPLORE. These instruments allow engineering students to explore their preferred role and to self-assess their competency mastery level. This may be the first step to assist students to develop their professional self during their degree from the moment they walk through the (virtual) doors of the university.
Additionally, it may be very wise to also employ a similar instrument that allows students to assess their ethical behaviour skills, as students overwhelmingly do not list ethical behaviour skills in their top three competencies for development.
Overall, the findings against the framework of professional roles again stress the importance of teaching transversal competencies alongside technical knowledge to students, ideally in hands-on practical situations.
Finally, students also indicated their need to develop their entrepreneurial competencies as well as application-oriented research. Application-oriented research skills are part of all three curricula; however, entrepreneurship is, for instance, not mandatory at TU Delft or TU Dublin. The outcomes of the survey may give reason to reconsider this.
Footnotes
Acknowledgements
The authors would like to thank their colleagues in the project, as well as those colleagues who assisted with the rolling out of the questionnaire. The authors in particular want to thank their former colleague Maarten Pinxten for all his hard work and support during the time he was involved with the project, without whom this tri-national study would not have taken place.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Erasmus+ programme of the European Union (grant agreement 575778-EPP-1–2016-1-BE-EPPKA2-KA) and is part of the PREFER project.