Abstract
Process performance management (PPM) has become one of the most important management tools in profit organizations. However, non-profit organizations also started to benefit from PPM aimed at the efficiency improvement. The goal of the paper is to investigate usefulness of embedding the simulation modelling approach for process performance management based on the case study of collaboration improvement in higher education. The case study methodology has been used in the study and the paper presents simulation modelling for PPM with the purpose of collaboration improvement at the University of Zagreb, Croatia.
1. Introduction
The modern literature on performance management has progressed from providing general recommendations on improving performance to implementing and using performance measurement systems to manage organizational performance [27], [8]. Nowadays, firms invest large amounts of resources (human and financial) into deploying performance management based on performance measurement systems [12].
Within the last decade, performance measurement has been a very popular research topic, but until recently the focus was on the profit organizations. Since nowadays process performance measurement systems are widely implemented in business practice the research focus shifts toward performance measurement within public organizations. Managing and measuring performance in public sector organizations is a growing phenomenon worldwide [33], [4], [14].
For two decades, Higher Education Institutions (HEIs) worldwide have been under increasing pressure to become more efficient in providing their services [32],[9], [1]. It can be stated that internal and external pressures push HEIs to renew and reshape their organizational structures and management practices.
According to Lam et al. [22] the degree of performance excellence that an organization can achieve greatly depends on the efficiency of business processes. Therefore these authors suggest quantitative methodologies to be used for supporting the business process improvement. Business process improvement efforts involve changes in people, processes and technology over time. As these changes happen over time, simulation appears to be a suitable process modelling method.
The goal of the paper is to investigate the use of simulation modelling as a tool for PPM. For that purpose and using the University of Zagreb, Croatia, as an example simulation modelling was applied to process improvement in a collaboration procedure.
2. Literature Review
2.1 Process performance management
According to Neely, Adams and Kennerley [24] a performance measurement and management system is a balanced and dynamic system that enables support of the decision-making process by gathering, elaborating and analyzing information. It uses different measures and perspectives in order to give a holistic view of the organization. Kueng [21] defines a performance measurement system as an information system which: (1) gathers relevant performance data through a set of indicators; (2) compares the current values against historical or planned values, and (3) disseminates the results to the process actors and managers. Many firms have developed a wide variety of performance indicators which they review periodically while some have very complex and sophisticated performance measurement systems that allow them to track what is happening in real time. According to Harmon [17], most companies still experiment with the specification of process-based performance measures. They rarely have their measures aligned with their strategic goals.
2.2. Characteristics of performance measurement in the public sector
A literature review on performance measurement and management systems implementation in the public sector highlights the factors driving performance in public organizations. Adcroft and Willis [2] described two examples of performance measurement systems in the UK National Health Service and the higher education sector and showed that the most likely outcomes of these systems are further commodification of services and deprofessionalisation of public sector workers. The characteristics of process performance measurement systems in justice organizations in Finland are analyzed and several specific critical success factors are found: (1) understanding the causal relationships; (2) improving the informativeness of used measures; (3) emphasizing the role of the measures as communication devices and (4) using the measures as incentives for improvement [29]. Goh [14] emphasizes three important factors that need to be taken into account in the effective implementation of the performance measurement system in the public sector. These are managerial discretion, a learning and evaluative organizational culture and stakeholder involvement. Besides, it is not reasonable to use the same performance measurement system within the public sector both for external reporting and for internal administrative development [16]. External stakeholders are more interested in high-quality services than in information on the internal processes efficiency. Presented evidence from business practice shows that what works in private (profit) companies does not automatically work in the public sector.
2.3. Process performance measurement in higher education institutions
A study conducted by Educause [13] shows that HEIs have invested heavily in business process change and redesign projects. These projects were driven mainly by budget shortages, information technology implementation and external requirements for improved efficiency and effectiveness [10]. Since expenditure on administration of HEIs is typically about 30% of expenditure allocated to academic activities, Casu and Thanassoulis [9] set up a data envelopment analysis (DEA) framework to identify good management practices leading to efficient administrative services in UK universities. This study demonstrated the problems in defining the unit of assessment and the relationship between inputs and outputs.
To promote HEIs operating performance, performance measurement indicators (PMIs) are needed. According to the literature overview, different methods and tools are used to measure performance in the education sector. Chen et al.[11] analyzed the literature and employed the established PMIs to identify important key performance indicators (KPIs). As a result of this study, 78 PMIs were developed and were categorized in 18 measurement dimensions. The authors recommend that universities use these indicators to measure their operating performance. Stoklasa et al. [34] have suggested a new model for the academic staff performance evaluation, which is based on fuzzy-rulebase systems. The model evaluates staff members' performance in the area of Pedagogical Activities and in the area of Research and Development and is currently being implemented at Palacky University (Czech Republic).
Paralič et al. [25] proposed the process-driven semantic approach which made possible not only to model, but also to support operationalization of selected parts of generic educational processes in the form of electronic services. During the first phase of this research, the authors developed more than 50 first and second level process models. In the operationalization phase the formalized form of processes (BPMN 2.0 models) could be straightforwardly transformed into executable processes (e.g. into the process-driven service-oriented architecture – SOA – based system). Once educational processes are supported by information and communication technologies, it becomes possible to track, monitor, measure and analyze these processes. According to the authors, there is no doubt that process models are very useful and powerful means for knowledge capture, analysis and improvement of the existing processes in business, but also in the public and the higher education sector.
2.4. Role of Simulation modelling in process performance measurement
Simulation has an important role in modelling and analyzing the processes in introducing business process changes since these changes can be explicitly quantified and the effects of changes can be measured. It enables quantitative estimations on the influence of the redesigned process on system performances [6]. Business processes simulation creates an added value in understanding, analyzing, and designing processes by introducing dynamic aspects [7]. It enables migration from a static towards a dynamic process model [3]. Nowadays most business process modelling tools include simulation capabilities, but in addition, there are some tools that are designed specifically for more demanding simulation projects [23].
2.5. Advantages of simulation modelling in PPM
Many authors examined and described the development and implementation of simulation models in order to analyze the existing business processes and to predict the performance of new designs. Some cases come from public sector organizations and only several are listed here to argue the advantages of simulation modelling in business process change initiatives. Greasley [15] describes the use of a process-based approach to change in relation to the implementation of an information system in the UK police force. This case study demonstrates the use of a simulation model to change the process and to prove the changes. The author also argues that by running the simulation through time it is also possible to gauge how changes at an operational level can lead to meeting strategic goals over time. Hays and Bebbington [18] illustrate the development and application of a simulation model to aid decision making in relation to the procedures followed in the Office of Film and Literature Classification.
Peček and Kovačič [28] describe a case study of filling unoccupied capacities in an old people's home. This research shows that by using simulation it is possible to predict the effects of changes and the duration of the processes and bottlenecks and to thereby avoid bad decisions.
Simulation has been used very often in healthcare management. Bertolini et al. [5] developed a case study on a surgical ward. A framework based on Event-driven Process Chain (EPC) diagrams, the entity-relationship model and the discrete event simulation are developed while a what-if analysis was conducted in order to assess various scenarios performance.
2.6. Limitations of simulation modelling in PPM
However, limitations of simulation modelling in business process change projects are also discussed. According to Greasley [15] business process simulation has the ability to predict behavior over time, but requires careful planning in order to ensure it is able to deliver results within cost and time targets. Besides, the results of the simulation analysis must be examined from the perspective of internal and external stakeholders of a public sector organization. The problem of data collection must be considered since simulation modelling requires the quantification of aspects that may be difficult to measure, especially in the public sector environment [19], [15]. Popovič and Jaklič [30] identified some other issues which could be a big obstacle to using simulation modelling in the public sector, such as: problem definition issues, socio-political issues and multi-perspective issues. According to Jarvis [20] understanding the organizational context and problems of an organization is necessary for a successful implementation of the business process redesign, but public sector organizations (e.g. universities) are bogged down by internal politics, bureaucratic procedures, lack of competencies and traditionalism. Semanco and Marton [31] stress that simulation's output performance measures can be very tricky because these depend on the calculations of particular simulation software. Besides, a clear vision and top management support are critical to the overall implementation and success of new designs [26]. Unfortunately, the resistance to change characterizes public sector organizations.
3. Methodology of Research
According to the suggested methodology, the main steps of embedding the simulation modelling approach for reengineering collaboration in higher education are the following:
1st Step: Initiation - Create the “As Is” process model including parameters for simulation.
2nd Step: Analysis - Run simulation based on the “As Is” process model and analyze simulation results. Compare simulation results with real data. If no significant statistical errors were measured, then deduct that the process model is good enough, i.e. simulation results correspond to the real measured KPI values.
3rd Step: Re-engineering - Suggest changes in order to improve/reengineer the process. Include parameters for simulation of the “To Be” process variants.
4th Step: Implementation - Run simulations based on the “To Be” process models. Analyze and compare different variants of the “To Be” models based on the simulation results.
5th Step: Evaluation – evaluate efforts, benefits or costs/benefits. Make decisions based on evaluation results.
4. Results
Process models in the following example are modelled as sequential models whereby activities are repeated one after the other depending on inputs to activities, outputs of activities, probabilistic rules, and available resources. Our process models consist of flow objects such as activities and gateways, data objects, connecting objects, swim lanes and artefacts. Each activity is described by several parameters: name, duration, resources needed to conduct an activity, availability of the resources assigned to activities, time a resource requires to conduct an activity, inputs of predecessors. Our case study is described in detail in the following sections.
4.1. First Step: Initiation
An illustration of embedding the simulation modelling approach for reengineering collaboration in higher education is given for the process of nomination and selection of final thesis themes for undergraduate and graduate students. The process is modelled with IBM Websphere Business Modeler version 7.0.0.4 and in accordance with BPMN 2.0. Important properties of the process are described in Tables 1 and 2. The process model shown in Figure 1 is used to perform the process simulation.
Description of the As Is process activities
Description of the As Is process conditions
The As Is process model of nomination and selection of final thesis themes for undergraduate and graduate students
4.2. Second Step: Analysis
Parameters that were used in the simulation of the process have been set according to real values gathered by a brief assessment based on questioning of students and authors' real experience. For simulation purpose in this case study the following simulation parameters were used: (i) Number of simulation instances: the simulation was run over 40 instances representing 40 student inquiries about themes; (ii) Frequency of instances: 50% of instances appear every 0.5 days, 20% of instances appear every day, in 15% of all cases the instance appearance is every 0.75 days, and in 5% of cases the instances are triggered every 2 days.
The simulation of the As Is process showed the average duration of 40 days 7 hours 14 minutes with the duration standard deviation of 8 days 7 hours 36 minutes. When the simulation results are compared with real data, an acceptable statistical error is under 5%, i.e. due to the lack of comparable real data the statistical significance is assessed based on real experience.
The analysis shows that the duration of the process is a feasible improvement opportunity. Other functional requirements of end-users (i.e. teachers or lecturers) imply the need of following early stages of theme suggestion and formalizing, as well as the need of tracking all communication of students and mentors. These changes are implemented in the To Be process model discussed in the next section.
4.3. Third Step: Re-engineering
Based on the analysis of the As Is process that shows that the duration of the process is a possible improvement opportunity and which identified other functional requirements of end-users the new reengineered process model (i.e. the To Be process) was developed. Its description and model are shown in Tables 3 and 4 and in Figure 2.
Description of the To Be process activities
Description of the To Be process conditions
The To Be process model of nomination and selection of final thesis themes for undergraduate and graduate students
4.4. Fourth Step: Implementation
Like in real implementations, a grace period for total transition must be well planned and the “new way of doing business” should be introduced progressively. Simulation results listed in Table 5 show the gradual improvement of the discussed process.
Simulation results for progressive introduction of the new process model
The gradual implementation consists of a progressive transition from the As Is process of handling the themes as shown in Figure 1 which implies initial introduction of the new process variant (shown by Figure 2) in 20% of all instances (i.e. 20% of 40 instances), then in the next stage in 40% of all instances (i.e. 40% of 40 instance), and finally the total transition into the To Be process variant. Simulation parameters for the simulation of the To Be process are the same as in the simulation of the As Is process.
4.5. Fifth Step: Evaluation
Simulation results for a progressive introduction of the new process model shown in Table 5 need to be evaluated when a decision on accepting the To Be process is made. The key performance indicators relevant for this case study may be the process/activity duration and the duration standard deviation. As it has been already stated earlier the duration standard deviation can be significant in analyzing, assessing or predicting the stability of the process. Hereby two conclusions regarding the duration and its standard deviation are presented: (1) if the ratio of the duration and its standard deviation is small, predictions for resources allocation are more precise and (2) if the ratio is tending towards 1 then the organization is close to guessing when allocating resources to activities and the possible threat of wasting resources is greater. However, data in Table 5 show that the two conclusions are not necessary generally applicable.
5. Conclusions
The overall conclusion of this case study should illustrate the significance of pondering KPIs and the need of conducting a detailed analysis of relevant KPIs corroborated by objective data. Implementation of a new scenario of the discussed process of nomination and selection of final thesis themes for undergraduate and graduate students has a relatively short duration and thereby reduces overall average duration of the process. At the same time the relatively short duration of this process influences the duration standard deviation in such a way that it enhances the duration standard deviation. This is natural because there are more process scenarios with durations which fall within the greater range. In our discussed case the reduction of the average duration has a greater significance over enhancement of the duration standard deviation as this was stated during the analysis stage as an important feasible improvement opportunity in combination with other functional requirements of end-users which lead to changes in the To Be process model. According to our results, simulation modelling has been proved as a valuable method in PPM in HEIs.