Abstract
Competency mapping in higher education is not a new technique for curriculum development. Competency-based curriculum in health care administration programs has appeared in the literature every few years. Texas State University has taught a bachelor’s in health administration (BHA) and a master’s in health administration (MHA) for decades and recently wanted to validate its curriculum. Professors began with one course, the management of health information technology (HIT). We had two objectives with this work: What are the current health care information technology competencies required of health care administrators? Is the quality and depth of the current HIT curriculum sufficient? A competency-mapping exercise was used for the first question, and a survey mechanism was used for the second. Three demographic, 10 Likert-type-scale statements, and one free-text question were presented through an online survey instrument to stakeholders (alumni and preceptors). Stakeholders were asked about the importance, quality, and capabilities enabled by the HIT course. Nonparametric tests were used for analysis. The competency-mapping exercise highlighted clear relationships with learning objectives and identified two shortcomings in the current syllabus for both BHA and a MHA programs. Sixty-nine alumni and 20 preceptors provided valid responses to the surveys. Stakeholders generally agreed with the content of the current courses, but asked for updated material and more exposure to live electronic health records. Our methods highlighted shortfalls in our curriculum for one of our courses. We will next create surveys for the rest of the curriculum, engaging with our stakeholders in a continuous quality-improvement manner.
Keywords
Introduction
The study of the management of health information technology (HIT) is logical to integrate into curriculum of health care administration degrees in the United States because HIT appears at most echelons of care along the health care continuum. In addition, the competencies from the Health Information Management Systems Society (HIMSS) are integrated into the competency model of the Health Leadership Alliance (HLA). HIT is defined as “the application of information processing involving both computer hardware and software that deals with the storage, retrieval, sharing, and use of healthcare information, data, and knowledge for communication and decision making”(Thompson & Brailer, 2004). It follows, then that focused degree programs might consider HIT as a course on its own integrated into the curriculum. A brief search of the database from the Commission on Accreditation of Healthcare Management Education (CAHME) yields a result of 97 graduate programs that are accredited by CAHME, (n.d.). A similar search of undergraduate programs certified by the Association of University Programs in Health Administration (AUPHA, n.d.) yields 38 programs. This article was written to suggest curriculum validation for an HIT course as a best practice.
With the ongoing implementation of the Affordable Care Act (ACA, 2010), the Health Information Technology for Economic and Clinical Health (HITECH; Stark, 2010), and the next American Healthcare Act, the health care industry is strongly encouraged (via governmental incentives and penalties from the Center for Medicare and Medicaid Services [CMS]) to implement and meaningfully use electronic medical records (EMRs). In addition, the role of the health care administrative professional continues to adapt to the industry’s information technology changes, which involves an extensive role in the management of such tools to improve population health, manage accountable care organizations (ACOs), and survive in a pay-for-performance (P4P) reimbursement environment (E. Weaver, personal communication, September 2, 2015).
Ongoing research continues to occur at various health care administration programs in an attempt to align industry stakeholder expectations of program graduates with program curriculum in various areas of health care administration (Caron, Hooker, & Ulrich-Schad, 2013; Clement et al., 2010; Mangelsdorff, 2014; Rissi & Gelmon, 2014).
In today’s era of health care reform, there has certainly been no lack of health care administration competency model analysis and related research at both the graduate and undergraduate levels (Clement et al., 2010; Lieneck, 2011; Yarbrough Landry, Stowe, & Haefner, 2012). As the health care administrator’s role continues to require individual competencies, which include various information technology skills and abilities, a thorough investigation into the necessary competencies is necessary for each MHA program. This investigation should gather input from its stakeholders on how to best develop and evaluate future health care administration leadership professionals. Knowledge of such stakeholder requirements will allow for better analysis and understanding of the effects of the dynamic health care landscape and related influences upon the health care administration discipline in general, and it will introduce the voice of the customer into an ongoing cycle of curriculum design. Moreover, such practical stakeholder analysis will further solidify and validate the current HIT course within the health care administration curriculum and properly distinguish it from HIM-focused curriculum.
HIT Course Design (Developing the Model: Integration of Competencies)
The HIT course in both bachelor’s in health administration (BHA) and master’s in health administration (MHA) programs in our school are designed around AUPHA and CAHME competencies, respectively. In line with best practices (Ball, Anderson-Butcher, Mellin, Green, 2010), our school performs a biannual crosswalk of all AUPHA and CAHME competencies based on courses offered to ensure all competencies are addressed. This is a practice that ensures all competencies upon which the program claims it is based, are integrated into specific courses, taught through deliberately worded learning objectives. In this most recent round, we added validity to our course design, and using the HLA Competency Directory as our model, added the HIMSS competencies to our crosswalk that identifies concepts to teach in the HIT-specific courses (see Online Appendix 1).
Following the guidance of capabilities-driven curriculum design, we chose to add HIT as a separate course because the skills taught in that course were difficult to integrate into other courses. A separate course was necessary. In capabilities-driven curriculum design, we ask several questions prior to the design of the course: What tasks should the student be capable of performing by the end of the course? What learning experiences would enable the student to perform that task (and retain that tacit skill)? What learning environment will best enable that learning experience? What is the role of the professor in this learning environment, and what background of that professor best suits teaching these tasks? Capabilities-driven curriculum design focuses on capabilities as learning goals.
Following standard practice in course design (Posner & Rudnitsky, 1994), alignment (Glatthorn, 1999) and mapping (Jacobs, 2004), course objectives for the HIT course were created, and from these course-level objectives deliverables were created (see Online Appendix 2). Objectives and assessments identify levels of and test, respectively, learning using Bloom’s Taxonomy. We felt confident in our assessment, but we felt hearing from stakeholders like alumni and those who hire our alumni would greatly validate our current design and strengthen our program.
Collecting the voice of the customer is an established practice across many industries in quality management. Japanese firms integrated the voice of the customer into a quality function development (QFD) in areas of research and development, engineering, and manufacturing (Griffin & Hauser, 1993). The idea is to identify, structure, and prioritize customers’ needs into the quality-improvement mechanism at the firm.
Collecting the voice of the customer (stakeholder) helps to establish a culture of agility and responsiveness within the School of Health Administration, while also supporting the industry’s initiative for patient-centered care (Epstein & Street, 2011). The separate HIT course has been a part of our curriculum in both undergraduate and graduate degree programs for more than a decade, but its validity had not been tested with stakeholder input.
Objective
The purpose of this study was to gather stakeholder (alumni and preceptors) input on their perceptions of the value of the HIT-specific curriculum from our programs and whether or not the depth of the courses was sufficient.
The intent is for a survey to be developed and administered to our school’s stakeholders annually. This practice will enhance our existing continuous-improvement model of the health care administration information management courses (at the BHA and MHA levels) to ensure it remains aligned with the Texas State University School of Health Administration’s competency model (SOHA Strategic Plan, 2015). This is a pedagogical approach to curriculum development through the voice of health care stakeholders (health care professionals, consumers, and related constituents).
Method
To identify the health care information technology competencies required of health care administrators, the authors turned to competency mapping (Johnston & Webber, 2003; McClelland, 1973; Stefl, 2008). The HLA in the United States provides excellent guidance on the competencies that young health care professionals need to bring to the workplace. We coupled this information with the Competency Tool (2017) from the American College of Healthcare Executives (ACHE), which takes the competencies outlined by the HLA and creates a self-assessment tool that has the potential to be used to highlight areas for improvement. We placed these competencies into a spreadsheet and added body-of-knowledge areas from HIMSS to help us zero-in on the HIT-specific foci that should exist in the HIT courses. As part of the certification and accreditation processes, we had already mapped some competencies into each course. This crosswalk served as an update to validate existing curriculum and highlight potential shortfalls that could result in lesser-prepared graduates from our programs. We then turned our attention to the voice of the customer.
A survey instrument was created to collect input from stakeholders on their perception of value received from our programs. The structured survey instrument for alumni and preceptors from/for our BHA/MHA programs was created to collect data from the answers to 10 questions using a Likert-type scale (ordinal data) and one unstructured, free-text field (to be consolidated and processed as qualitative data). Three questions of demographics (program, sex, age) started the survey for the purposes of analysis. A copy of the consent form and surveys are located in Online Appendix 3. Nine of the 10 structured questions were asked positively (expecting a positive answer) and one was asked negatively (expecting a negative answer). This was done to increase the validity of the instrument (Fowler, 1995). We designed the questions from the course learning objectives, which stemmed from the competencies and domains of the certification/accreditation bodies. This survey was submitted to our local institutional review board (IRB) for review. The probability of harm or injury to research subjects was very small. Any risks to the subjects were minimized by the anonymous nature of the survey. Because there was no potential benefit to subjects based on their responses, risks were greatly minimized. The IRB approved our proposal under an exempt status (IRB application reference number 2015N7481, approval granted on March 21, 2016).
The survey was distributed through the Texas State University’s Snap Survey server to members of the Texas State University School of Health Administration LinkedIn organizational site for alumni and through our private distribution list of preceptors who have mentored the alumni of our program. The members of the LinkedIn account are active students, alumni, and administrative residency preceptors, and employers of the graduates of our school (Festa, 2015). Surveyees were provided links to our website that contained the current curriculum, syllabi, and curriculum map.
Data Analysis
To enable analysis, responses were coded; strongly disagree (SD) = 1, disagree (D) = 2, neutral (N_) = 3, agree (A) = 4, strongly agree (SA) = 5. An exception to this coding was made on the last question; because it was deliberately coded negatively, we expected disagreement, so the coding was different: SD = 5, D = 4, N_ = 3, A = 2, SA = 1. Demographic detail also provides ways to compare within and between groups, if sufficient data are gathered. These data were coded for analysis: our MHA = 1, our BHA = 2; male = 1, female = 2; age stratifications were coded with an increasing number for each stratification of age. A nonparametric method, Mann–Whitney U
We approached Likert-type-scale responses as ordinal data because we recognize that there is no equal (or measureable) distance between SD and D or A and SA (Jamieson, 2004). The median was used as a measure of central tendency, and results were described through frequencies or proportions of response, according to established practice (Clegg, 1982; Jamieson, 2004). Summary statistics were used to test for association between two categorical variables. Pearson’s correlation coefficient was used to identify correlation between the responses, which is also established practice (Norman, 2010).
Results
Competency Crosswalk
Our competency crosswalk is illustrated in Online Appendix 1. It shows the relationship between competency models from CAHME and HIMSS. From this crosswalk, we added the learning objectives for our BHA/MHA courses, as seen in Online Appendix 2. From this exercise, we were able to identify two shortfalls in our current curriculum: under the domain of leadership, it was noted that there were not a sufficient number of questions in examinations and there was not a grading rubric for it in the large research paper; under the domain of communications and relationship management, it was noted that no formal instruction was taking place in presentation and communication-specific skills. The last observation was common for most courses in our curriculum.
Survey Perceptions of Student Learning, Alumni
Eighty surveys were distributed to the alumni listed in our LinkedIn list of email addresses. From these 80, only 69 reported that they were above the age of 18 years. Of the 69 qualified surveys, 63 (91%) completed the survey. The average time to take the survey was below 2.5 min. Blank answers were left blank; no data substitution method was used. Of the respondents, 30 of the 69 were male (43%), 36 of the 69 were female (52%), and 3 of the 69 (4%) did not answer the question. Surveys were completed between 7/13/2016 and 12/26/2016, and all were completed in the same day that they were started. The age stratification was as follows: 18-25 years (11 of 69, 16%); 26-35 years (30 of 69, 43%); 36-45 years (12 of 69, 17%); 46-55 years (10 of 69, 14%); and above 55 years (6 of 69, 9%). All respondents listed their age. Twenty-seven of the 69 (39%) graduated from the MHA program, 41 of the 69 (59%) graduated from the BHA program, and 1 of the 69 (1%) did not choose a degree program (Sullivan & Artino, 2013).
A summary of the questions and aggregate answers are listed in Table 1 along with the box and whiskers chart provided by SPSS; a descriptive-statistical output is listed in Online Appendix 5, and summary statistics are listed in Online Appendix 6. The skew of histograms for the first nine statements was to the left, showing general agreement. The skew of the tenth statement showed general disagreement (right skew), which follows with the expected response based on the negatively worded statement. Responses for all questions did not differ between the alumni for the program or gender. In the age strata, there was only one statistically significant stratum, and only for the first statement: The responses to this statement for the age stratification 18-25 for alumni (
Crosswalk of Competency Models.
Summary of Perceptions of HIT Proficiency From Alumni
Table 2 details the results from alumni. The free-text box offered many helpful suggestions. Some had graduated prior to the HIT course being added to the curriculum, but most comments positively asserted that a focused HIT course is absolutely essential to today’s curriculum. Some graduates from the last 5 years noted that the textbook and curriculum in general should be updated. A summary of the comments is listed in Online Appendix 4.
Survey Perceptions of Student Learning, Preceptors
Table 3 details the results from the preceptors. Twenty-one surveys were received from our preceptor email list maintained by our school. All 21 respondents were above 18 years and completed the survey. The average time to take the survey was below 2.25 min. Surveys were submitted between 6/1/2016 and 9/14/2016, and all were completed on the same day that they started. Of the 21 respondents, six were male (29%) and 15 were female (71%). Blank answers were left blank; no data substitution method was used. One record was discarded because in the comments the preceptor stated that their facility had never supervised a resident from our program so her data were not part of the population. This reduced the accepted sample to 20. The age stratification was as follows: 18-25 (0); 26-35 (5 of 20, 25%); 36-45 (3 of 20, 15%); 46-55 (10 of 20, 50%); and above 55 (2 of 20, 10%). All respondents identified their age bracket. Five (25%) were preceptors for MHA students and 15 (75%) were preceptors for BHA students. The small sample makes statistically calculated associations highly prone to a Type II error.
Reverse coded.
A summary of the questions and aggregate answers for the preceptors are listed in Table 2 along with the corresponding box and whiskers chart from SPSS; a descriptive-statistical output is listed in Online Appendix 4. The skew of histograms for the first nine statements was negative, showing general agreement. The skew of the tenth statement showed general disagreement, which follows with the expected response based on the negatively worded statement. A Mann–Whitney test was performed on programs and ages to identify any outliers. Responses for all questions did not differ between the preceptors for the MHA and BHA programs. Responses for male preceptors did not differ significantly from responses from the female preceptors. A Kruskal–Wallis test was performed on the age strata to identify outliers, with no statistically significant results. Bivariate correlation was conducted to ensure all responses were related to one another and Kendall’s tau was calculated. The correlation between statements was significant (
Summary of Responses From Preceptors
The free-text box offered additional information. A summary of the comments is listed in Online Appendix 4. Preceptors reinforced the necessity for a HIT-specific course.
Discussion
The competency crosswalk provided valuable information that enabled rapid action. This crosswalk was completed prior to the start of the spring semester of 2017, so the professor was able to make changes in each course to fill those shortfalls. Additional questions on leadership were added to each course’s exams, and formal instruction was added to all presentations (visual appeal, readability, etc.). Feedback that enables change is invaluable (Elliott, 2008).
The analysis of responses to our survey statements demonstrated general agreement with our HIT curriculum, in practice, but a few (older) alumni and preceptors were not aware of any specifics in our HIT courses. This highlights a need to publish our curriculum, or at least portions of it, in a location that is convenient for our stakeholders to view and assess.
Of curious note is the statistically significant difference between the recently graduated alumni in relation to the importance of the HIT course in the curriculum. The mean of their responses was neutral, while all other age strata highly regarded the instruction. One comment from a recent graduate was the age of teaching materials. As a result of this finding, we reenergized our textbook review process for both programs, contacted publishers about updates to existing texts, and searched for newly published texts that could serve as a suitable replacement, while the primary texts were being updated by the authors.
Another observation from this exercise is the need to reach out to our stakeholders more often. This first survey took a little longer than we had planned to get through the IRB process, code into the SNAP survey tool, and then collect and analyze the data. Now that we have been through the process once, we plan to develop several new surveys based on other core courses in our curriculum. A new IRB application will begin with all of these survey instruments, and surveys will be distributed more rapidly than before.
The primary weakness of this study is the lack of depth to the questions used in the survey. The response rate was extraordinarily high (86%) and the time spent answering the survey was short. Future versions of surveys to validate our curriculum with the voice of the customer will include questions that are richer in depth and detail.
Comparison to Previous Work
Some researchers observe their survey is far too long (Clement et al., 2010). Our observation was that the one we designed, while elegantly simple, could have yielded richer results had we made it longer or more complicated. The median time it took alumni and preceptors to take the survey was 2.43 and 2.17 min, respectively. A 10-min survey would be quite reasonable to expect a good return. Letting the stakeholders know the importance of the survey might also improve the response rate.
Our study builds on the work of others by focusing on one specific course to make specific changes to a course for both short-term and long-term benefit (Clement et al., 2010). Instead of asking if the HIT course was needed, we asked if the depth and breadth of the course was sufficient, if it prepared or empowered our stakeholders any more than they would have been without it, and if it enabled strategic-level participation of our alumni. We did not intend to imply that the HIT course was solely responsible, nor did we intend to imply that the survey should only apply to alumni who seek to work in HIT. The HLA already establishes its importance. We just need to know how well we are doing to prepare our graduates for management-level demands that may require some knowledge of the processes, vernacular, and importance of HIT operations.
We hope someday to have data as rich as that of experts in the field (Mangelsdorff, 2014). Our study is the first step to collect data on the voice of the customer for our School of Health Administration. Ideally, we would build our data over time to make decisions that are broader based for our program. The ability to assess across all courses, all competencies, in concert with a high percentage of our stakeholders would greatly enable the effectiveness of our program.
Conclusion
Through competency mapping and short survey to gather the voice of the customer, our program was able to highlight, relatively quickly and accurately, shortfalls in our curriculum for one of our courses. Our next task is to create surveys for the rest of the curriculum, engaging with our stakeholders in a continuous quality-improvement manner.
Supplemental Material
Appendix_1_ – Supplemental material for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer
Supplemental material, Appendix_1_ for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer by Clemens Scott Kruse and Cristian Lieneck in SAGE Open
Supplemental Material
Appendix_2_ – Supplemental material for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer
Supplemental material, Appendix_2_ for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer by Clemens Scott Kruse and Cristian Lieneck in SAGE Open
Supplemental Material
Appendix_3_ – Supplemental material for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer
Supplemental material, Appendix_3_ for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer by Clemens Scott Kruse and Cristian Lieneck in SAGE Open
Supplemental Material
Appendix_4_ – Supplemental material for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer
Supplemental material, Appendix_4_ for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer by Clemens Scott Kruse and Cristian Lieneck in SAGE Open
Supplemental Material
Appendix_5_ – Supplemental material for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer
Supplemental material, Appendix_5_ for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer by Clemens Scott Kruse and Cristian Lieneck in SAGE Open
Supplemental Material
Appendix_6 – Supplemental material for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer
Supplemental material, Appendix_6 for Validating a Competency-Based HIT Curriculum in a BHA and MHA Program Through the Voice of the Customer by Clemens Scott Kruse and Cristian Lieneck in SAGE Open
Footnotes
Authors’ Note
This article was prepared as a contribution to the pedagogy of curriculum development. This research did not involve humans as subjects.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
