Abstract
Objectives:
In 2014, the National Board of Public Health Examiners performed a job task analysis (JTA) to revise the Certified in Public Health (CPH) examination. The objectives of this study were to describe the development, administration, and results of the JTA survey; to present an analysis of the survey results; and to review the implications of this first-ever public health JTA.
Methods:
An advisory committee of public health professionals developed a list of 200 public health job tasks categorized into 10 work domains. The list of tasks was incorporated into a web-based survey, and a snowball sample of public health professionals provided 4850 usable responses. Respondents rated job tasks as essential (4), very important (3), important (2), not very important (1), and never performed (0).
Results:
The mean task importance ratings ranged from 2.61 to 3.01 (important to very important). The highest mean ratings were for tasks in the ethics domain (mean rating, 3.01). Respondents ranked 10 of the 200 tasks as the most important, with mean task rankings ranging from 2.98 to 3.39. We found subtle differences between male and female respondents and between master of public health and doctor of public health respondents in their rankings.
Conclusion:
The JTA established a set of job tasks in 10 public health work domains, and the results provided a foundation for refining the CPH examination. Additional steps are needed to further modify the content outline of the examination. An empirical assessment of public health job tasks, using methods such as principal components analysis, may provide additional insight.
The National Board of Public Health Examiners (NBPHE) was formed by the Association of Schools and Programs of Public Health (ASPPH) and other public health professional organizations in 2005. Its purpose is to ensure that public health professionals master the foundational knowledge and skills relevant to contemporary public health, which it does by administering a voluntary certification examination and maintaining the certification program. The NBPHE created and first offered the Certified in Public Health (CPH) examination in 2008.
Eligibility for the examination has gradually expanded. Initially, only graduates of Council on Education for Public Health (CEPH)–accredited schools and programs of public health were eligible to sit for the CPH examination. In 2013, eligibility was extended to active students who had completed 21 credit hours at a CEPH-accredited school or program of public health, including courses in the 5 core areas of public health. In 2016, eligibility for the examination was again expanded, this time to anyone who (1) was determined by the NBPHE board of directors to be part of the public health workforce, (2) had a bachelor’s degree, and (3) had 5 years of public health work experience.
The NBPHE board of directors developed the initial body of knowledge (content outline) for the CPH examination based on the Master of Public Health (MPH) Core Competencies, which were created by ASPPH in 2006. 1 By 2013, the NBPHE realized that the ASPPH core competencies did not necessarily reflect the curricula of all degree programs, and it revised the examination’s content outline. The NBPHE modeled the new outline after introductory-level core courses taught at a wide cross section of CEPH-accredited schools and programs of public health, and it was meant to cover subject matter beyond the original ASPPH core competencies.
The approach the NBPHE used to create the CPH examination differed from the approach used by other health professions to develop certification examinations. 2 –4 The traditional approach by most organizations has been to develop a content outline using a job task analysis (JTA), which defines the actual tasks that practitioners in the field must be able to perform, and then to design the examination to assess competency in performing these tasks. Until recently, public health had not used this approach, and it was one of the only health-related professions that had never performed a JTA. Yet public health had become a prime candidate for such an analysis, given that it is a diverse profession, one that includes health professionals working in city, county, state, and territorial health agencies; nonprofit organizations; the private sector; and academic settings, in areas such as health education and promotion, organizational management and leadership, and data collection and analysis.
In 2014, the NBPHE acknowledged that the CPH examination should, to the extent possible, reflect the actual knowledge, skills, and abilities needed by public health practitioners in the workplace, and so it set out to perform the first public health JTA. The new analysis, which was funded by the Robert Wood Johnson Foundation, would become the basis for the CPH examination. The JTA was conducted primarily to validate the CPH examination. However, it also provided a unique opportunity to examine the demographic profile of a segment of the public health workforce, along with the common tasks public health workers perform, by focusing on those actively working as public health professionals with experience in technical, clinical, or managerial roles.
The NBPHE assembled an advisory committee to develop the JTA. This committee comprised representatives of various public health employers who were themselves experts in public health and were directly responsible for hiring and supervising entry-level and mid-level employees. The members of the committee included 3 representatives from federal agencies, 6 from private employers, 3 from local or county health departments, and 5 from schools and programs of public health. 5
The committee began the JTA by reviewing 22 documents conceptualizing public health activity, including the Council on Linkages Between Academia and Public Health Practices’ Core Competencies for Public Health Professionals, 6 the Association of State and Territorial Health Officials Public Health Workforce Interests and Needs Survey, 7 the ASPPH Blue Ribbon Public Health Employers Advisory Board Framing the Future Task Force, 8 and the ASPPH report Framing the Future: A Master of Public Health Degree for the 21st Century. 9 Applied Measurement Professional, Inc, a business division of PSI Services, LLC, that provides certification organizations, professional associations, and private industry with assessment and psychometric consultation, facilitated the JTA by helping develop the task list and the survey instrument and helping analyze the survey results.
The objectives of this article were to describe the development, administration, and results of the JTA survey; to present an analysis of the survey results; and to review the implications of this first-ever JTA of public health.
Methods
Data Collection
The advisory committee first identified 10 public health work domains that would be used to help develop and categorize a series of public health job task statements. The domains were (1) critical and strategic analysis; (2) biological and environmental applications in public health; (3) leadership and systems thinking; (4) management, finance, and policy; (5) program planning; (6) collaborating and partnering; (7) communication; (8) advocacy; (9) ethics; and (10) diversity and cultural proficiency.
The committee then began the iterative process of developing a list of common job tasks or “identifiable work activities” in each content domain. This process resulted in the creation of a list of 200 tasks that would be used in the survey questionnaire. For each task, respondents would be asked to rate its importance based on their own current work (Box), using the following scale: essential (rating of 4), very important (rating of 3), important (rating of 2), not very important (rating of 1), and never performed (rating of 0).
Next, the committee identified the target population for the survey. The committee considered using a list of the occupational categories of the public health workforce published by the Health Resources and Services Administration in 2000. 10 However, that report described difficulties in identifying which segments of the workforce and which jobs should be included in the list. Recognizing these difficulties and given the limited resources available for the JTA, the committee decided to use a snowball sample rather than systematic sampling method to target the population.
The committee directed that access to the survey instrument be emailed to approximately 125 000 people for whom either the NBPHE or ASPPH had contact information. Email recipients were directed to a URL containing the survey. The committee recognized that the survey might not be relevant to many of these people, but it chose to distribute the instrument to as large a group as possible, in part to make the broader public health community aware of the JTA. The recipients included contacts at governmental agencies, public health and related nonprofit organizations, students and alumni of schools and programs of public health, and 51 major public health organizations. 11 The committee also asked the NBPHE board of directors, JTA Advisory Committee members, ASPPH school and program primary representatives, and contacts at CEPH-accredited schools and programs of public health to share the availability of the survey with their constituents. The survey was open from May 23 to June 30, 2014.
Question used in the Job Task Analysis Survey (National Board of Public Health Examiners) to assess task importance, May 23 to June 30, 2014
Use the scale below to express your judgment of the importance of each task as it applies to your
Never performed = 0
Not very important = 1
Important = 2
Very important = 3
Essential = 4
Do not rate statements based on what you think other public health professionals do or should do. Rather, base your ratings on your current work as a public health professional.
Data used in this job task analysis were collected for the development of the CPH examination content outline and the project was not considered human subjects research requiring institutional review board approval.
Data Analysis
NBPHE received 8100 survey responses. Of these, NBPHE removed 633 surveys from respondents who indicated that they had no public health experience, 18 duplicate survey responses, and 1071 surveys from respondents who provided no ratings of the tasks. Of the remaining 6378 surveys, NBPHE excluded an additional 1528 surveys because the respondents provided ratings on fewer than the required threshold proportion of tasks, which NBPHE had previously determined to be 48% (96 of the 200 tasks). After all of these adjustments, 4850 surveys were usable for the JTA.
NBPHE was unable to calculate a response rate, because it was not possible to estimate the number of people who received the initial invitation. However, the frequency distributions for sex, race/ethnicity, education level, employment experience, employment setting, employment level, and geographic location were consistent with the population distributions of these characteristics for the target population. Consequently, the number of responses was deemed acceptable for the purpose of conducting the statistical analyses.
The committee reviewed the combined responses for each job task, to determine whether the task was clearly part of public health practice and whether it was an important enough practice task to warrant inclusion in the subsequent JTA. With this in mind, the committee removed 40 tasks from the subsequent analysis based on the following criteria: (1) ≥40% of the respondents indicated that they never performed the task, (2) respondents gave the task a mean overall importance rating of ≤2.5, or (3) respondents from several key subgroups, specified by employment setting and employment level, gave the task a mean overall importance rating of ≤2.5. The committee made exceptions to these criteria for a small number of tasks that it deemed to be especially important to public health practice. 11
Statistical Methods
We used SPSS version 22.0 12 to conduct 2 types of statistical analyses. First, we assessed the reliability of the tasks identified in each domain; as part of this assessment, we calculated Cronbach α and interrater reliability. Second, we calculated the mean and standard deviation (SD) of task ratings for each domain, both for the entire sample and for subgroups of respondent characteristics, including sex, highest degree attained, employment setting, and employment level.
Results
Cronbach α for the tasks in each domain was ≥0.94, suggesting that each domain consisted of a consistent collection of tasks. Interrater reliability indexes were ≥0.53 for each domain except critical and strategic analysis, which had an index of 0.39. These results indicated that in general, raters agreed on the importance of tasks to a moderate degree.
The mean task importance ratings for the 10 domains ranged from 2.61 to 3.01, and the SDs ranged from 1.00 to 1.06 (Table 1). These numerical results indicated that the mean importance of these tasks ranged from important to very important on the response scale. The highest mean task importance ratings were for ethics (rating of 3.01; SD, 1.03); the lowest ratings were for biological and environmental applications (rating of 2.61; SD, 1.06). Variations in the mean task ratings for each domain were similar and ranged from an SD of 1.06 for biological and environmental applications to an SD of 1.00 for collaborating and partnering and program planning.
Task importance ratingsa for 10 public health work domains, by 4850 respondents to the National Board of Public Health Examiners Job Task Analysis survey, May 23 to June 30, 2014
aFor each task, respondents were asked to rate its importance based on their own current work, using a scale of essential, 4; very important, 3; important, 2; not very important, 1; and never performed, 0. Ratings of 0 (ie, never performed) were not included in task importance rating calculations.
bListed in order of mean scores, from highest to lowest.
cDomains consist of multiple tasks. The number of responses to each task in a domain differs because respondents did not respond to all tasks. Hence, this table indicates the maximum number of responses to any task in the domain and the minimum number of responses to any task in the domain.
Of the 200 tasks in the analysis, respondents rated 10 as most important, with mean task importance rankings ranging from 2.98 to 3.39 (Table 2). Of these tasks, 4 were in critical and strategic analysis, 4 were in ethics, and 2 were in collaborating and partnering. Six of the tasks were associated with data collection and analysis or with ethical issues related to that process (tasks ranked 1st, 2nd, 3rd, 5th, 9th, and 10th).
Task importance ratingsa for the 10 highest-rated public health job tasks, by 4850 respondents to the National Board of Public Health Examiners Job Task Analysis survey, May 23 to June 30, 2014
aFor each task, respondents were asked to rate its importance based on their own current work, using a scale of essential, 4; very important, 3; important, 2; not very important, 1; and never performed, 0. Ratings of 0 (ie, never performed) were not included in task importance rating calculations.
bListed in order of mean scores, from highest to lowest.
cThe No. of Task Ratings is the actual number of people who responded to each task. The maximum number of responses to any task was 4850.
The top 3 mean task ratings for both male and female respondents were for ethics, collaborating and partnering, and critical and strategic analysis (Table 3). For 7 of the 10 domains, women gave higher mean task ratings than men, although differences between men and women were small, ranging from 0 to 0.19. We found a subtle difference between men and women in their relative ranking of the importance of the domains. Men ranked the following 4 domains higher than women did: leadership and systems thinking; management, finance, and policy; advocacy; and biological and environmental applications. Women ranked 3 domains higher: diversity and cultural proficiency, program planning, and communication.
Task importance ratingsa for 10 public health work domains, by 3332 female and 1033 male respondentsb to the National Board of Public Health Examiners Job Task Analysis survey, by sex, May 23 to June 30, 2014
aFor each task, respondents were asked to rate its importance based on their own current work, using a scale of essential, 4; very important, 3; important, 2; not very important, 1; and never performed, 0. Ratings of 0 (ie, never performed) were not included in task importance rating calculations.
bLargest number of individuals that responded to each task.
cListed in order of mean scores for female respondents, from highest to lowest.
Respondents to the survey reported having >28 types of master’s and doctoral degrees, including a master of public health (MPH) degree and a doctor of public health (DrPH) degree. For 6 of the 10 domains, MPH respondents gave higher mean task ratings than DrPH respondents, although differences between the 2 types of respondents were generally small, ranging from 0 to 0.19 (Table 4). We found a subtle difference between MPH and DrPH respondents in their relative ranking of domains. MPH respondents ranked the following 3 domains higher than DrPH respondents: ethics, collaborating and partnering, and management, finance, and policy. DrPH respondents ranked 3 domains higher: critical and strategic analysis, program planning, and advocacy.
Task importance ratingsa for 10 public health work domains, by 2675 MPH and 142 DrPH respondentsb to the National Board of Public Health Examiners Job Task Analysis survey, by degree, May 23 to June 30, 2014
Abbreviations: DrPH, doctor of public health; MPH, master of public health.
aFor each task, respondents were asked to rate its importance based on their own current work, using a scale of essential, 4; very important, 3; important, 2; not very important, 1; and never performed, 0. Ratings of 0 (ie, never performed) were not included in task importance rating calculations.
bLargest number of individuals who responded to each task.
cListed in order of mean scores for respondents with an MPH degree, from highest to lowest.
Discussion
The JTA conducted by the NBPHE established a set of 200 job tasks in 10 public health work domains. The results of reliability testing demonstrated that each domain consisted of a consistent collection of tasks. However, the results suggest that respondents agreed on the importance of tasks to only a moderate degree. Similarly, the sizes of the SDs of the mean task importance ratings also indicated that respondents viewed the importance of tasks within the domains differently. These 2 findings reflect the diverse backgrounds and job responsibilities of the survey respondents.
Respondents rated the tasks in ethics as the most important to performing their work. These tasks included the following: ensuring the application of ethical principles in the collection, maintenance, use, and dissemination of data and information; identifying regulations related to privacy, security, and confidentiality; identifying the laws and regulations that govern the scope of one’s legal authority; and applying basic principles of ethical analysis to issues of public health research, practice, and policy. This JTA established that many of those working in the public health arena saw the most important aspects of their jobs as being focused on working with public health data, in a collaborative setting, and within an ethical framework.
We found subtle differences between men and women and those with different education levels in how they ranked the importance of various public health work domains. The differences in perceptions of importance by sex and education could result from different roles held by the various subgroups of professionals in the public health sector, but they could also be a consequence of the complementary nature of the work conducted by these subgroups of professionals in the field.
Limitations
The primary limitation of the JTA was that it relied on a snowball sample rather than a more systematically representative sampling procedure. Our resources for conducting the JTA did not allow us to use a more sophisticated procedure. The survey also relied on a limited 4-option response scale, (ie, not very important to essential), plus an indication as to whether the task was performed at all (ie, never performed). This approach limited the precision of the analysis of differences among the subgroups of the respondent characteristics.
Conclusions
The JTA conducted by the NBPHE established a set of 200 job tasks, in 10 public health work domains, performed by public health professionals, and the results provided a foundation for refining the CPH examination. However, additional work is needed to better understand the implications of the results of this JTA and to modify the content outline for the examination. The Advisory Committee identified the public health work domains conceptually, and although they are consistent with past conceptualizations by other major public health organizations, an empirical analysis of the domains and task categorization is needed, perhaps using a method such as principal components analysis. Then, the general topics and specific items in the current CPH examination could be systematically compared with the domains and tasks identified by the empirical analysis, thereby further strengthening the examination.
In addition, in the JTA, only a limited analysis was performed of the variables possibly influencing task importance ratings, and only subtle differences in task importance ratings were found based on sex and education. A multivariate analysis is needed to better understand which demographic and structural characteristics most strongly influence both individual task importance ratings and task importance ratings by domain.
This JTA should not be the only analysis conducted. Indeed, a JTA should be conducted approximately every 5 years, and it should be used as a tool to continue refining the CPH examination. With sufficient resources, the next JTA could be markedly improved by establishing a sampling framework that provides a more systematic approach to selecting respondents who better represent the many diverse fields in public health. Finally, the description and analysis provided in this review may also be used to inform how future JTAs could be modified.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.