Sage Journals: Discover world-class research

Abstract

Purpose:

In 2022, the American Board of Physical Medicine and Rehabilitation (ABPMR) contracted with Professional Services Industries LLC to complete a Pediatric Rehabilitation Medicine (PRM) practice analysis. The goal was to develop an updated examination content outline that accurately depicts the knowledge and skills expected for a competent PRM physician following completion of training. The purpose of this paper is to describe the process of practice analysis and to present candidate perceptions of scope of practice and relevancy of content from the first two examinations in which the new outline was applied.

Methods:

The practice analysis process was undertaken in four distinct stages: subject matter expert meetings to determine the tasks and knowledge areas; the development, issuance and analysis of a national survey; development of exam specifications; and results verification. Results were analyzed by a psychometrician and team to determine mean relative importance of task and knowledge items. A new outline was then created and used for the administration of the 2023 and 2024 ABPMR PRM Certification Examinations.

Results:

All active board certified PRM physicians identified were sent surveys via e-mail. One hundred ninety six (61.1%) respondents completed the survey. The top five task statements in order of perceived importance were performing history and physical examinations, managing muscle and tone abnormalities, prescribing orthoses and equipment, prescribing therapy services, and making appropriate specialty care referrals. The top five knowledge statements in order of perceived importance were cerebral palsy, spinal cord injury, abnormal growth and development, neuromuscular disorders, and musculoskeletal conditions. The new PRM Examination content outline created included six domains and 22 subdomains of knowledge content areas. A higher percentage of initial certification candidates sitting for the 2023 and 2024 ABPMR PRM Certification Examinations selected “Strongly Agree” or “Agree” when responding to the examination content reflecting both scope of training and relevancy.

Conclusion:

A practice analysis process was undertaken resulting in a new content outline for the ABPMR PRM Certification Examination. Following the implementation of the new content outline, initial certification candidate survey responses showed improved favorability regarding content reflecting scope of training and relevancy.

Keywords

pediatrics practice analysis rehabilitation medicine certification examination scope of practice

Introduction

The American Board of Physical Medicine and Rehabilitation (ABPMR) was first approved as a member board of the American Board of Medical Specialties (ABMS) in 1946. This was completed in efforts to provide the rehabilitation expertise required to treat large numbers of veterans returning home with physical impairment following World War II.¹ The first physiatrist recognized in the literature and with major impact upon the physical rehabilitation of children was George G. Deaver, MD, at the Institute for the Crippled and Disabled in New York City, New York (circa 1932).^2,3,4 The polio epidemic of the 1950s and 1960s resulted in a heightened number of children experiencing paralysis and the need for further development of the field of Rehabilitation Medicine for children.^2,3,5 Pediatric rehabilitation curricula was first required within Physical Medicine and Rehabilitation (PM&R) residency programs in 1967.⁶ In 1987, the ABPMR and the American Board of Pediatrics (ABP) acknowledged the need for additional and formal training in Pediatric Rehabilitation Medicine (PRM) through combined residency training programs leading to certification in both PM&R and Pediatrics.⁷ In 1999, the ABMS granted approval of PRM as a PM&R subspecialty,⁶ and curricula with program requirements for PRM fellowships (one year for combined program physicians and two years for PM&R trained physicians) were initiated.⁷ The Accreditation Council of Graduate Medical Education (ACGME) approved training requirements in 2002, and the ABPMR offered the first subspecialty certification examination in PRM in 2003.^7,8

Much has changed since Dr George Deaver provided rehabilitation services to children eight decades ago. As of December 2024, 424 ABPMR diplomates have received an initial subspecialty certificate in PRM (per the ABPMR Database at https://www.abpmr.org/). Most of these physicians practice in larger cities and urban areas with fewer practicing in rural and less populated geographic regions.^7,9,10 New diagnoses and medical and rehabilitation treatments have emerged, requiring training for the PRM physician to evolve.^11,12 Training and certification requirements must be regularly updated to reflect the demands of an ever-changing clinical landscape.

Each certification examination offered by the ABPMR is based on a content outline which specifies the medical topics and the relative percentage of each topic to be included on the examination. The outline is expected to reflect the unique knowledge a competent board-certified physician possesses to provide specialized care for their patients. From 2002 to 2022, the PRM Examination content outline has had one major revision, which took place in 2014, and annual reviews completed by the ABPMR PRM Examination Committee. All changes to the examination outline were made by the PRM Committee of the ABPMR with feedback from board-certified PRM physicians in the field.

A well-established systematic process of updating the examination outline termed “practice analysis” was applied to field of PRM in 2022–2023. The goal was to develop an updated examination content outline that accurately reflects the current breadth and depth of knowledge and skills expected for a competent PRM physician following completion of training pathways approved by the ABPMR for certification in PRM. The purpose of this paper is to describe the process of practice analysis and to present candidate perceptions of scope of practice and relevancy of content from the first two annual examinations in which the new outline was applied.

Methods

In 2022, the ABPMR contracted with Professional Services Industries LLC (PSI) to complete a PRM job analysis. PSI is a global leader with over 70 years of experience in credentialing and assessment solutions delivering over 28 million examinations annually related to education and career development.¹³ Three other vendors were interviewed and considered. PSI was chosen based on their model which follows industry standards for practice analysis and references from other licensure/certification programs. The ABPMR and PSI do not work together in any other way (e.g., test prep materials, examination development).

Industry standards, or criteria generally accepted by experts in the measurement field, suggest that validity of the use of assessment scores is supported by evidence-based processes for the development and evaluation of assessments. Job analyses are often the first step in the validation process for assessments because they identify the scope of the practice and serve as the primary basis for validity evidence.^14,15 The PRM analysis was conducted in accordance with principles and practices outlined in the Standards for Educational and Psychological Testing,^14,15 which describe guidelines for all aspects of test development, including content validation. It was also conducted in a manner consistent with the fellow Boards of Medicine and Dentistry as discussed below.^16–24

The job analysis process was undertaken in four distinct stages: job analysis subject matter expert (SME) meetings to determine the tasks and knowledge areas that reflect the job role; the development, issuance and analysis of a national survey; development of exam specifications; and results verification through a linkage analysis. The four stages involved in this practice analysis were based on industry/PSI best practices for the development of certification and licensure measurement.

SMEs were chosen by the ABPMR as advanced and well-recognized leaders in the field of PRM. The SMEs were then screened by PSI, according to rigorous industry standards, to assure they represented a variety of years in experience, professional backgrounds, practice settings (rural, urban, academic, non-academic), and regions of practice. Ten SMEs were selected but only seven were able to complete their initial interview. Appendix A lists the respective certifications and basic information regarding training background and current practices of these seven SMEs.

In preparation for the process that followed, interviews with the seven SMEs were conducted by a PSI psychometrician remotely, following rigorous industry standards, regarding job roles of a PRM physician as well as anticipated future changes of the role. The following questions were asked during these initial interviews:

What is your perspective (or experience if not a committee member or former committee member) on or with the PRM exam?

Do you feel the present content outline addresses all key components of PRM practice?

What is missing?

How current is it based on today's practice?

For teachers: What is different from the present outline from what you are teaching?

What is critical to the practice of PRM?

What are the trends in PRM practice that need to be considered or addressed?

Do you have other observations to share on the PRM exam or PRM practice that we have not discussed already?

The ABPMR PRM Examination Committee along with SMEs served as the Job Analysis Committee (JAC). Of the seven SMEs interviewed initially by PSI, three were able to commit to the workload requirements of the JAC. The JAC reviewed the recommendations of the interviewed SME group, the outgoing examination outline, and ACGME fellowship training requirements as a starting point for the practice analysis. Appendix B displays the physician members of the JAC, their respective certifications, and basic information regarding training background and current practices.

PSI then conducted a remote JAC meeting on January 14 and 15, 2022, to discuss the scope of practice and develop a list of tasks and knowledge areas that reflect the job role of the competent PRM physician following completion of training approved for certification by the ABPMR. PSI led the JAC in refining task and knowledge statements and organizing them into a domain and subdomain structure. Following the January committee meeting, the JAC went through multiple rounds of knowledge and task revisions as a means of ensuring all content was covered while still remaining relevant to busy practitioners. This iterative process took place over multiple months.

JAC members were then asked to identify domains, subdomains, and statements that may have required modification or removal. Subsequently, committee members were asked for domain, subdomain, and statements that may have needed to be added based on practice changes. All tasks, knowledge areas, and domains were debated vigorously and then established in final form by 100% consensus of the JAC members.

In this context, the terminology of “tasks” reflected specific work activities performed to fulfill job responsibilities such as performing a history and physical (H&P) exam. “Knowledge areas” referred to understanding information necessary to fulfill job responsibilities of selected tasks such as understanding the treatment of spasticity. The JAC then grouped the refined knowledge areas into eight domains: (1) neurologic disorders, (2) musculoskeletal disorders, (3) cancer, (4) cardiovascular disorders, (5) pain and functional syndromes, (6) developmental disorders, (7) growth and development, and (8) genetic disorders.

In order to gather input from the field, a survey was created which included the detailed list of tasks and knowledge areas established through the JAC. The survey was initially created by PSI staff and was sent for review by both the JAC and additional ABPMR staff. These stakeholders were asked to suggest edits to the survey for readability and comprehension. Any suggested edits were incorporated before the final survey was released to the public. A copy of the national survey can be found in Supplementary Material.

The survey asked respondents to rate the importance of each task and knowledge area and the frequency in which they apply the knowledge within their day-to-day practice. The survey sought to define both importance of specific topics to the field in general (the topics that someone in the field might apply to the rehabilitative care of children even if the person answering the question might not use it in their own practice) and the importance of each topic to the individual (what they individually used in their own practice). For each task and knowledge statement, survey respondents were asked, “How important is this task/knowledge to the field of Pediatric Rehabilitation Medicine?” using the four-point Likert scale of (1) not important, (2) minimally important, (3) moderately important, and (4) very important. Respondents were reminded that PRM physicians practice in a variety of settings, including rural areas, where access to medical specialists may be limited. Along with the importance ratings, respondents were asked, “How often do you perform this task in your individual Pediatric Rehabilitation Medicine practice?” The scale used for task statements was a 7-point scale of (0) never, (1) less than annually, (2) annually, (3) quarterly, (4) monthly, (5) weekly, and (6) daily. For knowledge statements, respondents were asked how often they applied knowledge of a specific topic to their practice: (0) never, (1) seldom, (2) occasionally, or (3) frequently. Demographic information was collected to ensure broad representation. The PRM national survey was distributed through SurveyMonkey between October 13^th and November 23^rd, 2022, to all 321 active subspecialty board-certified PRM physicians identified in the ABPMR database at the time.

As part of the post survey analysis, a psychometrician reviewed the raw survey data for signs of irregular survey responses. Primarily, the psychometrician reviewed the variance in responses to flag and remove any responses that lacked variance. For example, if a survey respondent used the same Likert rating for every statement within the knowledge or task section, this likely indicated a respondent who did not put the appropriate time and consideration into their responses. Using associated Likert scales, the mean relative importance of each task and knowledge item was determined. A frequency cut point of 2.5 was chosen, through careful deliberation by the JAC, as the identifying point for task or knowledge statements that were rarely completed and/or needed. During the exam specifications meeting, JAC members were informed that the primary rating scale for the inclusion or exclusion of a statement was “importance.” The frequency rating scale functioned as a secondary measure to assist in identifying statements that were done infrequently and should be flagged for exclusion. Statements that had a high importance rating but had a low frequency rating were deemed critical incidents which should be included for their vital importance. Statements that were flagged as having a low importance and frequency rating were flagged for removal, and the JAC were asked to either remove or defend the inclusion of the statement.

A measure of central tendency was then calculated based on respondents’ importance ratings for each of the task and knowledge statements. These measures were then used to calculate initial content weights using the two methods of “top-down” and “bottom-up” weighting.^14,15 For bottom-up weighting, the criticality value of each individual knowledge statement (mean importance rating multiplied by the mean frequency rating) was calculated, and a draft content weight for each statement was developed based on the statement's criticality relative to all other statements. For top-down weighting, the JAC focused on the relative criticality of each content area (i.e., each subdomain of the outline). The top-down approach helped detect when a content area might be over-represented simply based on the number of knowledge areas within the content area. The JAC then reviewed the draft content weighting, discussing any adjustments necessary to align the number of items per content area for adequate content coverage on the assessment. After thorough discussion, and with 100% consensus, final content weights for each domain were determined by the JAC to better align with the expectations and requirements of a current PRM Certification Examination. Following the creation of the task and knowledge lists, “linkages” between the task and knowledge statements were created in order to ensure that each task was covered by at least one relevant knowledge area and that each knowledge area had at least one relevant task identified.

Results

All active board certified PRM physicians as of October 7, 2022 (n = 321), were sent surveys via e-mail in addition to several reminders to complete the survey. One hundred ninety six (61.1%) respondents completed the survey. After analysis of the data, all input from the 196 respondents was included in the data set with no individual responses removed. One hundred ninety four physicians responded as currently employed, and two (1%), responded as being retired. All 196 physicians were certified by the ABPMR with 33 being dually certified by the ABPMR and the ABP.

Table 1 displays professional degree and Table 2, practice setting, by type and percentage.

Table 1.

Survey respondents by professional degree.

	MD (N/%)	DO (N/%)	MBBS (N/%)	Missing (N/%)
Professional Degree	133/67.9	21/10.7	1/0.5	41/20.9

Table 2.

Survey respondents by practice setting.

	Rural (N/%)	Suburban (N/%)	Urban (N/%)	Missing (N/%)
Practice Setting	11/5.6	34/17.3	110/56.1	41/20.9

Table 3 demonstrates the mean ratings of respondents for frequency and importance of the tasks statements. The top five task statements were, in order of perceived importance, performing H&P exams, managing muscle and tone abnormalities, prescribing orthoses and equipment, prescribing therapy services, and making appropriate surgical, psychological, and medical specialty referrals when needed. The top five tasks, in order of frequency performed, were perform H&P exams, prescribe and oversee therapy, manage movement and tone abnormalities, identify risk factors, and establish appropriate prognoses.

Table 3.

Frequency and importance ratings for task statements.

#	Task	FrequencyMean	ImportanceMean
1	Perform a history and physical exam	5.90	3.99
2	Order or interpret the results of diagnostic studies to assist rehabilitative care of patients
2A	Laboratory	4.61	3.30
2B	Genetic	3.09	2.94
2C	Imaging	5.07	3.72
2D	Electrodiagnosis	2.30	2.86
2E	Neuro Psychological	4.07	3.40
2F	Motion Analysis	2.83	3.08
2G	Urodynamics	2.28	2.78
2H	Cardio/Pulmonary Function	1.93	2.40
2I	Swallow	4.08	3.55
2J	Sleep	3.22	2.88
2K	Autonomic	2.27	2.63
2L	EEG	1.64	2.14
3	Determine diagnoses (primary or secondary) and etiology	5.14	3.61
4	Identify risk factors (e.g., premorbid, diagnosis-related, treatment-related, sociodemographic)	5.37	3.66
5	Establish prognosis	5.27	3.74
6	Prescribe and oversee therapy (e.g., physical therapy, occupational therapy, and speech therapy)	5.88	3.96
7	Perform procedures (Hands on participation)
7A	Botulinum toxin injections
7A1	Muscle	4.71	3.92
7A2	Salivary gland	1.67	2.90
7A3	Bladder	0.08	1.76
7B	Phenol/alcohol neurolysis	2.18	3.40
7C	Myofascial trigger point injections	1.24	2.46
7D	Joint injections	0.46	1.98
7E	Occipital or other nerve injections	0.45	1.92
7F	Spine Injections	0.10	1.53
7G	Procedural guidance using
7G1	Ultrasound	2.06	3.15
7G2	Electrical stimulation	3.50	3.49
7G3	EMG	2.16	2.98
7H	Conscious sedation	1.51	2.13
7I	Electrodiagnostic studies	1.01	2.80
7J	Intraoperative monitoring for SDR or other spine surgeries	0.55	2.57
7K	Baclofen pump
7K1	Refills	3.15	3.73
7K2	Programming	3.46	3.79
7K3	Problem solving/diagnostics	2.94	3.77
7L	Serial casting	2.44	3.32
7M	Urodynamics	0.81	2.51
8	Evaluation or oversite of the following care programs
8A	Fluid and electrolyte abnormalities	3.11	2.89
8B	Nutrition including feeding tubes	3.68	3.24
8C	Gastroesophageal reflux	3.07	2.69
8D	Respiratory insufficiency requiring assisted ventilation (e.g., cough assist, CPAP, BiPAP, tracheostomy)	2.73	2.90
8E	Sensory impairment (including vision and hearing disorders)	3.53	3.28
8F	Movement and tone abnormalities	5.77	3.98
8G	Seizures	2.34	2.56
8H	Hypercoagulable state and DVT/PE	2.69	2.95
8I	Pain	4.80	3.54
8J	Sleep disorders	3.87	3.10
8K	Bowel dysfunction	5.03	3.78
8L	Bladder dysfunction	4.61	3.71
8M	Pressure injury	3.93	3.70
8N	Behavioral disorders	4.13	3.24
8O	Bone health	3.83	3.42
8P	Pubertal disorders	1.89	2.42
8Q	Immunizations in individuals with disabilities	2.32	2.90
8R	Pharmacology management (i.e., inpatient)	4.02	3.73
8S	Vision and/or hearing assessment	2.60	2.88
8T	Burns	1.46	2.88
9	Refer for surgical, medical, psychological evaluation and/or treatment	5.32	3.93
10	Evaluate and prescribe orthosis	5.61	3.98
11	Evaluate and prescribe prosthesis	2.34	3.60
12	Prescribe DME and technology
12A	Prescribe equipment (e.g., mobility devices, adaptive aids, equipment, and augmentative communication)	5.53	3.98
12B	Prescribe environmental controls/robotics	2.77	3.27
13	Advocacy related to community integration
13A	Prescribe therapy related to recreation	4.05	3.59
13B	Prescribe driving evaluations	3.04	3.47
13C	Vocational and educational	3.96	3.67
14	Manage ethical issues	3.71	3.64
15	Facilitate pediatric to adult transitional care	3.95	3.90
16	Provide rehabilitative care for adults with childhood onset disability	3.32	3.32
17	Provide hospice and palliative care services	1.33	2.61

EEG: electroencephalogram; EMG: electromyography; SDR: selective dorsal rhizotomy; CPAP: continuous positive airway pressure; BiPAP: bilevel positive airway pressure; DVT: deep venous thrombosis; PE: pulmonary embolism; DME: durable medical equipment.

Table 4 demonstrates the mean ratings of respondents for frequency and importance of the knowledge statements. The top five knowledge statements were, in order of perceived importance, cerebral palsy (traumatic, hypoxic, and stroke included), spinal cord injury (traumatic and neural tube included), abnormal growth and development, neuromuscular disorders (myopathies and neuropathies included), and musculoskeletal (contractures, scoliosis, gait disorders, and hip dysplasia). The top five knowledge statements were, ordered by frequency of application, cerebral palsy (traumatic, hypoxic and stroke included), abnormal growth and development, musculoskeletal (contractures, scoliosis, gait disorders, and hip dysplasia), spinal cord injury (traumatic and neural tube included), neuromuscular, and neurologic developmental disorders.

Table 4.

Frequency and importance ratings for knowledge statements.

		Frequency	Importance
1.	Knowledge	Mean	Mean
1.	Neurologic Disorders
1.A.	Brain Disorders
1.A.1.	Cerebral palsy	2.94	4.00
1.A.2.	Traumatic (including non-accidental)	2.89	3.98
1.A.3.	Anoxic	2.78	3.98
1.A.4.	Stroke (such as ischemic, hemorrhagic)	2.62	3.92
1.A.5.	Infectious (such as meningitis, abscess, encephalitis)	2.31	3.76
1.A.6.	Autoimmune/inflammatory/demyelinating (e.g., acute disseminated encephalomyelitis, anti-NMDAR encephalitis, Rasmussen's encephalitis)	2.13	3.72
1.A.7.	Degenerative CNS (e.g., leukodystrophies)	1.91	3.58
1.B.	Spinal Cord Disorders
1.B.1.	Neural tube defects (including Chiari malformations and diastematomyelia)	2.33	3.90
1.B.2.	Traumatic	2.36	3.96
1.B.3.	Non-traumatic (such as vascular, infectious, inflammatory, autoimmune)	2.06	3.78
1.B.4.	Tethered cord/lipoma	2.01	3.73
1.B.5.	Syrinx	1.77	3.66
1.C.	Neuromuscular Disorders
1.C.1.	Neuropathies	2.06	3.70
1.C.2.	Plexopathies	1.70	3.63
1.C.3.	Neuromuscular junction disorders	1.29	3.31
1.C.4.	Myopathies/muscular dystrophies	1.96	3.83
1.C.5.	Motor neuron disease	1.71	3.66
2.	Musculoskeletal Disorders
2.A.	General
2.A.1.	Arthrogryposis	1.52	3.42
2.A.2.	Torticollis	1.99	3.50
2.A.3.	Infantile club foot	1.31	2.69
2.A.4.	Osteogenesis imperfecta	1.14	2.90
2.A.5.	Limb length discrepancy	2.15	3.44
2.A.6.	Torsional deformities (in toeing/out toeing)	2.46	3.47
2.A.7.	Fractures	1.23	2.31
2.A.8.	Sports-related injuries	1.54	2.92
2.A.9.	Toe walking	2.71	3.78
2.B.	Spine Conditions
2.B.1.	Idiopathic scoliosis	1.13	2.38
2.B.2.	Neuromuscular scoliosis	2.77	3.81
2.B.3.	Back, neck, and/or limb related pain	2.31	3.27
2.B.4.	Scheuermann's disease and vertebral compression fractures	0.96	2.39
2.B.5.	Spondylosis, spondylolisthesis, spondylitis	1.12	2.56
2.C.	Hip Disorders
2.C.1.	Slipped capital femoral epiphysis and/or Perthe's disease (avascular necrosis femoral head)	0.92	2.69
2.C.2.	Hip dysplasia	2.62	3.71
2.C.3.	Transient synovitis	0.83	2.43
2.C.4.	Contractures	2.94	3.96
2.C.5.	Heterotopic ossification	1.68	3.59
2.C.6.	Osteoporosis	2.02	3.33
2.C.7.	Overuse syndromes (e.g., lateral/medial epicondylitis, Osgood Schlatter's)	1.34	2.82
2.C.8.	Hemihypertrophy	1.03	2.58
2.D.	Limb deficiency/amputation
2.D.1.	Upper extremity	1.22	3.63
2.D.2.	Lower extremity	1.35	3.67
2.E.	Rheumatic Disease
2.E.1.	Juvenile idiopathic arthritis (e.g., enthesitis-related arthritis, systematic, oligoarticular, polyarticular, psoriatic, undifferentiated)	1.05	2.73
2.E.2.	Systematic autoimmune diseases (including SLE, systematic sclerosis, JDM, JPM)	1.03	2.69
3.	Cancer
3.A.	Central nervous system tumors including neurofibromatosis	1.92	3.47
3.B.	Musculoskeletal tumors	1.42	3.13
3.C.	Blood cancers	1.35	2.66
4.	Cardiovascular Disorders
4.A.	POTS	1.49	2.75
4.B.	Transplant	1.07	2.50
4.C.	Lymphedema	0.77	2.29
4.D.	Heart failure and insufficiency	0.89	2.30
5.	Pain and Functional Syndromes
5.A.	Chronic Pain Syndromes
5.A.1.	Hypermobility	2.15	3.26
5.A.2.	CRPS	1.69	3.36
5.A.3.	Headache	1.54	2.55
5.A.4.	Fibromyalgia	1.13	2.50
5.A.5.	Traumatic	1.82	3.20
5.A.6.	Functional/conversion disorders	1.85	3.32
6.	Developmental Disorders
6.A.	PDD/autism	2.01	2.90
6.B.	ADHD (primary ADHD)	1.83	2.72
6.C.	Developmental coordination disorders/apraxia	1.88	3.15
6.D.	Neuromuscular, neurologic, or cognitive related	2.64	3.65
7.	Growth and Development
7.A.	Normal growth	2.16	3.43
7.B.	Abnormal growth	2.43	3.53
7.C.	Normal development	2.26	3.63
7.D.	Abnormal development	2.90	3.90
8.	Genetic Disorders
8.A.	Cystic fibrosis	0.33	1.86
8.B.	Congenital dystonias	1.51	3.30
8.C.	Metabolic disorders (including MPS)	1.38	2.93
8.D.	Mitochondrial disorders	1.47	3.05
8.E.	Chromosomal entities (e.g., Trisomy 21, Angelman syndrome, Lesch Nyhan syndrome, tuberous sclerosis, neurofibromatosis, Rett syndrome, ataxia telangiectasia, Friederich's ataxia, Prader-Willi, Fragile X, skeletal dysplasia, Marfan syndrome, hereditary spastic paraplegia, hemophilia, sickle cell)	2.31	3.54

Anti-NMDAR: anti-N-methyl-D-aspartate receptor; SLE: systemic lupus erythematosus; JDM: juvenile dermatomyositis; JPM: juvenile polymyositis; POTS: postural orthostatic tachycardia syndrome; CRPS: complex regional pain syndrome; PDD: pervasive developmental disorder; ADHD: attention deficit hyperactivity disorder; MPS: mucopolysaccharidoses.

Table 5 lists the number of years in practice (post PRM training) of the respondents. Data is lacking for 41 individuals (20.9% of respondents). All of the respondents had at least one year of practice post formal PRM training. Training backgrounds of the respondents are listed in Table 6. Data is missing for 42 (21.4%) of the respondents. Table 7 displays the number of clinical care days per week with 71 respondents (36.2%) providing direct patient cares services five days per week. Table 8 identifies distribution by state, district and/or territory.

Table 5.

Number of years in practice (post PRM training) of the survey respondents.

How long have you been practicing PRM (post formal training)?	n	%
Less than a year	0	0.0%
1 to 3 years	25	12.8%
4 to 6 years	17	8.7%
7 to 10 years	30	15.3%
11 to 15 years	26	13.3%
16 to 20 years	21	10.7%
21 to 30 years	29	14.8%
31 years or more	7	3.6%
Missing	41	20.9%

PRM: Pediatric Rehabilitation Medicine.

Table 6.

Training backgrounds of the the survey respondents.

Which of the following BEST describes your training background culminating with PRM initial certification?	n	%
Fellowship trained	98	50.0%
Combined pediatric and PM&R program	36	18.4%
Distinct and separate Pediatrics and PM&R programs with at least 6 months of PRM training	5	2.6%
Practice pathways from preceding multi-year PRM practice	15	7.7%
Missing	42	21.4%

PRM: Pediatric Rehabilitation Medicine; PM&R: Physical Medicine & Rehabilitation.

Table 7.

Number of clinical care days per week of the survey respondents.

How often do you provide direct clinical care to patients in days per week? Please select the choice that most closely approximates your individual practice.	n	%
1/2 day per week	1	0.5%
1 day per week	3	1.5%
2 days per week	8	4.1%
3 days per week	25	12.8%
4 days per week	49	25.0%
5 days per week	71	36.2%
Missing	39	19.9%

Table 8.

Distribution by state, district and/or territory.

In which geographic location do you PRIMARILY work?	n	%
Alabama	3	1.5%
Alaska	0	0.0%
Arizona	2	1.0%
Arkansas	1	0.5%
California	9	4.6%
Colorado	10	5.1%
Connecticut	0	0.0%
Delaware	2	1.0%
Florida	6	3.1%
Georgia	4	2.0%
Hawaii	1	0.5%
Idaho	2	1.0%
Illinois	4	2.0%
Indiana	1	0.5%
Iowa	0	0.0%
Kansas	1	0.5%
Kentucky	1	0.5%
Louisiana	1	0.5%
Maine	0	0.0%
Maryland	5	2.6%
Massachusetts	3	1.5%
Michigan	8	4.1%
Minnesota	9	4.6%
Mississippi	0	0.0%
Missouri	8	4.1%
Montana	0	0.0%
Nebraska	0	0.0%
Nevada	0	0.0%
New Hampshire	0	0.0%
New Jersey	4	2.0%
New Mexico	1	0.5%
New York	3	1.5%
North Carolina	1	0.5%
North Dakota	1	0.5%
Ohio	20	10.2%
Oklahoma	1	0.5%
Oregon	3	1.5%
Pennsylvania	6	3.1%
Rhode Island	0	0.0%
South Carolina	1	0.5%
South Dakota	1	0.5%
Tennessee	1	0.5%
Texas	11	5.6%
Utah	1	0.5%
Vermont	0	0.0%
Virginia	2	1.0%
Washington	4	2.0%
West Virginia	2	1.0%
Wisconsin	5	2.6%
Wyoming	0	0.0%
District of Columbia	1	0.5%
American Samoa	0	0.0%
Guam	0	0.0%
Northern Mariana Islands	0	0.0%
Puerto Rico	0	0.0%
U.S. Virgin Islands	0	0.0%
Other (please specify)	1	0.5%
Missing	45	23.0%

The JAC met on January 24, 2023, to review the survey results and create the new examination content outline and content weighting for the examination. Results and demographics were felt to be consistent with a broad representation of the PRM provider population, the majority of whom are fellowship trained. As a result of the information gained from the survey, the JAC excluded eight task statements because of low scores on task importance and frequency ratings. The JAC also excluded nine knowledge statements due to a variety of factors including content covered by similar statements, low ratings, or merger with another section. For example, the JAC approved merger of multiple domains (cardiovascular, pain, and functional syndromes) into the singular domain of “Pain, Functional Syndromes, and Activity Intolerance.” As a part of this merger, specific topics of transplant medicine, lymphedema, and heart failure/insufficiency were removed as individual categories. The topics of acute pain and postural orthostatic tachycardia syndrome were added to the outline. Table 9 provides a summary of the content weighting determination. The output of the process was a new PRM Examination content outline, which included six domains and 22 subdomains of knowledge content areas (Table 10).

Table 9.

Content weighting determination.

Knowledge Statements		Criticality	Percentage	# Items
A	Neurologic Disorders		50%	120
A1	Brain Disorders	9.72	30%	72
A2	Spinal Cord Disorders	8.03	12%	29
A3	Neuromuscular Disorders	6.37	8%	19
B	Musculoskeletal Disorder		21%	51
B1	General	6.34	15%	36
B2	Limb Difference/Amputation	5.69	4%	10
B3	Rheumatic Disease	2.81	2%	5
C	Cancer	4.91	5%	12
D	Pain, Functional Syndrome, and Activity Intolerance	5.07	7%	17
E	Growth and Development	9.99	8%	18
F	Genetic Disorders	5.57	9%	22

Table 10.

Exam content outline for 240 total scoring items.

1.	Neurologic Disorders	120
1.A.	Brain Disorders	72
1.A.1.	Cerebral palsy
1.A.2.	Traumatic (including non-accidental)
1.A.3.	Anoxic
1.A.4.	Stroke (e.g., ischemic, hemorrhagic)
1.A.5.	Infectious (e.g., as meningitis, abscess, encephalitis)
1.A.6.	Autoimmune/inflammatory/demyelinating (e.g., acute disseminated encephalomyelitis, anti-NMDAR encephalitis, Rasmussen's encephalitis)
1.A.7.	Degenerative (e.g., leukodystrophies)
1.B.	Spinal Cord Disorders	29
1.B.1.	Neural tube defects
1.B.2.	Traumatic
1.B.3.	Non-traumatic (e.g., vascular, infectious, inflammatory, autoimmune)
1.B.4.	Tethered cord
1.B.5.	Syrinx
1.C.	Neuromuscular Disorders	19
1.C.1.	Neuropathies
1.C.2.	Plexopathies
1.C.3.	Neuromuscular junction disorders
1.C.4.	Myopathies/muscular dystrophies
1.C.5.	Motor neuron disease
2.	Musculoskeletal Disorders	51
2.A.	General	36
2.A.1.	Arthrogryposis
2.A.2.	Torticollis
2.A.3.	Osteogenesis imperfecta
2.A.4.	Torsional deformities (e.g., toeing/out toeing)
2.A.5.	Fractures
2.A.6.	Sports-related injuries
2.A.7.	Toe walking
2.A.8.	Bone health disorders
2.A.9.	Contractures
2.A.10.	Heterotopic ossification
2.A.11.	Overuse syndromes
2.A.12.	Spine Conditions
2.A.12.a.	Idiopathic scoliosis
2.A.12.b.	Neuromuscular scoliosis
2.A.12.c.	Scheuermann disease, vertebral compression fractures
2.A.12.d.	Spondylosis, spondylolisthesis, spondylitis
2.A.13.	Hip Disorders
2.A.13.a.	Slipped capital femoral epiphysis, avascular necrosis femoral head
2.A.13.b.	Hip dysplasia
2.A.13.c.	Transient synovitis
2.B.	Limb Difference/Amputation	10
2.B.1.	Upper extremity
2.B.2.	Lower extremity
2.B.3.	Limb length discrepancy
2.C.	Rheumatic Disease	5
2.C.1.	Juvenile idiopathic arthritis (e.g., enthesitis-related arthritis, systematic, oligoarticular, polyarticular, psoriatic, undifferentiated)
2.C.2.	Systematic autoimmune diseases (e.g., systemic lupus erythematosus, systematic sclerosis, juvenile dermatomyositis, juvenile polymyositis)
3.	Cancer	12
3.A.	Central nervous system tumors
3.B.	Musculoskeletal tumors
3.C.	Blood cancers (e.g., leukemia)
4.	Pain, Functional Syndromes, and Activity Intolerance	17
4.A.	Hypermobility
4.B.	Complex regional pain syndrome
4.C.	Headache
4.D.	Fibromyalgia
4.E.	Acute pain
4.F.	Functional neurologic disorder
4.G.	Orthostatic intolerance
5.	Growth and Development	18
5.A.	Growth
5.B.	Normal development
5.C.	Abnormal development (e.g., pervasive developmental disorder, autism spectrum disorder, developmental delay, developmental coordination disorders/apraxia, intellectual disability)
6.	Genetic Disorders	22
6.A.	Metabolic disorders (e.g., mucopolysaccharidosis)
6.B.	Mitochondrial disorders
6.C.	Other chromosomal entities (e.g., Trisomy 21, Angelman syndrome, Lesch Nyhan syndrome, tuberous sclerosis, neurofibromatosis, Rett syndrome, ataxia telangiectasia, Friederich's ataxia, Prader-Willi, Fragile X, skeletal dysplasia, Marfan syndrome, hereditary spastic paraplegia, hemophilia, sickle cell, dystonias)
	Total	240

Anti-NMDAR: anti-N-methyl-D-aspartate receptor.

The new outline was used for the 21^st administration of the ABPMR PRM Certification Examination on October 30, 2023, for 37 candidates which included 23 first-time candidates and one repeat candidate taking the examination for initial certification. In addition, there were 12 first-time candidates taking the examination for continuing certification (CC) and one repeat CC candidate.

The new outline was also utilized for the 22nd administration of the ABPMR PRM Certification Examination on October 8, 2024, for 40 candidates. This included a cohort of 25 first-time candidates seeking initial certification, 13 first-time candidates taking the examination for CC in PRM, and two repeat CC candidates.

Figures 1 and 2 display a 10-year analysis of examination content relating to training programs and relevancy to the field of PRM. A greater percentage of initial certification candidates sitting for the 2023 and 2024 ABPMR PRM Certification Examinations selected “Strongly Agree” or “Agree” when responding to the content reflecting scope of training compared to past candidate responses. Similarly, a greater percentage of initial certification candidates sitting for the 2023 and 2024 ABPMR PRM Certification Examinations selected “Strongly Agree” or “Agree” when responding to the relevancy of examination content compared to past candidate responses.

Figure 1.

Ten year analysis of examination content reflecting scope of residency/fellowship training.

Figure 2.

Ten year analysis of examination content reflecting relevancy to the field of PRM.

Discussion

As the field of PRM evolves over time, careful analysis of core knowledge and skills is a necessity. As the certifying body for PRM physicians, a fundamental responsibility of the ABPMR is to assure adequate knowledge base for certified individuals. The methodology of practice analysis facilitates a systematic assessment of a specialty of medicine that creates a credible product available for functions ranging from facilitating item writing, content outlines, and course work for graduate medical education (residents and fellows) to longitudinal assessments of practicing PRM providers.^16,17,18 The PRM practice analysis identified the job tasks and knowledge base that a newly trained PRM physician must possess to perform the job. The first two examination administrations, using items reflecting the new content outline, demonstrated improved favorability from initial certification candidates with respect to content reflecting scope of training and relevancy. Whether these improvements relate to this practice analysis remains somewhat unclear but is encouraging to the process overall.

The process of practice analysis has been utilized by the ABP for General Pediatrics¹⁹ and in a number of pediatric specialties including Pediatric Critical Care Medicine,¹⁶ Pediatric Hospital Medicine,¹⁷ Pediatric Cardiology,²⁰ Neonatal-Perinatal Medicine,²¹ and Pediatric Rheumatology.²² Additional published reports of use include Post-Acute and Long-Term Care Medicine,¹⁸ Occupational and Environmental Medicine,²³ and Pediatric Dentistry.²⁴ Each of these groups used the process to define the unique set of knowledge, skills, and abilities of a new practitioner in their field and to develop either a new or updated certification examination outline.

The new PRM Examination outline differs in format from previous outlines, while the content is largely similar. Prior PRM examination outlines included two components: Class 1 representing type of problem or organ system and Class 2 representing the focus of the question or patient management. Each examination question was previously developed from a combination of Class 1 and Class 2. The new outline is simplified to a singular list of eight domains with 22 subdomains of knowledge content areas. Each domain carries a weight. For example, neurologic rehabilitation comprises 50% of the PRM Examination. The simplification is intended to improve clarity for the examinee studying for their certification examination, board examination development, ongoing test item writing assignments, evaluation of success of test items, and compilation of year over year examinations and data. Under the new outline, identification of questions by the ABPMR will hopefully become easier, eliminating outdated queries and locating gaps in knowledge and tasks that need to be addressed. Fellowship program directors may find it more helpful in their quest to develop a comprehensive program curriculum eliminating targeted deficiencies in their education of residents, fellows, and departmental staff. The new outline is also the product of a comprehensive practice analysis completed using professional industry standards, which was not accomplished in previous outlines.

Strengths of the process include a robust 61% survey completion rate with broad representation of diplomates from 38 different US states and the District of Columbia, including those serving rural and urban populations. This response rate exceeded rates of 10–43% for other medical specialties that have reported their practice analysis results^15–23 and provides some assurance that the outcome reflects the current entry level job role for PRM providers in the United States. Weaknesses include missing data from 20.9% of respondents with respect to number of years in practice, educational background, and practice setting with 21.4% of data absent regarding individual training programs. Some of the missing data may have been the result of survey fatigue secondary to the survey requiring 30 min to complete. The use of importance and frequency Likert scales are standard within the measurement industry when conducting a practice analysis^{14, 15}. However, there are typical limitations when working with survey data such as response bias, answers being influenced by previous survey questions, and respondents tending to avoid choosing extreme options such as ‘totally agree’ or ‘totally disagree’ even when these options are accurate. The four-point Likert scale was chosen to allow for meaningful responses from survey participants by avoiding the use of a middle ground rating. Some inherent bias is likely present despite the diverse, competent task force of SMEs. Potential bias is also recognized and noted in efforts to be as transparent as possible. Self-reporting error may have occurred in addition, with respect to diplomates’ recollection of their individual practice patterns.

Conclusion

A practice analysis process was undertaken resulting in an updated content outline for the ABPMR PRM Certification Examination. Following the implementation of the new content outline, initial certification candidate survey responses regarding content reflecting scope of training and relevancy showed improved favorability. The PRM practice analysis results will hopefully improve clarity for the examinee studying for their certification examination along with board examination development and evaluation of test item success. It is intended to help define competencies for the new graduate and those with established practice in a more scientific way, defensible to colleagues and the concerned public. It is anticipated that an improved understanding of the vast nature of individual PRM practices spread across large geographic regions and serving diverse patient care needs will be fostered by this work. It is ultimately hoped that this practice analysis will benefit the patients and families served in PRM with improved patient care for many years to come.

Supplemental Material

sj-pdf-1-prm-10.1177_18758894251353067 - Supplemental material for Determination of core knowledge and skills for Pediatric Rehabilitation Medicine certification: A systematic practice analysis

Supplemental material, sj-pdf-1-prm-10.1177_18758894251353067 for Determination of core knowledge and skills for Pediatric Rehabilitation Medicine certification: A systematic practice analysis by Kevin P Murphy, David W Pruitt, Susan Apkon, Carolyn L Kinney, Mikaela M Raddatz, Siddiq M Kassam and Sherilyn W Driscoll in Journal of Pediatric Rehabilitation Medicine

Supplemental Material

sj-pdf-2-prm-10.1177_18758894251353067 - Supplemental material for Determination of core knowledge and skills for Pediatric Rehabilitation Medicine certification: A systematic practice analysis

Supplemental material, sj-pdf-2-prm-10.1177_18758894251353067 for Determination of core knowledge and skills for Pediatric Rehabilitation Medicine certification: A systematic practice analysis by Kevin P Murphy, David W Pruitt, Susan Apkon, Carolyn L Kinney, Mikaela M Raddatz, Siddiq M Kassam and Sherilyn W Driscoll in Journal of Pediatric Rehabilitation Medicine

Footnotes

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplemental material

Supplemental material for this article is available online.

Appendix A

Job Analysis Initial Interviews

Appendix B

Job Analysis Committee Members

Physicians Only

References

https://www.aapmr.org/about-physiatry/history-of-the-specialty

Dennis Matthews MD (original source). Dr. Molnar Lectureship, AAPM&R Annual Assembly 2015; Boston, MA.

https://www.aacpdm.org/about-us/history

Flanagan

Diller

. Dr. George deaver: the grandfather of rehabilitation medicine. PM&R 2013; 5: 355–359,. https://doi.org/10.1016/j.pmrj.2013.03.031

https://history.nih.gov/display/history/Polio+Timeline

Turk

Neufeld

. Pediatric rehabilitation medicine subspecialty training. PM R 2009 Dec; 1: 1055–1057. PMID: 20006313.

Turk

Gans

Kim

, et al. A call for action: increasing the pediatric rehabilitation medicine workforce. J Pediatr Rehabil Med. 2023; 16: 449–455. PMID: 37718879; PMCID: PMC10578265.

Driscoll

Massagli

McMahon

, et al. Performance of pediatric rehabilitation medicine candidates on the subspecialty board certification examination from 2003 to 2015. PM R 2018 Apr; 10: 391–397. Epub 2017 Oct 10. PMID: 29024755.

Murphy

Raddatz

Robinson

. Performance on the American board of physical medicine and rehabilitation certifying examinations: rural and urban physicians. PM&R. 2023; 15: 87–93.

10.

Houtrow

Zigler

Pruitt

. The State of the Field: Results from the 2014 and 2017 Pediatric Rehabilitation Medicine Practice Surveys. PM R. Aug 9 2019. doi:https://doi.org/10.1002/pmrj.12235

11.

Alexander

Matthews

Murphy

(eds). Pediatric Rehabilitation Principles and Practice 5th Ed. New York: Demos Medical, 2015.

12.

Murphy

McMahon

Houtrow

(eds). Pediatric Rehabilitation Principles and Practice 6th Ed. New York: Demos Medical, 2021.

13.

https://www.psionline.com/; psiexams.com

14.

ISO/IEC 17024:2012 Conformity Assessment standards – General requirements for bodies operating certification of persons. ISO/IEC, Geneva, Switzerland, 2012. Standard reviewed and confirmed again in 2018.

15.

American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association, 2014.

16.

Turner

Boyer

Dwyer

, et al. Establishing the knowledge and skills necessary in pediatric critical care medicine: a systematic approach to practice analysis. Pediatr Crit Care Med 2020 Jul; 21: 667–671. PMID: 32195904.

17.

Mittal

Shah

Dwyer

, et al. Developing content for pediatric hospital medicine certification examination using practice analysis. Pediatrics 2020 Aug; 146: e20193186. PMID: 32727825.

18.

Morton

Kaplan

Nelson

, et al. Defining the core skills and activities of the attending physician in post-acute and long-term care. J Am Med Dir Assoc 2021 Sep; 22: 1778–1783.e4. Epub 2021 Jun 29. PMID: 34214464.

19.

Dwyer

Althouse

. Validity evidence for the general pediatrics board certification examinations: a practice analysis. J Pediatr 2018 Dec; 203: 4–6.e3. PMID: 30470383.

20.

Brucia

Taggart

Sagiv

, et al. Applying practice analysis to develop a new test content outline for the pediatric cardiology certification examination. Pediatr Cardiol 2023 Jun; 44: 1057–1067. Epub 2022 Dec 12. PMID: 36508019.

21.

Hubbard

Wambach

LaTuga

, et al. Identifying the essential knowledge and skills for neonatal-perinatal medicine: a systematic analysis of practice. J Perinatol 2022 Sep; 42: 1266–1270. Epub 2022 Jun 22. PMID: 35732728.

22.

Brucia

Hayward

Brunner

, et al. Practice analysis and determining the knowledge and skills expected of a pediatric rheumatologist. ACR Open Rheumatol 2023 Nov; 5: 619–623. Epub 2023 Sep 25. PMID: 37749708; PMCID: PMC10642243.

23.

Baker

Levin

O'Hara

, et al. American Board of preventive medicine updated occupational and environmental medicine examination content and practice analysis. J Occup Environ Med 2023 Nov 1; 65: 891–898. Epub 2023 Sep 11. PMID: 37705422.

24.

Davis-Becker

Younger

, et al. Developing content for American board of pediatric dentistry certification examinations using practice analysis. J Am Dent Assoc 2023 Oct; 154: 922–929. Epub 2023 Aug 28. PMID: 37642612.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.35 MB