Predictors of success on the American Board for Certification’s prosthetics certification examination

Abstract

Background:

Many factors can potentially impact pass or fail performance of a certification examination. While studies regarding predictors of success have been conducted in other professions, no such studies have been conducted in prosthetics and orthotics.

Objectives:

Determine whether there are significant differences in prosthetics certification examination success or failure based on gender, Carnegie ranking of the institution from where the candidate received the degree, and whether the candidate is extending credential from orthotics to include prosthetics.

Study design:

Retrospective study.

Methods:

Data of candidates who completed prosthetics residency in 2011 and 2012 were evaluated for relationships with the dependent variables.

Results:

Only credential extension showed a statistically significant relationship with written multiple choice (p = 0.000), written simulation (p = 0.006), and clinical patient management examinations (p = 0.005) as well as with success or failure in obtaining prosthetics certification (p < 0.001).

Conclusions:

Currently available data for analysis regarding predictors of success are limited. Collection of additional variables (i.e. pre-requisite grade point average) in the future will permit analysis of more robust information. Presently available data reveal that candidates seeking to extend credential are more successful in obtaining certified prosthetist status compared with those testing for the first time.

Clinical relevance

Understanding factors contributing to attaining licensure is important as the prosthetic/orthotic practitioner population ages. This is the first study to investigate this dimension of prosthetics and orthotics training, which upholds professional standards and protects patients. It helps the profession develop best educational and clinical practices in managing patients who utilize prosthetic technologies.

Keywords

Education prosthetics

Background

Professional certification serves as a measure of competency that a student has acquired the requisite knowledge for practice. Certification enhances the individual’s credibility and also serves to give the consumer confidence that a recognized authority endorses the individual.¹ In healthcare professions, it serves as a measure of safety. In professions where certification is not mandatory, it is a way for professionals to differentiate themselves from non-certified competitors.

The American Board for Certification in Prosthetics, Orthotics & Pedorthics’s (ABC) certification is recognized as the most rigorous credential in the prosthetic and orthotics professions in the United States.² Students who graduate from a practitioner program in prosthetics and orthotics must pass ABC certification in order to be licensed and become an independent practitioner in 16 states.³

ABC certification is predicated upon passing three examinations: (1) written multiple choice assesses the student’s knowledge of patient and practice management, including knowledge of anatomy, physiology, biomechanics, kinesiology, disease process, pathologies, material science, ethics, and professionalism; (2) written simulation employs seven simulated case scenarios to test the student’s analytic and clinical problem solving skills; and (3) clinical patient management, a hands-on practical examination of knowledge and skills assessed through direct examiner and patient model interaction in a clinical environment.

Various factors can potentially impact pass or fail performance of a certification examination. Other professions including nursing, nurse midwives, athletic training, physical therapy, several medical specializations, education, and financial planning have conducted studies to assess whether specific variables predict performance on their respective certification examinations. American College Testing (ACT) and Scholastic Aptitude Test (SAT) scores, cumulative and course-specific grade point average (GPA), gender, and standardized examinations (by profession) are examples of predictors of success that have been researched for various certification examinations.^4–10

A literature search on predictors of success did not reveal any such studies specific to the prosthetics and orthotics professions. However, these and other variables specific to prosthetics education in the United States could affect the success of students on the ABC prosthetic certification examination. For example, in the United States, the concept of credential extension exists in prosthetic and orthotic certification. Credential extension is when an individual already has the certified prosthetist (CP) or certified orthotist (CO) designation and takes the certification examination for the designation not already held.

Additionally, over the past 3 years, considerable changes have been made in prosthetics and orthotics education in the United States. In 2010, the Master’s degree became the entry-level degree. This movement to the Master’s degree also changed the type of institution eligible to award Post-Baccalaureate degrees. Academic institutions within the United States are classified using different systems such as the Carnegie Classification of Institutions of Higher Education. The Carnegie rankings group institutions based on a variety of criteria, including the type and number of academic degrees awarded, academic focus, size, and others.¹¹

In addition to the change in degree, the National Commission on Orthotic & Prosthetic Education (NCOPE) moved away from an hour-based residency model and adopted an outcome-based format for residencies effective 1 January 2012. The reasoning for doing so was to maintain professional pace with the other health professions (e.g. physical therapist education) and to accommodate an expanding body of prosthetic knowledge resulting from increased emphasis on direct patient care, thus producing professionals capable of carrying out research and education.¹² The expectation is that this will enable prosthetists and orthotists to better serve the profession by improving patient care and help to ensure the profession’s survival.^12–14

It is important to understand variables that may be related to certification examination results. Such analysis may facilitate targeted quality improvement measures to accurately assess professional abilities. Ultimately, improvements in credentialing and examination can only serve to benefit the cadre that will carry out future work in the field.

The purpose of this study was to determine whether there are significant differences in prosthetics certification examination success or failure based on gender, Carnegie ranking of the institution from where the candidate received the degree, and whether the candidate is extending credential. While this study focused on candidates in the United States, the topic could be of interest to the international prosthetic and orthotic education community based on academic and certification requirements by region.

Methods

The study protocol was approved by the University of South Florida’s Institutional Review Board. A retrospective secondary data analysis was used for the study. Discussions with NCOPE determined that the first cohort of residents under the outcome-based regulation would be eligible to take the certification examination as of summer 2013. Thus, in order to avoid the residency confounder, it was determined that the sample would consist of candidates (n = 158) who completed prosthetic residency in 2011 and 2012, All candidates were graduates from one of the 11 NCOPE-accredited programs at the time.

Once received, the data were surveyed for correctness and completeness. Incomplete records were returned to the ABC and NCOPE for verification and correction, where necessary. Following correction, they were included in the data sample.

At the time of this study, the variables used were the only ones that ABC and NCOPE collected. Institution type was a four-level independent variable, grouping institutions according to Carnegie classification.¹¹ In order to protect candidate confidentiality, any institutions which were alone in a particular Carnegie classification were grouped into a category labeled “other.” The resulting classification levels were as follows: research universities with very high research activity (RU/VH), Master’s colleges and universities—larger programs (Master’s L), special focus institutions—medical schools and medical centers (Spec/Med), and Other.

Statistical analysis

Descriptive statistics were reported for gender and institution type, credential extension, the three prosthetics certification examinations, and prosthetics certification results. Chi-square test of independence¹⁵ was used to test whether the categorical variables were independent of one another when the cell count was greater than 5. Fisher’s exact test was used when expected cell counts were less than five. Independent sample t-tests were used to determine relationships between two-level independent variables and continuous dependent variables. Multiple logistic regression was used for prediction in the case of a dichotomous dependent variable. Linear regression was used for prediction with continuous dependent variables. SPSS (v. 22; IBM, Armonk, NY, USA) was used for statistical analysis. Significance was at p ⩽0.05.

Results

Twelve records had incomplete data or data inconsistencies, such as discrepancies between examination results (pass/fail) and certification status. All were corrected and thus included in the data sample. Six records had been assigned a number, meaning that the student had registered for at least one of the examinations, but did not have any examination scores, meaning they never took any of the examinations. These were excluded from the analysis. Two institutions had unique Carnegie classifications, thus potentially making candidates identifiable, and were grouped into the Other category.

Of the candidates seeking prosthetics certification (n = 158) during the study period, 63.9% were male. Regardless of gender, 84.2% achieved certification (85 males, 48 females). Per examination, there were as many as three attempts by candidates to obtain a passing score. After all examination attempts were complete, the percentages were equal by gender, with 84.2% passing and 15.8% failing after all examination attempts were complete.

By institution type, graduates from the special focus institutions—medical schools and medical centers—had a 100% pass rate in this analysis, followed by those from RU/VH with 86.3% pass rate. Those from the other institutions category had an 83.3% success rate and, finally, those from Master’s L, with 76.7% (Table 1).

Table 1.

Certified prosthetist credential pass or fail, by institution type.

Institution type	Certified prosthetist credential				Total
	Pass		Fail
	N	%	N	%
RU/VH	69	86.3	11	13.7	80
Spec/Med	6	100.0	0	0.0	6
Master’s L	23	76.7	7	23.3	30
Other	35	83.3	7	16.7	42
	133		25		158

RU/VH: research universities with very high research activity; Spec/Med: special focus institutions—medical schools and medical centers; Master’s L: Master’s colleges and universities—larger programs.

Of the 158 candidates who took the examination, a total of 73 (46.2%) were seeking to extend credential from CO to certified prosthetist–orthotist (CPO). Certified prosthetist credential pass or fail outcomes, by credential extension and gender, are summarized in (Table 2). While the majority of candidates passed, it is worth noting that the failure rate for those attempting credentialing for the first time was higher.

Table 2.

Certified prosthetist credential pass or fail, by credential extension.

Credential extension	Certified prosthetist credential				Total
	Pass		Fail		N
	N	%	N	%
Yes
Female	21	91.3	2	8.7	23
Male	45	100.0	0	0.0	45
No
Female	27	79.4	7	20.6	34
Male	39	78.3	17	30.4	56

Moderate correlations existed between the three prosthetics certification examinations. Written multiple choice correlated (r = 0.442) with both written simulation and clinical patient management examinations. Written simulation and clinical patient management examinations also correlated (r = 0.446).

Statistically significant relationships were observed between credential extension and success in achieving ABC prosthetics practitioner certification (p < 0.001), written multiple choice examination (p = 0.000), written simulation examination (p = 0.006), and clinical patient management examinations (p = 0.005). No statistically significant relationships were found between gender or institution type and the dependent variables (Tables 1 and 2).

Discussion

The pass rates on the ABC prosthetics certification examination were comparable to those found in similar studies of other professions. The overall pass rate of 84.2% was in the range of other studies, which included 66.3% pass rate on the National Paramedic Certification Examination,¹⁶ a 93% pass rate on the National Certification Examination for nurse midwives,¹⁷ a 92.2% pass rate on the National Council Licensure Examination for practical nursing students,¹⁷ and an 82% pass rate on the National Certification Exam for Nurse Anesthetists.¹⁸

Statistically significant relationships did not exist between gender and any of the dependent variables: prosthetics certification (p = 0.865), performance on the written multiple choice (p = 0.695), performance on the written simulation (p = 0.498), and performance on the clinical patient management (p = 0.319). This was not entirely surprising, since 6 of 11 studies summarized in the literature review that included gender as a predictor variable did not find it to be statistically significant.^9,19–23 The studies that did find statistically significant relationships observed that males performed better than females.^{16,18,24–26} However, further analysis revealed that the studies that reported statistical significance tended to have larger sample sizes, ranging from 273 to 5208.^{16,18,24–26} The studies that did not report statistical significance by gender had sample sizes ranging from 77 to 175,^19,20,22,23 with the exception being one study with a sample size of 505.⁹ It is unclear whether statistically significant relationship by gender might be observed on the prosthetics certification examination if a larger sample size was available to study. Furthermore, this perspective only addresses gender difference by sample size as opposed to subject matter and more importantly, how to address potential differences by gender should they be present.

Similarly, no statistically significant relationship resulted between Carnegie classification and the dependent variables. The literature search yielded just one other study²⁷ which used this variable as a predictor of success. That study also found no significant relationship with Carnegie classification (p = 0.078). In the same study, however, geographic region proved to be a significant predictor of failure on the certification examination (p < 0.001). Given the limited number of prosthetics and orthotics programs across the country, making any predictions based on geographic location or similar variables could prove inconclusive.

Dual credentialing is uncommon in other health fields and finding a comparable scenario in other professions proved slightly more challenging. However, one study²⁸ used certified public accountant (CPA), enrolled agent (EA), or securities licenses and found a relationship (p < 0.05) between those certifications and success on the Certified Financial Planner (CFP) examination. In this study, credential extension was the only independent variable that produced statistically significant results with regard to success on achieving prosthetics certification as well as performance on the three prosthetics certification examinations. The fact that this occurred is plausible. Candidates seeking prosthetics certification who previously obtained orthotics certification presumably had familiarity with the examination process as a whole. Having already gone through one discipline’s examination process, candidates already understood the rigors of studying for such an examination. Candidates also might have learned strategies on how to better study or prepare for the examinations. Specific to the clinical patient management examination, candidates experienced a similar setting once previously, in terms of patient and tester, and were more familiar with what to expect in the examination setting. Furthermore, it is possible that they were able to draw upon their orthotics clinical knowledge and apply it in a prosthetics setting, thus providing an advantage over those who only completed prosthetics residency. Not every graduate from a prosthetics and orthotics program will seek dual-credentialing by credential extension. Educators and residency directors in the prosthetics profession should consider whether learning can be facilitated in those not extending credential so that their success rates can be increased to rates comparable to those who are.

Limitations of the study included the type and amount of information collected by ABC and NCOPE. Studies identified as part of the literature review for this study revealed that variables of interest such as entrance examination scores and GPA were available for predictors of success studies in other professions. However, at the time of the study, neither NCOPE nor ABC captured such information. Additionally, due to the limited data available, it was not possible to further control for the variety of possible degrees that students in the sampled cohort could have and be eligible to take the prosthetics certification examination. Conducting a similar study should the agencies begin to collect variables of interest could provide valuable insight.

Due to the retrospective study design, the study is bound to historical occurrences that cannot be modified. It is not possible to control factors such as the type of candidates who have taken the examination, the time between completing residency and taking the examination, how the examination was administered, and other factors that could potentially affect outcome variables. Also, no intervention has been included in the study design.

Numerous extraneous variables could further limit the study. For example, candidates in the studied cohort could have earned one of three possible credentials—Bachelor’s degree, Master’s degree, or a certificate, and still be eligible to take the same prosthetics certification examination. The difference in degree could potentially affect the amount and type of learning students acquired in their respective program. Also, it is possible that within the group of prosthetic residents who took the certification examination, some may have acquired a longer work history in the field prior to entering the academic setting. This could possibly provide these residents with an advantage when taking the examination. Since students are not required to take the certification examination immediately after completing residency, this gap in time could also potentially affect test scores. Therefore, the fact that neither the ABC nor NCOPE track variables such as prior work history in their respective databases could also pose a limitation.

At the time of this study, ABC was selected because, historically, states wishing to implement CPO licensure had adopted ABC’s standards. However, more recently, the Board of Certification/Accreditation (BOC) has instituted rigorous psychometric testing for its certification examination. A study with BOC data would be valuable in the future as well.

Conclusion

This study was the first conducted regarding predictors of success on the ABC prosthetics certification examinations and prosthetics certification. It presented an opportunity to make the first contribution to the knowledge base and inform the profession, educators, students, and the accrediting bodies regarding the potential influence of these factors on success or failure on the prosthetics certification examinations and prosthetics certification.

The majority of the variables studied here were not predictive of success on the prosthetics certification examination. Only credential extension proved statistically significant as a predictor of performance on the prosthetics certification examination. However, tremendous opportunity exists to gather further data, which would allow analyses similar to those covered in this study’s literature review to be conducted in prosthetics. For example, GPA produced statistically significant results as a predictor of success on some certification examinations^{19,21,29–31} Time between residency completion and taking the prosthetics certification examination is another variable of interest. Currently, there is no requirement limiting the amount of time between one and the other. However, one study found an inverse relationship between time from graduation to success on the registered nurse examination.²¹

With the focus on keeping pace with “other allied health professionals”¹² as an onus, making the recommended changes would undoubtedly help the prosthetics profession keep pace and become a leader in best educational and clinical practices in managing patients who utilize prosthetic technologies.

Footnotes

Author contribution

All authors contributed equally in the preparation of this manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Lysaght

Altschuld

. Beyond initial certification: the assessment and maintenance of competency in professions. Eval Program Plann 2000; 23: 95–104.

About ABC, 2014, https://www.abcop.org/WhoWeAre/Pages/default.aspx

States with O&P Licensure. American Board for Certification in Prosthetics, Orthotics & Pedorthics, 2014, https://www.abcop.org/State-Licensure/Pages/state-licensure.aspx

Burke

. Teacher certification exams: what are the predictors of success? Coll Student J 2005; 39: 784–793.

Foti

DeYoung

. Predicting success on the National Council Licensure Examination-Registered Nurse: another piece of the puzzle. J Prof Nurs 1991; 7: 99–104.

Glick

McClelland

Yang

. NCLEX-RN: predicting the performance of graduates of an integrated baccalaureate nursing program. J Prof Nurs 1986; 2: 98–103.

Froman

Owen

. Predicting performance on the national council licensure examination. West J Nurs Res 1989; 11: 334.

Barkley

Jr Dufour

Rhodes

. Predictors of success on the NCLEX-RN among baccalaureate nursing students. Nurs Health Care Perspect 1998; 19: 132–137.

Beeson

Kissling

. Predicting success for baccalaureate graduates on the NCLEX-RN. J Prof Nurs 2001; 17: 121–127.

10.

Roehrig

. Prediction of licensing examination scores in physical therapy graduates. Phys Ther 1988; 68: 694–698.

11.

Carnegie classification of institutions of higher education, n.d., http://classifications.carnegiefoundation.org

12.

Malas

. The move toward entry-level masters. The O&P Edge, 2006, http://www.oandp.com/articles/2006-03_11.asp

13.

Nielsen

Altman

Gillespie

. A model for graduate education in orthotics and prosthetics. Clin Prosthet Orthot 1987; 11: 63–66.

14.

Supan

. Residency research, part I: why should an orthotic/prosthetic resident conduct research? J Prosthet Orthot 1995; 7: 142–146.

15.

Hosmer

Lemeshow

. Applied logistic regression. 2nd ed. New York: Wiley, 2000.

16.

Fernandez

Studnek

Margolis

. Estimating the probability of passing the national paramedic certification examination. Acad Emerg Med 2008; 15: 258–264.

17.

Fullerton

Severino

. Factors that predict performance on the national certification examination for nurse-midwives. J Nurse Midwifery 1995; 40: 19–25.

18.

Zaglaniczny

. Factors which predict performance on the national certification examination for nurse anesthetists. AANA J 1992; 60: 533–540.

19.

Englert

. The relationship between selected variables and the national council licensure examination for registered nurses: a comparative analysis of pass/fail performance for traditional and second-degree baccalaureate students. Ann Arbor, MI: ProQuest LLC, 2009.

20.

Fortier

. Predictors of success on the national council licensure examination for registered nurses among transfer BSN students. Ann Arbor, MI: ProQuest LLC, 2010.

21.

Benefiel

. Predictors of success and failure for ADN students on the NCLEX-RN. Ann Arbor, MI: ProQuest LLC, 2011.

22.

Shellito

Osland

Helmer

. American Board of Surgery examinations: can we identify surgery residency applicants and residents who will pass the examinations on the first attempt? Am J Surg 2010; 199: 216–222.

23.

Kim

Wallace

Allbritton

. Predictors of success on the written anesthesiology board certification examination. Int J Med Educ 2012; 3: 225–235.

24.

Stewart

Bates

Jr Smith

. Does performance on school-administered mock boards predict performance on a dental licensure exam? J Dent Educ 2004; 68: 426–432.

25.

McClintock

Gravlee

. Predicting success on the certification examinations of the American Board of Anesthesiology. Anesthesiology 2010; 112: 212–219.

26.

Lary

. A critical analysis of variables that predict success in the physician assistant program of study and on the National Commission on Certification of Physician Assistants examination. Doctoral dissertation, Kansas State University, Manhattan, KS, 1991.

27.

Utzman

Riddle

Jewell

. Use of demographic and quantitative admissions data to predict performance on the national physical therapy examination. Phys Ther 2007; 87: 1181–1193.

28.

Grange

Hampton

Cutler

. Factors associated with success on the CFP certification examination. Financ Serv Rev 2003; 12: 95–114.

29.

Utzman

Riddle

Jewell

. Use of demographic and quantitative admissions data to predict academic difficulty among professional physical therapist students. Phys Ther 2007; 87: 1164–1180.

30.

Bolender

. Predictors of certification scores in family nurse practitioners: personality, academic, and demographic factors. Milwaukee, WI: Cardinal Stritch University, 2001.

31.

Middlemas

Manning

Gazzillo

. Predicting performance on the National Athletic Trainers’ Association Board of Certification Examination from grade point average and number of clinical hours. J Athl Train 2001; 36: 136.