Abstract
Most members of the American College of Veterinary Pathologists (ACVP) believe that the ACVP certifying examination is an excellent process for recognizing entry-level competency in veterinary clinical pathology and veterinary anatomic pathology; indeed, some individuals believe that it is the most rigorous and best-designed certification process within the veterinary medical specialties. Recent changes include allowing anatomic pathology candidates to choose modules in the Veterinary Pathology section, elimination of the Toxicologic Pathology module within the Veterinary Pathology section, use of digital images instead of Kodachrome slides, and spreading the examination over a 3-day period. These modifications have not changed the structure, format, or grading of the examination or provided for objective evaluation of current certification processes. The ACVP leadership is discussing strategies to evaluate and validate the current certification process. These actions are necessary to demonstrate that the ACVP certifying examinations are fair, relevant to current entry-level practice, and compliant with current published standards for excellence in professional certification. To remain a leader in veterinary specialty certification, the College must critically evaluate all aspects of its certification process. To achieve these ends, it is important for the ACVP to be able to show the following:
The contents of the certifying examinations are relevant to the critical skills and knowledge required for successful practice of the profession.
The difficulty of the examination content is appropriate for entry-level professionals entering a broad range of veterinary pathology positions.
The examinations are of equivalent difficulty from year to year.
The minimal passing score is determined fairly and objectively and can be justified.
The ACVP certification process is compliant with current, widely accepted standards.
Several task forces were established as a result of the 2007 ACVP Strategic Plan (http://acvp.org/members/pdf/07stratplan.pdf) to address specific issues related to certification, including role delineation, training programs, and (later on) maintenance of certification. The recommendations from these groups contributed to the development of the 2012–ACVP Strategic Plan (http://www.acvp.org/members/docs/2012-2017_STRATEGIC_PLAN_FINAL.pdf). As part of this plan, the Council appointed a Certifying Examination Board to recommend appropriate certification strategies and action plans to the Council and to oversee implementation of recommendations approved by the Council. Specific charges from the Council to the Certifying Examination Board included several elements: Develop a long-term strategy for improvements in the examination and examination process based on established criteria for certifying examinations. Plan and provide oversight for a cut-score analysis. Revise the examination format incorporating new technologies and accreditation standards. Develop Standard Operating Procedures (SOPs) for the role delineation-based mapping of examination questions. Modernize the examination question databases (in conjunction with the Examination Committee). Implement a phased 2-part certifying examination (in conjunction with the Examination Committee and Training Program Committee).
It is the responsibility of any professional certifying body (eg, the ACVP) to objectively demonstrate that its certifying processes, including the contents of its examinations, are fair, reasonable, relevant to practice of the profession, and consistently applied for all candidates. The first step is to define the skills and knowledge essential for the current proficient entry-level practice of pathology by a new diplomate. The ACVP has conducted a role delineation survey for each veterinary pathology specialty to define these skills and knowledge (see http://acvp.org/members/pdf/RDSReportV2.pdf). The second step is to define the content of the examination in written test plans that include the types of information to be tested as derived from the information in the role delineation report to ensure that the examination content is relevant to entry-level practice. The test plans also state the relative weights of different categories of information to be tested. Once such test plans are established, the certifying body must ensure that the examinations follow the test plans. Role delineation information and the test plans should be available to candidates and mentors to guide the training of future veterinary pathologists. The ACVP is creating test plans to be used to construct the certifying examinations based on information from the role delineation surveys.
According to best practices for professional certification, the minimal passing score (the cut score) for the certifying examination must be set objectively. The cut score should be determined by conducting an objective, documented assessment of all questions in an examination. Objective determination of the cut score also is recommended by the American Board of Veterinary Specialties, and the ACVP is working to accomplish this. To appropriately set the cut score, a group of experts (diplomates) must review every question used in the examination, determine if a competent entry-level candidate should be able to correctly answer each question, and determine whether the question reflects the critical entry-level knowledge expected of successful candidates. This process of standardization should be led by a professional psychometrician experienced in professional certification procedures. Once this process is completed for one examination, the results may be used to set the cut score for future examinations if a significant proportion of the questions used in one examination are used in subsequent examinations. This extrapolation process for the cut score, known as equating, is only valid if the role delineation information and test plans are equivalent across examinations, the structure and content of the examinations remain essentially similar, and enough questions from all examination sections are reused to allow a statistical comparison of the candidates’ performance from one examination to another. This requires a searchable, flexible database of examination questions, correct answers, foils, references, year(s) used, and candidate performance. Such a database will also support Examination Committee functions, including creation and review of questions, reuse of high-quality questions, mapping of questions to test plans, and construction of each examination.
Current standards of best practice for professional certifying bodies such as the ACVP are found in guidelines such as those published by the Institute for Credentialing Excellence (ICE). 1 One of the ICE recommendations is that certifying activities should have independent governance separate from the governance of other undertakings of the certifying organization (managed by the ACVP Council). In the future, a Certifying Examination Board would administer the ACVP certification process, preventing real or perceived conflicts of interest for Council, as well as allowing individuals qualified and dedicated to the certification process to lead certification activities. The Council will consider this recommendation in the future as new processes are developed.
Changes are under way to validate and modernize the ACVP certification process to better demonstrate its objectivity, fairness, and relevance for ACVP members, individuals interested in becoming certified, and society at large. Such major changes in the organization and function of the ACVP certification process are being made with considerable and careful deliberation. Such changes will require great effort from many ACVP volunteers and also a significant financial investment. ACVP members should be actively involved in these decisions and their implementation. Please address your questions, opinions, and suggestions to the Council.
Footnotes
Author Note
The views expressed in this letter reflect the opinions of the authors and do not necessarily indicate policies or views of the ACVP.