The first 10 years of the American Board of Vascular Medicine

Abstract

The American Board of Vascular Medicine (ABVM) was conceived through the Society for Vascular Medicine and this year will complete 10 years of certifying physicians who practice vascular medicine and endovascular medicine. The value of certification to our physicians, patients, and field cannot be understated. This paper reviews the highlights of the test development process, quality assurance measures, and management of these high stakes examinations.

Introduction

The field of specialty care of vascular medicine encompasses a wide spectrum of illness. The rapid expanse of knowledge, technology, and patients (baby boomers) that are affected with vascular disease requires an expanding health care team. From of these challenges, the Society for Vascular Medicine (a multidisciplinary nonprofit society), came the realization that certification of physicians who practiced vascular medicine was critical to the advancement of the field and formed the American Board of Vascular Medicine (ABVM). The ABVM offered their inaugural board certification examinations in 2005 and gave their tenth set of examinations in the fall of 2014. To date, there have been 651 and 820 successful diplomats of the general and endovascular tests, respectively. These physicians have dedicated themselves to the practice of vascular medicine and endovascular medicine for patients with peripheral vascular disease.

Preparation for the first ABVM examination

The Society for Vascular Medicine (SVM) surveyed its members in 2002 to determine the degree of interest in vascular medicine certification. Ninety-five percent of responding membership favored vascular medicine recognition through board certification. Thus, the Board of Trustees of the SVM provided a loan to establish the ABVM as a 501(c)(6) corporation in 2003. During the ensuing 18 months, a nine-member physician board was elected, bylaws and a charter were written, trade marking and tax status were legally established, a budget was approved, a testing company and staff were hired (administrative director Micheline Watt and accountant David Grob). Simultaneously, volunteer subject matter experts (SMEs) representing cardiology, vascular medicine, radiology, and vascular surgery met in New York City and Cleveland to develop content outlines for the general and endovascular examinations. The SMEs created a comprehensive test blue print from which item development was undertaken by another set of volunteer experts. These items were written in the standard form of a multiple-choice question with one key (correct answer) and three distractors. The items were reviewed and edited by committee, by ACT (ACT, Inc., Iowa City, IA, USA) (the first testing company utilized by ABVM) and by the chair for each test. Items were selected in the proper weight from each category to provide 175 items for the general examination and 100 items for the endovascular examination. A simulation examination was also developed with the assistance of Dr Mark Bates and the Mentice corporation (Odinsgatan, Sweden). This was the inaugural use of computer simulation for endovascular procedural testing.

Modifications to the ABVM process

The initial testing company for the 2005 and 2006 examinations was ACT, an organization which is most widely known for its college entrance exam. The American Registry of Diagnostic Medical Sonographers (ARDMS), Rockville, MD (the testing organization which developed certification pathways for vascular laboratory technologists and vascular laboratory interpreting physicians) was contracted by ABVM in 2007 and provided testing support services through 2012. In 2013, a 3-year contract was established with PRI, San Diego, CA to provide testing. Each of these companies has provided high-quality leadership; ensured psychometric validity of the examinations; helped to establish cut scoring, and provided marketing assistance to the ABVM. The endovascular examination blue print was revamped in 2014 in an attempt to remain current to this ever-changing field. The number of items for each test was modified in 2014, consisting of 135 items on the endovascular examination and 150 items on the general examination. The simulation examination was trialed in 2005 and 2006, but was removed from the active test because of inconsistencies with the metrics and haptics of the simulators.

Many committed physicians have served the ABVM through writing, editing, supporting, advertising, and educating. The current board of directors includes Drs J Michael Bacharach, Teresa Carman, Mark A Creager, Bruce H Gray, Jonathan L Halperin, Scott Kinlay, Michael R Jaff, Greg O vonMering, and David Paul Slovut.

Examinees

Qualifications to sit for either examination include an active state medical license, successful American Board of Medical Specialties (ABMS) primary board recognition, and a letter of attestation regarding commitment to the practice of vascular or endovascular medicine. Candidates can apply through the formal fellowship pathway or the practice pathway (only available through 2015). Formal fellowship training in vascular medicine has been defined in previous documents.^1,2 Because of the limited number of fellowships available in vascular medicine, the practice pathway was open for those candidates who have committed at least 50% of their practice to vascular medicine. The number of candidates applying through the practice pathway has gradually declined with the majority now receiving formal fellowship training (Figure 1). The majority of candidates have their primary certificate in internal medicine and cardiovascular medicine (Figure 2). The geographic distribution of candidates shows that the majority of candidates come from the East and Midwest regions compared to the South and West regions of the United States (Figure 3).

Figure 1.

Distribution of the application pathway for the general and endovascular ABVM examinees from 2008 to 2012.

Figure 2.

Distribution of the training background for examinees of the ABVM general and endovascular examinations from 2005 to 2013.

Figure 3.

Geographic distribution of examinees of the ABVM general and endovascular examinations from 2005 to 2013.

Knowledge content

The knowledge content required for the examinations is provided in Tables 1 and 2. The general examination is comprehensive with content ranging from carotid, aneurysmal and peripheral artery disease (PAD) to venous thromboembolic disease to congenital, environmental, and occupational vascular disorders. The endovascular examination is focused on patient selection, technical issues, complications and expected outcomes for arterial and venous disease.

Table 1.

Content outline for the general vascular examination (2005–2014).

Topic	No. of items	% of total
Peripheral artery disease	24	14%
Cerebrovascular disease	12	7%
Visceral artery disease	11	6%
Aortic disease	2	12%
Vasospastic and thermal disease	6	4%
Venous disease	21	11%
Thrombosis	3	2%
Lymphatic disease	4	2%
Vasculitis and connective tissue disease	17	10%
Leg ulcers	7	4%
Risk factors and prevention	12	7%
Congenital vascular anomalies	3	2%
Neurovascular compression syndrome	4	2%
Management of vascular surgery	7	4%
Vascular laboratory	15	9%
Vascular biology	7	4%
Total	175

Table 2.

Content outline for endovascular examination (2005–2013).

Topic	No. of items	% of total
Anatomy and physiology	7	7%
Pathology	4	4%
Patient care	8	8%
Treatment and intervention	31	31%
Protocols	7	7%
Diagnostic studies	12	12%
Case studies	29	29%
Other	2	2%
Total	100

Test reliability and results

After test administration, items are psychometrically analyzed for precision and accuracy (p-value and point-biserial correlation).^3,4 A p-value measures item difficulty and is interpreted as the percentage of candidates who answered the item correctly. Point-biserial correlation measures the item’s ability to differentiate between higher and lower ability candidates. Items with a high point-biserial correlation (>0.2) suggest that high-achievers were more likely to correctly answer the item, whereas low-achievers were more likely to incorrectly answer the item. Items with low point-biserial correlation (<0.05) suggest the item may contain an ambiguous question, contain more than one right answer, or is based on rarely known fact(s). Items that detract from the measurement precision are deleted from the final analysis.

Two types of reliability statistics are reported. Alpha is a commonly cited reliability statistic for norm-referenced tests that assesses whether people would rank in the same way across multiple sets of items or administrations of the examination. On the scale from zero to 1.0, a value of near 1.0 is desirable for high-stakes examinations. Over the first 9 years, both the general and endovascular examinations have had alpha scores consistently between 0.87 and 0.92. The second statistic is the Livingston’s coefficient, which measures the quality of the pass/fail decision using a zero to 1.0 scale. The ABVM examinations have consistently measured at 0.90, indicating a satisfactory level of decision-making precision.

In 2013, the average general examinee answered 71% of the scored items correctly and the endovascular examinee answered 69% of the scored items correctly. Percent-correct scores are impacted by both the ability of the examinees and the difficulty of the items and so cannot be directly compared across tests or years. Item response theory scores enable longitudinal comparisons and have remained acceptable.

Recommended minimum passing (cut) scores were established using the bookmark method of standard setting. ⁵ These cut scores are performed initially and then periodically (~3 years). ABVM uses a percentage of items from previous test administrations allowing the ‘carry over’ of the cut score from one year to another. The bookmark method begins with collaboration among subject matter experts to establish a definition of minimum acceptable knowledge competence for the certification. After the definition of minimum knowledge competence was established, subject matter experts were presented a list of ordered examination items beginning with the least difficult and ending with the most difficult. Subject matter experts were then asked to individually identify the item at which they expected a minimally competent candidate to begin struggling and the item at which a minimally competent candidate would likely answer incorrectly. Following the completion of item rating assignment, ratings are aggregated and the mean bookmark rating for all items is used to determine the panel-recommended cut score. Using these scores, pass rates are calculated.

Table 3 depicts the number of tests given per year and the yearly pass rates. The pass rates are lower than expected according to the published pass rates for most American Board of Internal Medicine (ABIM) specialties of 88–95%. This may be reflective of the need for more vascular medicine education in our primary specialties, as well as more opportunities for review courses and continuing medical education dealing with peripheral vascular disease. As of October 2014, the total number of diplomats for the general examination is 651, and 820 diplomats have passed the endovascular examination.

Table 3.

The number of tests and percent pass rates given by the American Board of Vascular Medicine for the general and endovascular examinations by year.

Year	2005	2006	2007	2008	2009	2010	2011	2012	2013	Totals
Gen
N=	146	126	99	103	92	104	87	71	62	890
%	76%	71%	74%	73%	73%	66%	82%	72%	59%	72%
Endo
N=	145	92	105	109	102	136	119	106	95	1009
%	85%	84%	90%	82%	75%	73%	78%	81%	80%	82%

Recertification

The primary ABVM certificate in both general and endovascular medicine has a time limit of 10 years. Starting in 2015, recertification examinations are available for diplomats from 2005 and 2006. Each examination will consist of 75 items, cost less than the primary certificate, and after successful completion will be time honored for 10 years. Examinations will be administered on-demand, year-round, with immediate score reporting at PSI secure testing sites. Each candidate is given a unique set of items that is content-equivalent and statistically equivalent. Candidates who fail an examination can retake with a minimum of 12 weeks in between administrations. Those who fail the examination three times must reapply for primary certification.

The future direction of the ABVM

The specialty recognition of vascular and endovascular medicine through the ABMS requires a primary board sponsor (i.e. the ABIM). It is the desire of the ABVM and the Society for Vascular Medicine to continue our pursuit of this status. The expanding body of knowledge that encompasses our specialty practice and the expanding population with vascular disease requires recognition. Hopefully, as the number of ABVM diplomats grows our petition for specialty recognition will be accepted. In the meantime, we need to continue to attract more practitioners into our field and embrace the recognition of each other through the ABVM.