Is the Diagnosis of fifth Metatarsal Fracture Type Consistent? An Interobserver Reliability Study

Abstract

Background:

Treatment pathways of fifth metatarsal fractures are commonly directed based on fracture classification, with particular proximal fractures typically requiring closer observation and possibly more aggressive management. Our aim is to investigate the interobserver reliability of assessment of subtypes of fifth metatarsal fractures.

Methods:

We included all patients referred to our virtual fracture clinic with a suspected or confirmed fifth metatarsal fracture. Plain anteroposterior radiographs were reviewed by 2 novice observers, initially trained on the fifth metatarsal classification identification. Eight different zones were defined based on anatomical location. Patients were excluded from analysis if neither observer could identify a fracture. An interobserver reliability analysis using Cohen κ coefficient was carried out, and degree of observer agreement described using Landis and Koch’s description. All data were analyzed using IBM SPSS, version 27.

Results:

A total of 1360 patients who met the inclusion criteria were identified. The 2 observers had moderate agreement when identifying fractures in all zones, apart from zone 1.2 and distal metaphysis (DM) fractures, which only achieved fair agreement (κ = 0.308 and 0.381 respectively). Zone 3 has slight agreement with zone 2 proximally, and there is an apparent difficulty with distal diaphyseal shaft (DS) fractures, resulting in a lot of crossover with DM, achieving a fair level of agreement (DS 312 vs 196; DM 120 vs 237; κ = .398, P < .001).

Conclusion:

Reliability of subcategorizing fifth metatarsal fractures using standardized instructions conveys moderate agreement in most cases. If the region of the fracture is going to be used in an algorithm to guide a management plan and clinical follow-up during a virtual clinic review, defining fractures of zones 1-3 needs careful consideration.

Level of Evidence:

Level III, retrospective cohort study.

Introduction

Metatarsal fractures are one of the most frequently encountered injuries of the foot and account for approximately 5% to 6% of all fractures seen in primary care, with fifth metatarsal fractures being the most prevalent subtype.^7,25 Fifth metatarsal fractures can further be categorized into zones using various classification systems that have been used to guide treatment protocols. ²⁵ For example, proximal fractures are divided into 3 zones based on their relationship to the tuberosity and the fourth-fifth intermetatarsal articulation (zones 1, 2, and 3) as defined by the most frequently used Lawrence and Botte classification, as well as by mechanism. ¹⁵ Management of fifth metatarsal fractures can depend on the classification of the fracture and therefore in many cases the mechanism of injury, as well as other injuries sustained and patient demographics. ¹ It has been shown that it is safe and cost-effective to discharge many fifth metatarsal fractures following virtual review.^2,9,33 However, fractures of the fifth metatarsal base have been a source of confusion since their original description by Jones in 1902 and there is a lack of clear consensus regarding what is now considered a “Jones fracture,” the prognosis and treatment pathways for such fractures.^5,6,13,18

Multiple classification systems exist, especially for the fifth metatarsal base fractures, the most widely known being the description by Lawrence and Botte. ¹⁵ The reproducibility of the various classification systems and eponymous terms has been investigated in prior studies, demonstrating inconsistency in the use of the term “Jones fracture” and levels of interobserver variability that is considered concerning when using 3-zone fifth metatarsal base systems.^17,22 However, these studies involve low numbers of radiographs reviewed and do not address the fifth metatarsal as a whole.

The ability to accurately diagnose the fracture location is key to decision making, and mistakes in this region may partly be at fault for issues with the management of fractures around the watershed area. ²² Most of these can be managed nonoperatively, but intervening surgically is more common. Historically, these zones 2 and 3 have been preferred for operative management, although over time conservative management with closer observation has become more common and may require aggressive operative management especially when displaced. ¹ The early operative management for such fractures in competitive athletes with early intramedullary screw fixation is strongly advocated for in the existing literature.^19,20,27,29 However, there is a lack of high-quality Level I research to back up management decisions.

If classification systems are used to dictate management, they should be valid and clinicians should be able to reliably grade fractures to assist in planning treatment. In this study, we sought to investigate if the location of fifth metatarsal fracture could be reliably reproduced between “novice” observers using a defined classification on a large number of radiographic images, with the null hypothesis that there was poor agreement between observers.

Materials and Methods

Patient Identification and Selection

A retrospective cohort study was undertaken in a foot and ankle unit within a category 1 major trauma center. Identification of patients was accomplished by interrogating the departmental prospectively collected electronic database for cases of suspected or confirmed fifth metatarsal fracture from February 1, 2016, until July 31, 2021. Inclusion criteria were patients of skeletal maturity referred to our VFC with a suspected or confirmed fifth metatarsal fracture on plain radiographs of the foot, and at least 1 of the observers identifying a fracture. Radiographs would be excluded if both observers could not identify a fracture. Observers were obstructed from accessing routine demographic data.

Radiographic Analysis

Two medical students who were novice observers analyzed anteroposterior, oblique and lateral plain radiographs of the foot. They were trained using an identical presentation on fracture classifications and interpretation (images used in the presentation are included in Figures 1 and 2). They were not informed during the training process of the clinical or historical relevance of the Jones fracture or watershed region and had the resource available throughout data collection for reference. We chose these observers to attempt to account for any predetermined bias around fifth metatarsal fractures. Proximal fifth metatarsal fractures were classified according to the Lawrence and Botte classification. This splits the proximal fracture into proximal tuberosity (zone 1), distal tuberosity and metaphyseal-diaphyseal junction known as the Jones fracture (zone 2), and proximal 1.5 cm of diaphysis (zone 3) as displayed in Figure 1. ¹⁰ Zone 1 can be further subdivided according to insertion of fascia and muscles—insertion point of lateral band of plantar fascia (zone 1.1), insertion of peroneus brevis (zone 1.2), and insertion of peroneus tertius (zone 1.3)— per the Ekrol and Court-Brown classification. ⁸ Distal fifth metatarsal fractures were classified as diaphyseal shaft (DS), distal metaphysis (DM), and head as depicted in Figure 2. If a fracture crossed 2 regions, it was classified by the most proximal zone identified.

Figure 1.

Image showing fracture regions used for the proximal fifth metatarsal fractures, including the Ekrol and Court-Brown classification.

Figure 2.

Image showing classification of fifth metatarsal fracture regions.

Statistical Analysis

Categorical data are reported as frequencies and proportions. Interobserver reliability was analyzed using Cohen κ coefficient and all fracture zones were compared, not just adjacent zones. ¹⁶ The degree of observer agreement was determined using the Landis and Koch ¹⁴ description (Table 1). Statistical analyses were conducted with SPSS, version 27 (IBM corporation).

Table 1.

Landis and Koch Description of Interobserver Reliability.

Kappa Value	Strength of Agreement
<0.00	Poor
0.00-0.20	Slight
0.21-0.40	Fair
0.41-0.60	Moderate
0.61-0.80	Substantial
0.81-1.00	Almost perfect

Results

A total of 1360 patients were included with a mean age 48.1 years (SD 19.1, range 16-95); 75 patients were excluded because of missing data or no fracture being identified by both observers. Moderate agreement achieved significance for zones 1.1, 1.3, 2,3, DS, and head (Table 2). Zone 1.2 and DM fractures achieved fair agreement (κ = 0.308 and 0.381 respectively, both P < .001). Zones 1.1 (κ = .538), 1.3 (κ = .511), 2 (κ = .515), 3 (κ = .57), DS (κ = .545), and head (κ = .558) all tended toward but did not achieve substantial agreement. Slight agreement with the next proximal and distal adjacent zones were found in zones 1.1, 1.2, and 1.3. Slight agreement with the adjacent distal zone was found in zone 2 and DM fractures (Table 3). Zone 3 has slight agreement with zone 2 proximally, and there is an apparent difficulty with distal DS fractures resulting in a lot of crossover with DM, achieving a fair level of agreement (DS 312 vs 196; DM 120 vs 237; κ = .398, P < .001). As observers were allowed to review the imaging in whatever order they wanted, we were unable to assess if correlation between observers improved throughout data collection.

Table 2.

Kappa Values for All Fracture Regions Achieving Significance in “Moderate Agreement” With the Same Fracture Zone.

Zone	Kappa Value a	P Value
Zone 1.1	0.538	<.001
Zone 1.3	0.511	<.001
Zone 2	0.515	<.001
Zone 3	0.57	<.001
Diaphyseal shaft	0.545	<.001
Head	0.558	<.001

a

Kappa values achieving significance are in bold type.

Table 3.

Kappa Values for All Fracture Regions Achieving Significance in “Slight Agreement” Between the 2 Observers and Adjacent Distal Zones.

Observer 1	Observer 2	Kappa Value a	P Value
Zone 1.1	Zone 1.2	0.077	.005
Zone 1.2	Zone 1.3	0.117	<.001
Zone 1.3	Zone 2	0.06	.022
Zone 2	Zone 3	0.122	<.001
Diaphyseal metaphysis	Head	0.123	<.001

a

Kappa values achieving significance are in bold type.

Discussion

Our aim in this study was to investigate the reproducibility of classifying fracture location for all fractures of the fifth metatarsal, with the null hypothesis that there was poor agreement between observers. The volume of radiographs analyzed provides the main strength of our study, with 1360 being analyzed; the maximum level of agreement found in most cases was moderate with agreement in zone 1.2 and distal metaphyseal fractures only rated as fair. This discrepancy is mirrored in the literature by Michalski et al ¹⁸ demonstrating similar levels of agreement in 18 fifth metatarsal base fractures analyzed by 223 reviewers. The null hypothesis can therefore be rejected.

Classification of fifth metatarsal fractures serves as a guideline for treatment and may form part of management algorithms used by virtual fracture clinics. The reliability of fracture classification depends on the interobserver agreement with a low level of agreement hindering their use. Because of the scarcity of thorough randomized control trials for the fifth metatarsal anatomic area, evidence-based treatment decisions are challenging. In the studies that do exist, the reliability of classifying fifth metatarsal fractures is varied, with zones 2 and 3 often confused.^1,18,22 Therefore, this study was intended to ascertain whether fifth metatarsal zone classification is reliably interpreted by observers.

The Michalski et al article assessed interrater agreement in 223 surgeons (median experience 12 years) assessing 18 fifth metatarsal base fractures. Interrater agreement using the Fleiss κ coefficient was found to be moderate by 3-zone classifications (κ = 0.537) and for defining the region considered a Jones fracture (κ = 0.533). ¹⁸ This is in keeping with our results for the base of the fifth metatarsal. However, our study trained 2 novice raters, who had no previous knowledge on this subject, on predefined zones using a larger volume of radiographs and classified fractures of the entire fifth metatarsal, with greater detail sought with regard to zone 1, which was subclassified using the Ekrol and Court-Brown classification. ⁸ This may not have much current clinical implication, but the anatomical etiology of these fracture types may become a future area of interest. We used the Lawrence and Botte 3-zone classification system for base fractures rather than comparing opinions on 3 separate classification systems and did not assess clinical management based on zone classification because of the clinical inexperience of our raters. In addition, Packard et al report concerningly low interobserver reliability (16.67%) with 5 observers who graded 60 radiographs using the Lawrence and Botte classification, potentially demonstrating inherent shortcomings of this classification system. ²⁴ We recognize that our learners were “novice” but they will have gained experience in interpreting foot radiographs through the data collection process. A study in learning curves in pediatric ankle fractures demonstrated that in principle, orthopaedic radiographic analysis follows the Thurstone pattern of learning. ²⁸ With the volume of radiographs reviewed in our study, it is reasonable to assume a level of skill in fracture identification would have developed during data collection. Regardless, the synergy in the results of Michalski et al and our article demonstrates that when considering fractures of the fifth metatarsal base, moderate agreement is achievable in both experienced and novice raters despite differing study design.

Noori et al ²² report substantial overall interrater reliability; however, the zone 2 to 3 transition demonstrated a lower results when compared to the interface between zone 1 to 2 (κ = 0.66 vs κ = 0.83, respectively). This raises concerns where virtual management is based on zonal diagnosis, and Noori et al recommend abandonment of the Lawrence and Botte classification with a focus on development of a classification system more aligned to management and based on “avulsion” vs stress fractures. The Michalski article had substantial agreement using the simplified Polzer classification (κ = 0.705), which defines the base by metaphyseal and meta-diaphyseal regions.^18,26 Michalski’s conclusion supports the use of a simpler 2-zone system proposed by Polzer and others, and this is certainly worth considering if treatment algorithms are sought.

Any management protocol must include a failsafe for identifying or following up patients at risk of symptomatic nonunion. Recent publications have shown that virtual protocols can effectively manage patients with fifth metatarsal base fractures remotely with patient-initiated follow-up built into the advice given. ⁹ The study demonstrated good Manchester-Oxford Foot Questionnaire outcomes at 1 year. One patient had a nonunion of zone 1, and none of those requesting follow-up with injuries found in the watershed area developed nonunion. However, this study is limited by small numbers in the zone 2 and zone 3 regions and does not provide sufficient reassurance that all fifth metatarsal fractures can be discharged without appropriate safety netting.

Zone 2 and 3 fractures are often both called Jones fractures, and therefore the literature can be difficult to accurately interpret. ³ There was slight agreement demonstrated when analyzing overlap in the diagnosis of zone 2 and 3 fractures in our study (κ=0.122; Table 3), suggesting this region may be more difficult to delineate; our raters were not informed of the historical and clinical importance of the Jones fracture in the classification training to attempt to avoid bias. Michalski et al demonstrated that of the 223 American Orthopaedic Foot & Ankle Society (AOFAS) members surveyed, 84.6% considered a Jones fracture to be zone 2 and 77.9% also felt it was at the zone 2-3 junction; only 33.1% considered zone 3 to be a Jones fracture. ²² Zone 2 fractures are often recommended to be managed conservatively, and systematic review has demonstrated 77% union rates in a mean time of 11 weeks, compared with 96% in 9 weeks in operatively managed patients. ³⁰ Further systematic review by Herterich et al ¹¹ recommends treating zone 1 and 2 fractures as the same entity. Others describe the watershed area being in zone 2 and therefore contribute to high nonunion rates. ³² Operative management is often reserved for the athletic and high-demand population to allow earlier return to exercise, or those who have displaced zone 2 and 3 fractures and nonunion. ⁴ Zone 3 fractures have higher incidence of delayed and nonunion (25%-67%), as well as up to 50% refracture rate found in one review. ³ Chuckpaiwong et al ⁴ proposed disregarding this differentiation as they “[did] not find a reason to distinguish between fracture of the fifth metatarsal in these 2 locations” and proposed referring to all base fractures, excluding avulsions, as “Jones Fractures.” Seventy percent of respondents in the Michalski study managed zone 2 and 3 fractures the same regardless, and our study demonstrating some crossover in agreement may add weight to this argument. ¹⁸

Michalski’s findings related to the varied agreement in the diagnosis of a Jones fracture and where the demarcations of the metatarsal base zones are, and the literature concerning union rates between zones becomes difficult and inconsistent to scrutinize with accuracy. Conservative management of the base fractures historically can involve prolonged limitations on weightbearing status. Our study shows moderate agreement in diagnosing these regions using the Lawrence and Botte classification between our observers, with some overlaps demonstrated by the slight agreement found. Having more than one 3-zone classification system being used creates confusion, questioning whether despite the moderate agreement achieved in the papers discussed the variation in the classification of fifth metatarsal fractures allows for inconsistency; the academic diagnosis of a “Jones fracture” is ultimately less important than the identification of patients at risk of symptomatic nonunion.

In our study, diaphyseal and head fractures achieved moderate agreement and tended close to substantial, although there was near moderate agreement between the observers when considering distal DS and DM fractures. This may be due to the spiral nature of the “dancer’s fracture” of the diaphysis extending into the metaphysis. Clinically this likely has little impact, with the importance over diagnosis in the fifth metatarsal relating to base fractures. Historical registry data has demonstrated good rates of healing with conservative management ⁵ ; this is regardless of the fracture pattern, degree of shortening, or displacement.^21,31 In professional dancers, distal fractures have been shown to have no impact on long-term function regardless of surgical or conservative management. ²³ A recent study has demonstrated 3 distinct clusters of fracture patterns distal to the proximal metaphyseal-diaphyseal junction, and this may prove useful in the development of a simplified classification system. ¹²

This study has limitations to consider. First, the observers employed were inexperienced and “novice”; this was deliberate and demonstrates that reasonable levels of correlation can be found with simple training, and the comparison to recent results in similar studies involving experienced clinicians demonstrates correlation with the current literature. As the observers are not subspecialist, the results do show generalizability. We also addressed that our observers would likely surpass novice level by the end of data collection. Only 2 observers were used to classify the fractures; more observers would improve the methodological quality of this retrospective analysis. Finally, the images used to train the raters in diagnosing fracture location only accounted for the named specific classifications and therefore do not account for all methods of classifying fifth metatarsal fractures available.

Conclusion

A clear consensus on prognosis and treatment of fifth metatarsal fractures from good-quality prospective studies is lacking, especially where the location of the fracture is at the base. Several recent studies look at the management, diagnosis, and the level of agreement between large amounts of reviewers of fifth metatarsal fractures, and we believe we present the largest case series in the literature investigating the reliability of subcategorizing fifth metatarsal fractures using standardized instructions, conveying moderate agreement in most cases using novice observers. The defining of zone 1-3 fractures needs careful consideration if fracture region is to be considered in management algorithms, and there is strong recent evidence for the use of a simplified universal classification in base fractures. Algorithms should be designed to ensure crossover and possible inaccuracy of diagnosis is accounted for in the patients considered at risk of nonunion or suitable for early surgical intervention.

Supplemental Material

sj-pdf-1-fao-10.1177_24730114241278712 – Supplemental material for Is the Diagnosis of fifth Metatarsal Fracture Type Consistent? An Interobserver Reliability Study

Supplemental material, sj-pdf-1-fao-10.1177_24730114241278712 for Is the Diagnosis of fifth Metatarsal Fracture Type Consistent? An Interobserver Reliability Study by James Chapman, Shubhi Gupta, Zain Choudhary, Thomas Davies, Grace Airey MBChB and Lyndon Mason in Foot & Ankle Orthopaedics