The Calgary Kids’ Hand Rule: External Validation of a Prediction Model to Triage Pediatric Hand Fractures

Abstract

French

Background: The Calgary Kids’ Hand Rule (CKHR) is a clinical prediction rule intended to guide referral decisions for pediatric hand fractures presenting to the emergency department, identifying “complex” fractures that require surgical referral and optimizing care through better matching of patients’ needs to provider expertise. The objective of this study was to externally validate the CKHR in two different tertiary pediatric hospitals in Canada. Methods: We partnered with British Columbia Children's Hospital (BCCH) and the Children's Hospital of Eastern Ontario (CHEO) to externally validate the CKHR using data from retrospective cohorts of pediatric hand fractures (via electronic medical record and x-ray review). Model performance was evaluated at each site using sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and the C-statistic. Results: A total of 954 hand fractures were included in the analysis (524 at BCCH and 430 at CHEO. At BCCH, the CKHR had a sensitivity of 91.1% (133 predicted complex out of 146 total complex fractures), specificity of 71.4% (269 predicted simple out of 377 total simple fractures), and C-statistic of .81, 95% CI [0.78-0.84]. At CHEO, the CKHR had a sensitivity of 98.3%, specificity of 30.2%, and C-statistic of .64, 95% CI [0.61-0.67]. Conclusion: The CKHR performed well at two different tertiary care centres with high sensitivity, supporting its ability to facilitate hand fracture triage in other populations without further modification. This work should be followed by rigorous implementation analysis to determine its impact on patient care.

Keywords

clinical prediction rule external validation hand fracture pediatric

Introduction

Pediatric hand fractures are a common presentation to emergency departments (incidence of up to 624 per 100 000 patients per year).¹ In Canada, these fractures are commonly referred to pediatric plastic surgeons, even though most are stable, do not require surgery, and have excellent outcomes.² This results in many potentially unnecessary specialist referrals and contributes to increased wait times for all plastic surgery patients.

To help referring physicians separate the subset of fractures that require surgical intervention or assessment by a hand (plastic) surgeon from those that can be managed by alternative care providers, we developed the Calgary Kids’ Hand Rule (CKHR).³ The CKHR is a clinical prediction rule for pediatric hand fractures that considers six predictor variables: two based on physical exam and four based on radiographic findings. It is meant to be used by emergency room physicians or urgent care physicians to predict which fractures require management by a hand surgeon, also referred to as “complex” fractures. We define complex fractures as any fracture requiring one or more of the following: surgery, closed reduction, or four or more visits with a hand surgeon (Figure 1). The CKHR was derived and internally validated in a retrospective cohort of 1045 pediatric hand fracture patients in Calgary, Alberta and demonstrated a sensitivity of 96.4%, specificity of 45.5%, and C-statistic of .88.³ A following prospective validation at this site, with emergency physicians applying the CKHR in their practice, also showed good performance, with a sensitivity of 84%, specificity of 71%, and C-statistic of .77.⁴

Figure 1.

The Calgary Kids’ Hand Rule predictors.

External validation of clinical decision rules is an important step before introduction into clinical use,^5–8 particularly in pediatric settings where prediction models have been shown to be of lower quality than prediction models for adults.⁹ Therefore, the aim of this study was to externally validate the CKHR in two new cohorts of children with acute, non-reduced hand fractures, at two different tertiary pediatric hospitals in Canada: British Columbia Children's Hospital (BCCH) and the Children's Hospital of Eastern Ontario (CHEO).

Methods

Study Design and Setting

The external validation was comprised of two separate retrospective cohort studies. The first was performed at the BCCH in Vancouver, British Columbia from using patient electronic medical records (EMRs) dated November 2016 to January 2020 and the second was performed at the CHEO in Ottawa, Ontario using EMRs from April 2019 to December 2019.

Study Population

Consecutive pediatric (age 0-17 years) hand fractures seen at either BCCH or CHEO pediatric plastic surgery clinics during the study periods were reviewed for inclusion and exclusion criteria. All hand fractures involving phalanges and metacarpals of patients presenting to the BCCH or CHEO pediatric plastic surgery clinic were considered for inclusion. Fractures were excluded if they did not present acutely (over 3 weeks post-injury), if they occurred in patients over 17 years of age at time of injury, if x-ray images were unavailable, if soft tissue injury occurred without a concomitant fracture (ie, joint dislocation without fracture) or if carpal fractures occurred without concomitant hand fractures.

Predictor Assessment

Predictors of complex fractures identified in our previous study were collected as follows: the two physical exam predictors (malrotation and open fracture) were collected as dichotomous variables and noted as present if documented by either the referring physician or hand surgeon at time of patient assessment. Patient radiographs were assessed by one or more members of the research team at each of the study centres. Prior to the study these research team members met with the original derivation team to review interpretation of the four radiographic predictors (displacement, angulation, condylar involvement, subluxation or dislocation) as previously described in our derivation and internal validation study (Figure 1).³ Displacement and angulation were collected as continuous variables. Condylar involvement and subluxation or dislocation were collected as dichotomous variables.

Outcome Measure

“Complex” fractures requiring management by a hand surgeon were defined as those fractures requiring surgery, closed reduction, or four or more visits with a hand surgeon (Figure 2). The outcome measure was assessed at 3 months from date of injury. At BCCH, predictor data was extracted from charts and x-rays by a medical student under the supervision of a senior plastic surgery resident. At CHEO, data from charts was extracted by a medical student but all X-rays were interpreted by a pediatric plastic surgeon. All reviewers were instructed on the intended X-ray interpretation method prior to data collection. In addition, at the beginning of the data collection period, an initial sample of interpretations from each site was reviewed by two surgeons at ACH to ensure agreement.

Figure 2.

Definition of complex hand fracture requiring management by a hand surgeon.

Power Calculation

We estimated that a sample size of 500 children with acute hand fractures would be required to reach 100 complex fracture outcomes at each external validation site. This would provide sufficient power to generate precise estimates (using 95% confidence intervals and an alpha of .05) based on an estimated 35% incidence of complex fractures.¹⁰

Statistical Analysis

Outcomes were collected at the patient level (closed reduction, operation, and number of appointments) while predictors were assessed at the fracture level. We examined demographic and clinical characteristics using descriptive statistics for each centre. Univariate analysis was performed with a 2-sided t-test for continuous variables and the Pearson’s Chi-squared test for categorical variables. All analyses were performed using STATA Version 17 with an alpha ≤ .05.¹¹

Next, we converted each predictor variable into a dichotomous variable, as per the original derivation of the CKHR, with a threshold of greater than 5 degrees for the variable “angulation” and a threshold of greater than 2 millimeters for the variable “displacement”. We then evaluated the CKHR performance at each centre by calculating sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and C-statistic with 95% confidence intervals. As this tool is to be used for screening/triage, sensitivity is especially important because high sensitivity indicates a low false negative rate (ie, fractures predicted by the CKHR as simple but were complex according to actual outcome). These are fractures that hand surgeons do not want to miss. Consequently, careful review and a descriptive analysis of all false negatives was conducted for each study site. The C-statistic simultaneously considers the sensitivity and specificity of a clinical prediction rule, quantifying the relationship between the true positive and false positive rate. A C-statistic of 1.0 indicates 100% true positive rate and 0% false positive rate. In contrast, a C-statistic of .5 indicates equal true positive and false positive rates (no better than random chance). Likelihood ratios also simultaneously consider sensitivity and specificity, and a higher positive likelihood ratio indicates strong evidence to “rule in” a condition. Conversely, a low negative likelihood ratio (close to 0) indicates strong evidence to “rule out” a condition. Thus, an ideal clinical prediction rule possesses high sensitivity and specificity resulting in a high positive likelihood ratio (>2) with a low negative likelihood ratio (between 0 and .5) and a C-statistic close to 1.0.

Ethics Approval and Reporting

Ethics approval was obtained for each study site (University of Calgary Conjoint Health Research Ethics Board REB20-0256, University of British Columbia Children & Women's Research Ethics Board H20-00563, Children's Hospital of Eastern Ontario Research Ethics Board 20/16X). A data sharing agreement was also completed for each external site. This study adheres to the reporting guidelines as outlined by the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement (Supplementary File 1).¹²

Results

At BCCH, 500 patients were initially included; one patient had missing data; therefore 499 patients with 524 hand fractures were available for complete case analysis. At CHEO, 500 patients were initially included; 70 patients had missing data; therefore 430 patients with 430 hand fractures were available for complete case analysis.

Table 1 shows the clinical characteristics, including sex, age, and CKHR predictors of the included hand fractures at both BCCH and CHEO, as well as the original derivation cohort from the Alberta Children's Hospital (ACH).³ Two-thirds of all patients were male. Hand fractures from the CHEO site reported a higher proportion of angulation greater than 5 degrees seen on any radiographic view. The hand fractures collected at BCCH had a lower proportion of condylar involvement and displacement.

Table 1.

Clinical Characteristics of Hand Fractures From all Sites.

		British Columbia (BCCH)	Ontario (CHEO)	Alberta (ACH)
Sample	Hand fractures	524	430	1170
Sample	Patients	499	430	1045
Patient demographics	Male sex	69.7% (348)	66.1% (284)	70.3% (735)
	Median age (years)	11	12	13
	Age range (years, inter-quartile)	9-14	10-12	10-14
Calgary Kids’ Hand Rule Predictors	Angulation >5 degrees, seen on any view	26.6% (139)	66.3% (285)	37.5% (439)
	Condylar involvement	2.7% (14)	10.9% (47)	6.1% (71)
	Displacement >2 mm, seen on any view	27.0% (141)	50.7% (218)	57.6% (674)
	Malrotation	3.4% (18)	5.8% (25)	8.7% (102)
	Open fracture	1.9% (10)	5.4% (23)	4.0% (47)
	Subluxation or Dislocation	0.8% (42)	1.9% (8)	2.8% (33)

Table 2 reports the management and outcomes of the included hand fractures at BCCH and CHEO, as well as the original derivation cohort at ACH.³ Both external validation sites had similar complex fracture rates (BCCH 27.9% and CHEO 27.7%), which was slightly less than the ACH rate (35.6%). The need for closed reduction was the most common reason a fracture qualified as complex (25.1% at BCCH and 23% at CHEO). At BCCH, 5.5% of fractures required close follow-up by a hand surgeon (≥4 visits) whereas 15.4% of fractures required close follow-up at CHEO. Surgical management was required for only 2.3% of fractures at BCCH and 5.6% of fractures at CHEO.

Table 2.

Management and Outcomes of Hand Fractures at Each Site.

			British Columbia% (N)	Ontario% (N)	Alberta% (N)
All hand fractures			524	430	1170
Simple fractures			72.1% (378)	72.3% (311)	64.4% (753)
Complex fractures			27.9% (146)	27.7% (119)	35.6% (417)
	Closed reduction		23% (121)	18.4% (79)	25.1% (294)
	>4 appointments^a		5.5% (29)	15.4% (66)	8.7% (102)
	Surgery		2.3% (12)	5.6% (24)	9.8% (115)
		Open reduction internal fixation	0.2% (1)	1.9% (8)	4.7% (55)
		Closed reduction K-wire fixation	1.9% (10)	3.3% (14)	4.0% (47)
		Other	0.2% (1)	0.4% (2)	1.1% (13)

Defined as 4 or more visits with the hand surgeon.

The 2-by-2 tables showing predicted fracture complexity versus actual complexity are reported in Tables 3a and 3b. Table 4 summarizes the performance of the CKHR for detection of complex fractures at each centre, where sensitivity is the proportion of predicted complex fractures out of the total number of complex fractures, and specificity is the proportion of predicted simple fractures out of the total number of simple fractures. In the BCCH cohort, the CKHR achieved a sensitivity of 91.1% (133 predicted complex out of 146 total complex fractures), specificity of 71.4% (269 predicted simple out of 377 total simple fractures), positive likelihood ratio of 3.2, negative likelihood ratio of 0.12, and C-statistic of .81, 95% CI [0.78-0.84]. At CHEO, the CKHR had a sensitivity of 98.3% (117 predicted complex our of 119 total complex fractures), specificity of 30.2% (94 predicted simple out of 311 total simple fractures), positive likelihood ratio of 1.4, negative likelihood ratio of 0.056, and C-statistic of .64, 95% CI [0.61-0.67].

Table 3.

(a) BC Children's Hospital 2 × 2 Table for the Calgary Kids’ Hand Rule.

	Prediction simple	Prediction complex	Total
Outcome simple	269	108	377
Outcome complex	13	133	146
Total	282	241	523
Table 3. (b) Children's Hospital of Eastern Ontario 2 × 2 table for the Calgary Kids’ Hand Rule.
	Prediction simple	Prediction complex	Total
Outcome simple	94	217	311
Outcome complex	2	117	119
Total	96	334	430

Table 3a and 3b could be combined as you have done but the titles of each data set ("BC Children's Hospital 2x2 Table for the Calgary Kids' Hand Rule" and "Children's Hospital of Eastern Ontario 2x2 Table for the Calgary Kids' Hand Rule"), need to be clearer. Alternatively just make them 2 different tables.

Table 4.

Calgary Kids’ Hand Rule Performance at Each Site.

	British Columbia (BCCH)	Ontario (CHEO)	Alberta (ACH)
Sensitivity	91.1%	98.3%	96.4%
Specificity	71.4%	30.2%	45.5%
LR+	3.2	1.4	1.8
LR−	0.12	0.056	0.079
C-statistic	0.81 [0.78-0.84]	0.64 [0.61-0.67]	0.88 [0.85-0.91]

C-statistic, concordance statistic; the probability that predicting the outcome is better than chance (0.50 = no better, 1.0 = perfect). LR+, positive likelihood ratio; the larger the LR+, the stronger the argument that the disease is present. LR−, negative likelihood ratio; as the LR− approaches zero, the stronger the argument that no disease is present.

There were two false negatives at CHEO (ie, two fractures predicted as simple but required complex management) and 13 false negatives at BCCH. Details of these fractures are reported in Table 5. None of the false negatives required surgery. Seven of the 15 false negatives required closed reduction, of which six were performed by the emergency physician and one was performed by the plastic surgeon. Five of the false negatives were patients with multiple fractures, and three of these patients had a combination of predicted simple and predicted complex fractures.

Table 5.

False Negatives for Each Site (ie, Fractures That Were Predicted Simple but Required Complex Management).

Site	False negative case	Patient age (years), Sex	Fracture description	Complex management required	Comments
BCCH	1	8, M	Transverse non-displaced fracture of first metacarpal head	Appointments (4)	Splint uncomfortable, needed fourth appointment to adjust
	2	9, M	Non-displaced SH2 fracture of fifth proximal phalanx	Closed reduction (Emergency Department)	Initial reviewer assessed post-reduction film
	3	10, M	SH1 fracture of first metacarpal	Appointments (4)	Patient did not like cast position and returned to clinic once for new cast
	4	10, F	Non-displaced fracture of fifth proximal phalanx	Appointments (4)	Persistent swelling thus needed an extra X-ray to reassure
	5	14, M	Non-displaced SH2 fracture of second metacarpal	Closed reduction (Emergency Department)	Initial reviewer assessed post-reduction film
	6	12, M	SH2 buckle fracture of first metacarpal	Appointments (6)	Extra splinting and follow-up required due to reinjury involving ligament
	7, 8	12, F	SH2 buckle fracture of second proximal phalanx	Appointments (5)	Multiple fractures, both simple
	7, 8	12, F	SH2 buckle fracture of third proximal phalanx	Appointments (5)	Multiple fractures, both simple
	9	14, M	SH2 buckle fracture of fifth proximal phalanx	Appointments (4)	Extra appointment to remove cast material
	10	11, M	SH2 fracture of third distal phalanx	Closed reduction (Emergency Department)	Multiple fractures – other fracture had angulation and both had closed reduction
	11	14, F	Non-displaced oblique fracture base of third proximal phalanx	Closed reduction (Emergency Department)	Multiple fractures: other fracture was displaced
	12	13, M	Oblique and comminuted fracture of second metacarpal	Closed Reduction (Hand Surgeon)Appointments (4)	Multiple fractures: third and fourth metacarpals were also fractured with angulation and displacement
	13	15, M	Avulsion fracture of first proximal phalanx	Closed reduction (Emergency Department)	Initial reviewer assessed post-reduction film
CHEO	1	9, F	Transverse fracture through base of fifth metacarpal	Appointments (4)	Extra appointment due to repeat injury to same site
CHEO	2	11, M	SH2 fracture of first distal phalanx	Closed reduction (Emergency Department)	Initial reviewer assessed post-reduction film

Discussion

We have conducted a geographic external validation of the CKHR, a clinical prediction model for pediatric hand fracture triage. The CKHR performed well at two different centres across Canada, with high sensitivity, suggesting that no further refinement or adjustment to the rule is necessary. In this study, sensitivity indicates the probability that the CKHR predicts a hand fracture as complex among those patients with a complex fracture. In the literature, it has been argued that few physicians would use a risk index with a sensitivity less than 95% because they would not tolerate missing more than 5% of outcomes.^6,9 The Ottawa Ankle Rules and the Ottawa Knee Rules, both well validated and clinically implemented fracture prediction models, calibrated their models for a sensitivity of 100%.^13,14

External validation is important to assess the performance of a clinical prediction model in a setting separate from the original derivation setting. Differences in performance between settings can be attributed to numerous factors, including model overfitting, the exclusion of important predictors that may improve model performance, differences in how the predictors are interpreted and measured across jurisdictions, differences in the patient demographics, and differences in the prevalence of disease.¹⁵ The recruitment period was significantly shorter at CHEO to obtain a similar number of patients. We speculate that this may be due to differences in local referral practices and increased availability of alternative sites for pediatric hand fracture management in B.C. Collaborators at BCCH present further detail regarding the epidemiology, management, outcomes, and resource utilization of pediatric hand fractures studies at their site in a recent article.¹⁶

Considering potential utility of the CKHR, sensitivity is paramount. Without good sensitivity large numbers of complex fractures would be missed, resulting in delayed or missed treatment and poor patient outcomes. It is essential to understand the occurrence of false negatives to inform modifications that maximize sensitivity or at least minimize the consequences of inaccurate prediction. Secondly, specificity is also very important. At BCCH, the rule accurately predicted 269 hand fractures as simple (71.4%). At CHEO, specificity was lower and 94 simple fractures were accurately predicted (30.2%). If these true negatives were redirected to care providers other than a hand surgeon, hand clinic capacity to see other patients would potentially increase, thus reducing wait times. The sensitivity and specificity of a prediction model are often inversely related. Increased sensitivity often results in reduced specificity due to over-prediction. This is observed in the CHEO cohort with concurrence of lower specificity (30.2% vs 71.4%) and fewer false negatives compared to BCCH (1.7% vs 8.9%). In this instance, we speculate that differences between these sites are attributable to differences in local referral patterns (ie, differences in the underlying characteristics of fractures being referred) and variance in data collection (ie, inter-rater reliability).

There are three main stages in the development of clinical prediction models: derivation, external validation, and impact analysis.⁸ Few prediction models are externally validated, especially within the pediatric population. A systematic review conducted by Maguire et al in 2011 found that 76% were derived but not validated, 17% prospectively validated in one setting, and 8% were validated in more than one setting.⁹ Successful external validation allows for the generalizability of a prediction model beyond the original patient demographics and study constraints of the derivation sample. Geographic validation studies, such as this study, are likely to have different patient characteristics, as well as variable interpretation and measurements of predictor and outcomes variables when compared to the original derivation study. Greater differences between derivation and validation samples provides stronger evidence that a clinical prediction model can be generalized to new patients.¹⁷ Successful external validation also allows for small-scale, structured implementation in a clinical setting to assess real-world efficacy (ie, implementation analysis).

Ultimately this prediction rule will be integrated into a clinical care pathway where it will be used to discriminate between those patients that could be safely managed without a hand surgeon (ie, simple fractures) and those patients that could benefit from specialized care by a hand surgeon (ie, complex fractures). The exact logistics and implementation methods of such a care pathway will need to be adapted to local practice patterns, resources, and clinical culture. It is important to note that despite strong performance with high sensitivity, some health care practitioners will be reluctant to implement a prediction model without 100% sensitivity. Thus, barriers and facilitators to adoption need to be explored before clinical implementation to facilitate uptake.

This study should also be interpreted considering its limitations. First, data was collected using retrospective chart review of hand fractures referred to hand surgeons. Hand fractures not referred to hand surgeons at either BCCH or CHEO were therefore not included in this study, resulting in a potential over-estimation of the true prevalence of complex fractures seen in the community. Second, both sites in this study were tertiary pediatric centres in large Canadian cities. While it is reassuring that the CKHR performed well at these two geographically distinct sites, its application to new populations that are substantially different than the ACH, BCCH, or CHEO should be cautiously considered. Moreover, clinical prediction models are ultimately user-dependent; inter-rater reliability may differ based on several variables including clinical setting and years of practice, personal experiences or biases, and understanding of the prediction rule itself. In this validation study, data extraction was performed by individuals with varying levels of knowledge and expertise. At BCCH, data extraction was performed by a medical student under the supervision of a plastic surgery resident. At CHEO, data extraction from x-rays was performed by a plastic surgeon. In real-world applications of the CKHR, the rule is intended to be applied primarily by emergency physicians who have variable knowledge of hand anatomy and hand fractures that is generally less than that of plastic surgeons. Therefore, attention must be paid to maximize reliability through consistent and ongoing education and engagement with end-users.^13,14

In summary, we have conducted a geographic external validation of the CKHR, a clinical prediction model for pediatric hand fracture triage. This rule performed well at two different centres, with high sensitivity, suggesting that this rule may be useful to aid in hand fracture triage without the need for modification. However, this successful external validation should be followed by rigorous implementation analysis to determine the impact on patient care. Future directions for the CKHR will be focused on translational medicine and guided by the Knowledge to Action cycle.¹⁸ Our team will assess barriers and facilitators to its use in clinical practice and evaluate inter-rater reliability among users in different settings. Next, we will tailor and implement a clinical care pathway for pediatric hand fractures that centers around this risk prediction tool. This care pathway will also be pilot tested in a local context where it can be monitored and evaluated before scale and spread to locations across Canada.

Supplemental Material

sj-docx-1-psg-10.1177_22925503231190933 - Supplemental material for The Calgary Kids' Hand Rule: External Validation of a Prediction Model to Triage Pediatric Hand Fractures

Supplemental material, sj-docx-1-psg-10.1177_22925503231190933 for The Calgary Kids' Hand Rule: External Validation of a Prediction Model to Triage Pediatric Hand Fractures by Rebecca L. Hartley, Paul Ronksley, A. Robertson Harrop, Altay Baykan, Sabrina Wei, Diana Forbes, Jugpal Arneja, Toros Canturk, Kevin Cheung, and Frankie O. G. Fraulin, in Plastic Surgery

Footnotes

Acknowledgements

The authors would like to thank Dr Donald McPhalen for acting as a content expert and for helping to refine the original Calgary Kids’ Hand Rule; Marija Bucevska and Patrick Sachsalber for assisting with study coordination and data collection at the external sites for the present study.

Author Contributions

Concept: Hartley, Ronksley, Harrop, Fraulin. Ethics and Protocol Development: Hartley, Baykan, Fraulin. Data Collection: Wei, Forbes, Arneja, Canturk, Cheung. Data Analysis: Baykan, Hartley, Ronksley, Harrop, Fraulin. Manuscript Writing: Hartley, Baykan, Fraulin. Manuscript Revision: Hartley, Ronksley, Harrop, Baykan, Fraulin.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Approval

IRB Approval Statement: This study was approved by the University of Calgary Conjoint Health and Research Ethics Board (REB20-0256).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Sharpest Knife Live Grant Competition, Funding Department of Surgery, University of Calgary. Dr. Jugpal Arneja is a recipient of a UBC Surgery Clinical Faculty Research Support Award.

Helsinki Statement

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5).

ORCID iDs

Rebecca L. Hartley, MD, MSc, FRCSC

Jugpal Arneja, MD, MBA, FAAP, FACS, FRCSC

Kevin Cheung, MD, MSc, FRCSC

Frankie O. G. Fraulin, MD, FRCSC

Supplemental Material

Supplemental material for this article is available online.

References

Landin

. Fracture patterns in children. Analysis of 8,682 fractures with special reference to incidence, etiology and secular changes in a Swedish urban population 1950-1979. Acta Orthop Scand Suppl. 1983;202:1-109.

Nellans

Chung

. Pediatric hand fractures. Hand Clin. 2013;29(4):569-578. doi: https://doi.org/10.1016/j.hcl.2013.08.009. PMID: 24209954; PMCID: PMC4153349.

Hartley

Fraulin

FOG

Harrop

Faris

Wick

Ronksley

. Derivation and internal validation of a prediction model for pediatric hand fracture triage. Plast Reconstr Surg—Glob Open. 2021;9(4):e3543. doi: https://doi.org/10.1097/GOX.0000000000003543.

Baykan

Hartley

Ronksley

Harrop

Fraulin

FOG

. Prospective validation of the Calgary Kids’ Hand Rule: a clinical prediction rule for pediatric hand fracture triage. Plast Surg. 2022. doi:https://doi.org/10.1177/22925503221101939

Laupacis

Sekar

Stiell

. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA. 1997;277(6):488-494.

Stiell

Wells

. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med. 1999;33(4):437-447.

Wasson

Sox

Neff

, et al. Clinical prediction rules: applications and methodological standards. N Engl J Med. 1985;313(13):793-799.

Charlson

Ales

Simon

, et al. Why predictive indexes perform less well in validation studies. Arch Intern Med. 1987;147(12):2155-2161.

Maguire

, Kulik, DM, Laupacis, A, Kuppermann, N, Uleryk, EM, Parkin, PC. Clinical prediction rules for children: a systematic review. Pediatrics. 2011;128(3):e666-e677.

10.

Vergouwe

Steyerberg

Eijkemans

MJC

Habbema

JDF

. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005;58(5):475-483.

11.

Stata Statistical Software: Release 17. StataCorp LLC. StataCorp; 2019.

12.

Collins

Reitsma

Altman

Moons

. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Br Med J. 2015;350:g7594.

13.

Stiell

, Greenberg GH, McKnight RD, Nair RC, McDowell I, Worthington JR.

A study to develop clinical decision rules for the use of radiography in acute ankle injuries. Ann Emerg Med. 1992;21(4):384-390.

14.

Stiell

, Greenberg GH, Wells GA, et al. Prospective validation of a decision rule for the use of radiography in acute knee injuries. JAMA. 1996;275(8):611-615.

15.

Cowley

Farewell

Maguire

Kemp

. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn Progn Res. 2019;3:16. Published 2019 Aug 22. doi:https://doi.org/10.1186/s41512-019-0060-y

16.

Wei

Forbes

Hartley

, et al.

Pediatric hand fracture outcomes: how often do we need to operate?

Plast Surg. 2022. doi:https://doi.org/10.1177/22925503221085076

17.

Bleeker

Moll

Steyerberg

, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56(9):826-832.

18.

Straus

Tetroe

Graham

. Defining knowledge translation. CMAJ. 2009;181(3-4):165-168.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.84 MB