Objective: To evaluate the reliability and responsiveness to change of an audit tool to assess adherence to evidence of effectiveness in the speech and language therapy (SLT) management of poststroke dysphagia.
Design: The tool was used to review SLT practice as part of a randomized study of different education strategies. Medical records were audited before and after delivery of the trial intervention.
Setting: Seventeen SLT departments in the north-west of England participated in the study.
Subjects: The assessment tool was used to assess the medical records of 753 patients before and 717 patients after delivery of the trial intervention across the 17 departments. A target of 10 records per department per month was sought, using systematic sampling with a random start.
Analysis: Inter- and intra-rater reliability were explored, together with the tool's internal consistency and responsiveness to change.
Results: The assessment tool had high face validity, although internal consistency was low (ra=0.37). Composite scores on the tool were however responsive to differences between SLT departments. Both inter- and intra-rater reliability ranged from ‘substantial’ to ‘near perfect’ across all items.
Conclusions: The audit tool has high face validity and measurement reliability. The use of a composite adherence score should, however, proceed with caution as internal consistency is low.
Get full access to this article
View all access options for this article.
References
1.
1 Effective Health Care. Getting evidence into practice. University of York, 1999.
2.
2 Pennington L, Roddam H, Burton C, Russell I, Godfrey C, Russell D. Promoting research use in speech and language therapy: a cluster randomized trial to compare the clinical effectiveness and cost of two training strategies. Clin Rehabil2005; 19: 387–397.
3.
3 Agency for Health Care Policy and Research. Using clinical practice guidelines to evaluate quality of care, Volume 2: Methods. Agency for Health Care Policy and Research Publication No 95-0046. US Department of Health & Human Services, Public Health Services, 1995.
4.
4 Intercollegiate Working Party. National clinical guidelines for stroke. Royal College of Physicians, 2000.
5.
5 Intercollegiate Working Party. National clinical guidelines for stroke (Update). Royal College of Physicians, 2002.
6.
6 Scottish Intercollegiate Guideline Network. Management of patients with stroke (Part 3–identification and management of dysphagia). Scottish Intercollegiate Guideline Network, 1997.
7.
7 Royal College of Speech and Language Therapists. Clinical guidelines by consensus for speech and language therapists. RCSLT, 1998.
8.
8 Cluzeau FA, Littlejohns P, Grimshaw J, Feder G, Moran SE. Development and application of a generic methodology to assess the quality of clinical guidelines. Int J Qual Health Care1999; 11: 21–28.
9.
9 Landis JR, Koch GC. The measurement of observer agreement for categorical data. Biometrics1977; 33: 159–174.
10.
10 Hutchinson A, McIntosh A, Cox S, Gilbert C. Towards efficient guidelines: how to monitor guideline use in primary care. Health Technol Assess2003; 7-7.
11.
11 Grimshaw JM, Thomas RE, MacLennan G et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess2004; 8: 1–84.
12.
12 Rethans JJ, Martin E, Metsemakers J. To what extent do clinical notes by general practitioners reflect actual medical performance? A study using simulated patients. Br J Gen Pract1994; 44: 153–156.
13.
13 Intercollegiate Working Party. National clinical guidelines for stroke, second edition. Royal College of Physicians, 2004.
14.
14 Donner A, Klar N. Design and analysis of cluster randomization trials in health research. Arnold, 2000.
15.
15 Royal College of Speech and Language Therapists. Report from the panel of the policy review forum on the issues of the management of the dysphagia patient. RCSLT, 1998.
16.
16 Perry L. Screening swallowing function of patients with acute stroke. Part one: identification, implementation and initial evaluation of a screening tool for use by nurses. J Clin Nurs2001; 10: 463–473.
