Background: The use of validated retrospective tools to assess disease activity in an observational cohort of patients would allow researchers the flexibility to analyze unique exploratory questions. Valid, reliable tools exist for assessing disease activity and flare for Systemic Lupus Erythematosus (SLE) patients. However, these tools have been designed for use in structured settings. Many of these populations under study are subject to strict inclusion, exclusion criteria and disease management protocols. The ability to apply these tools to populations not subject to such control would allow researchers to explore questions about unique populations, new treatment applications or predictors of clinical outcomes. This study sought to establish the reliability and validity of retrospective medical record abstraction of SLE disease activity and flare instruments by two rheumatologists.
Methods: From university rheumatology outpatient clinics, 22 patients were randomly selected to establish intra-rater reliability and 26 patients were selected to establish inter-rater reliability. Two rheumatologists using a Physician Global Assessment (PGA), the Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) and the Safety of Estrogens in Lupus Erythematosus<Eth>National Assessment (SELENA) flare tool retrospectively abstracted patient charts. Agreement between the tools was evaluated to assess validity.
Results: The mean patient age was 39 y; 96% were female, 54% Caucasian, 27% Hispanic, 19% Asian; median PGA was 1.4 (on a 0±3 scale) and median SLEDAI score was 4 (range 0±27). Intrarater reliability for PGA, SLEDAI and the SELENA flare tool was 0.88, 0.87 and 0.52, respectively. Inter-rater reliability for PGA, SLEDAI, and SELENA flare was 0.79, 0.75, and 0.50, respectively. To assess validity, the tools were compared against each other to assess agreement. From the parent study sample (n ‘ 54 patients), the disease activity measures, PGA and SLEDAI demonstrated adequate agreement (r ‘ 0.60). However, the SELENA flare tool demonstrated poor agreement with either PGA-defined flare or SLEDAI-defined flare (weighted kappa, 0.29 and 0.40 respectively). PGA-defined and SLEDAI-defined flare also demonstrated poor agreement (weighted kappa, 0.35).
Conclusions: These data suggest that investigators can reliably reproduce patient disease activity through retrospective chart abstraction using PGA and SLEDAI. Assessing flare is a more difficult concept. The validity of assessing flare at a specific patient-visit is poor. Retrospective assessment of patient flare risk over a specified time period is conceptually more valid and avoids difficulties assessing timing and duration of flare. As there have been no similar published prospective analyses of validity using the SELENA flare tool, it is not clear if this problem was unique to the method of retrospective chart abstraction, the nature of non-protocol study patient visits, the tool itself or a combination of all three aspects.
Liang MH , Socher SA , Larson MG , Schur PH . Reliability and validity of six systems for the clinical assessment of disease activity in systemic lupus erythematosus . Arthritis Rheum1989; 32: 1107–1118 . 2
2.
Hawker G , Gabriel S , Bombardier C et al. A reliability study of SLEDAI: a disease activity index for systemic lupus erythematosus (published erratum appears in J Rheumatol 1993 Jun; 20(6): 1091) . J Rheumatol1993; 20: 657–660 .
3.
Wluka AE , Liang MH , Partridge AJ et al. Assessment of systemic lupus erythematosus disease activity by medical record review compared with direct standardized evaluation . Arthritis Rheum1997; 40: 57–61 .
4.
Arce-Salinas A , Cardiel MH , Guzman J , Alcocer-Varela J . Validity of retrospective disease activity assessment in systemic lupus erythematosus . J Rheumatol1996; 23: 846–849 .
5.
Petri M , Hellmann D , Hochberg M . Validity and reliability of lupus activitymeasures in the routine clinic setting . J Rheumatol1992; 19: 53–59 .
6.
Petri M , Genovese M , Engle E , Hochberg M . Definition, incidence, and clinical description of flare in systemic lupus erythematosus .A prospective cohort study. Arthritis Rheum1991; 34: 937–944 .
7.
Petri M , Buyon JP , Skovorn ML . Reliability of SELENA SLEDAI and flare as clinical trial outcome measures . Arthritis Rheum1998; 41: S218-s218 (Abstract).
8.
Kalunian K . Personal communication, January 1999, Components of the SELENA visit tool.
9.
Bombardier C , Gladman DD , Urowitz MB , Caron D , Chang CH . Derivation of the SLEDAI. A disease activity index for lupus patients . The Committee on Prognosis Studies in SLE. Arthritis Rheum1992; 35: 630–640 .
10.
Fleiss J . The Design and Analysis of Clinical Experiments. In: Wiley (ed) The Design and Analysis of Clinical Experiments. 1986, Wiley: New York .
11.
Liang MH , Socher SA , Roberts WN , Esdaile JM . Measurement of systemic lupus erythematosus activity in clinical research . Arthritis Rheum1988; 31: 817–825 .
