We used the Dutch–Flemish version of the USA PROMIS adult V1.0 item bank for Anxiety as input for developing a computerized adaptive test (CAT) to measure the entire latent anxiety continuum. First, psychometric analysis of a combined clinical and general population sample (N = 2,010) showed that the 29-item bank has psychometric properties that are required for a CAT administration. Second, a post hoc CAT simulation showed efficient and highly precise measurement, with an average number of 8.64 items for the clinical sample, and 9.48 items for the general population sample. Furthermore, the accuracy of our CAT version was highly similar to that of the full item bank administration, both in final score estimates and in distinguishing clinical subjects from persons without a mental health disorder. We discuss the future directions and limitations of CAT development with the Dutch–Flemish version of the PROMIS Anxiety item bank.
AgrestiA.KateriM. (2011). Categorical data analysis. In International encyclopedia of statistical science (pp. 206-208). Berlin, Germany: Springer.
2.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
3.
BabcockB.WeissD. J. (2013). Termination criteria in computerized adaptive tests: Do variable-length CATs provide efficient and effective measurement?Journal of Computerized Adaptive Testing, 1, 1-18.
4.
BaxterA. J.ScottK. M.VosT.WhitefordH. A. (2013). Global prevalence of anxiety disorders: A systematic review and meta-regression. Psychological Medicine, 43, 897-910.
5.
BernsteinI. H.NunnallyJ. C. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
6.
BollenK. (1989). Structural equations with latent variables. New York, NY: John Wiley.
7.
BrouwerD.MeijerR. R.ZevalkinkJ. (2013). Measuring individual significant change on the Beck Depression Inventory-II through IRT-based statistics. Psychotherapy Research, 23, 489-501.
8.
CarlierI. V. E.MeuldijkD.van VlietI.van FenemaE.van der WeeN.ZitmanF. (2012). Routine Outcome Monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluation in Clinical Practice, 18, 104-110.
9.
CellaD.RileyW.StoneA.RothrockN.ReeveB.YountS.. . . HaysR. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63, 1179-1194.
10.
CellaD.YountS.RothrockN.GershonR.CookK.ReeveB.. . . RoseM. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl. 1), S3-S11.
11.
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1-29.
ChoiS. W.GibbonsL. E.CraneP. K. (2011). Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 39, 1-30.
14.
ChoiS. W.GradyM. W.DoddB. G. (2011). A new stopping rule for computerized adaptive testing. Educational and Psychological Measurement, 71, 37-53.
15.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
16.
CraneP. K.GibbonsL. E.JolleyL.van BelleG. (2006). Differential item functioning analysis with ordinal logistic regression techniques. Medical Care, 44, S115-S123.
17.
CrinsM. H.RoordaL. D.SmitsN.de VetH. C.WesthovensR.CellaD.. . . TerweeC. B. (2015). Calibration and validation of the Dutch-Flemish PROMIS Pain Interference item bank in patients with chronic pain. PLoS ONE, 10, e0134094. doi:10.1371/journal.pone.0134094
18.
CrinsM. H.RoordaL. D.SmitsN.VetH. C. W.WesthovensR.CellaD.. . . TerweeC. B. (2016). Calibration of the Dutch-Flemish PROMIS Pain Behavior item bank in patients with chronic pain. European Journal of Pain, 20, 284-296.
19.
de BeursE.den Hollander-GijsmanM.van RoodY.van der WeeN.GiltayE.van NoordenM.. . . ZitmanF. G. (2011). Routine outcome monitoring in the Netherlands: Practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clinical Psychology & Psychotherapy, 18, 1-12.
20.
de GraafR.ten HaveM.van GoolC.van DorsselaerS. (2012). Prevalence of mental disorders, and trends from 1996 to 2009: Results from the Netherlands Mental Health Survey and Incidence Study-2. Social Psychiatry and Psychiatric Epidemiology, 47, 203-213.
21.
EmbretsonS.ReiseS. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
22.
FlensG.SmitsN.CarlierI.van HemertA. M.de BeursE. (2016). Simulating computer adaptive testing with the Mood and Anxiety Symptom Questionnaire. Psychological Assessment, 28, 953-962.
23.
FlensG.SmitsN.TerweeC. B.DekkerJ.HuijbrechtsI.de BeursE. (2017). Development of a computer adaptive test for depression based on the Dutch-Flemish version of the PROMIS item bank. Evaluation & the Health Professions, 40, 79-105.
24.
FliegeH.BeckerJ.WalterO. B.BjornerJ. B.KlappB. F.RoseM. (2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14, 2277-2291.
25.
FokkemaM.SmitsN.KeldermanH.CuijpersP. (2013). Response shifts in mental health interventions: An illustration of longitudinal measurement invariance. Psychological Assessment, 25, 520-531.
26.
FriedE. I.van BorkuloC. D.EpskampS.SchoeversR. A.TuerlinckxF.BorsboomD. (2016). Measuring depression over time . . . or not? Lack of unidimensionality and longitudinal measurement invariance in four common rating scales of depression. Psychological Assessment, 28, 1354-1367.
27.
FriesJ.RoseM.KrishnanE. (2011). The PROMIS of better outcome assessment: Responsiveness, floor and ceiling effects, and Internet administration. Journal of Rheumatology, 38, 1759-1764.
28.
FriesJ. F.KrishnanE.RoseM.LingalaB.BruceB. (2011). Improved responsiveness and reduced sample size requirements of PROMIS physical function scales with item response theory. Arthritis Research and Therapy, 13, R147. doi:10.1186/ar3461
29.
GreenB. F.BockR. D.HumphreysL. G.LinnR. L.ReckaseM. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21, 347-360.
30.
HambletonR. K. (1988). Principles and selected applications of item response theory (3rd ed.). New York, NY: American Council on Education.
31.
HastieT.TibshiraniR.FriedmanJ. H. (2001). The elements of statistical learning: Data mining, inference, and prediction. New York, NY: Springer.
32.
HavermanL.GrootenhuisM. A.RaatH.van RossumM. A.van Dulmen-den BroederE.HoppenbrouwersK.. . . TerweeC. B. (2016). Dutch-Flemish translation of nine pediatric item banks from the Patient-Reported Outcomes Measurement Information System (PROMIS)®. Quality of Life Research, 25, 761-765.
33.
JakobT.NaglM.GrammL.HeyduckK.FarinE.GlattackerM. (2015). Psychometric properties of a German translation of the PROMIS® Depression item bank. Evaluation & the Health Professions, 40, 106-120.
34.
KimS.MosesT.YooH. H. (2015). Effectiveness of item response theory (IRT) proficiency estimation methods under adaptive multistage testing. ETS Research Report Series, 2015, 1-19.
35.
KocaleventR. D.RoseM.BeckerJ.WalterO. B.FliegeH.BjornerJ. B.. . . KlappB. F. (2009). An evaluation of patient-reported outcomes found computerized adaptive testing was efficient in assessing stress perception. Journal of Clinical Epidemiology, 62, 278-287.
36.
MagasiS.RyanG.RevickiD.LenderkingW.HaysR. D.BrodM.. . . CellaD. (2012). Content validity of patient-reported outcome measures: Perspectives from a PROMIS meeting. Quality of Life Research, 21, 739-746.
37.
McDonaldR. P. (1999). Test theory: A unified treatment. Mahwah, NJ: LEA.
38.
MokkenR. J. (1971). A theory and procedure of scale analysis. The Hague, Netherlands: Mouton.
39.
MokkinkL. B.TerweeC. B.PatrickD. L.AlonsoJ.StratfordP. W.KnolD. L.. . . de VetH. C. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63, 737-745.
40.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50-64.
41.
OrlandoM.ThissenD. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27, 289-298.
42.
PazS. H.SpritzerK. L.MoralesL. S.HaysR. D. (2013). Evaluation of the patient-reported outcomes information system (PROMIS®) Spanish-language physical functioning items. Quality of Life Research, 22, 1819-1830.
43.
PilkonisP. A.ChoiS. W.ReiseS. P.StoverA. M.RileyW. T.CellaD. (2011). Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, anxiety, and anger. Assessment, 18, 263-283.
44.
PilkonisP. A.YuL.DoddsN. E.JohnstonK. L.MaihoeferC. C.LawrenceS. M. (2014). Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS®) in a three-month observational study. Journal of Psychiatric Research, 56, 112-119.
45.
R Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
46.
ReeveB. B.HaysR. D.BjornerJ. B.CookK. F.CraneP. K.TeresiJ. A.. . . LiuH. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45, S22-S31.
47.
ReiseS. P.WallerN. G. (2009). Item response theory and clinical measurement. Review of Clinical Psychology, 5, 27-48.
48.
ReiseS. P.YuJ. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27, 133-144.
49.
RevelleW. (2013). psych: Procedures for personality and psychological research. Evanston, IL: Northwestern University.
50.
RiceM. E.HarrisG. T. (2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29, 615-620.
51.
RosseelY. (2012). lavaan: An R Package for structural equation modeling. Journal of Statistical Software, 48, 1-36.
52.
SamejimaF. (1969). Estimation of latent ability using a pattern of graded responses. Psychometrika Monograph Supplement, 17, 1-100.
53.
SchaletB. D.PilkonisP. A.YuL.DoddsN.JohnstonK. L.YountS.. . . .CellaD. (2016). Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. Journal of Clinical Epidemiology, 73, 119-127.
54.
SheehanD. V.LecrubierY.SheehanK. H.AmorimP.JanavsJ.WeillerE.. . . DunbarG. C. (1998). The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59, 22-33.
55.
SmitsN. (2015). On the effect of adding clinical samples to validation studies of patient-reported outcome item banks: A simulation study. Quality of Life Research, 25, 1635-1644.
56.
SmitsN.CuijpersP.van StratenA. (2011). Applying Computerized Adaptive Testing to the CES-D scale: A simulation study. Psychiatry Research, 188, 147-155.
57.
SmitsN.ZitmanF. G.CuijpersP.den Hollander-GijsmanM. E.CarlierI. V. (2012). A proof of principle for using adaptive testing in Routine Outcome Monitoring: The efficiency of the Mood and Anxiety Symptoms Questionnaire-Anhedonic Depression CAT. BMC Medical Research Methodology, 12, 4. doi:10.1186/1471-2288-12-4
58.
TerweeC. B.RoordaL. D.de VetH. C. W.DekkerJ.WesthovensR.van LeeuwenJ.. . . BoersM. (2014). Dutch-Flemish translation of 17 item banks from the Patient-Reported Outcomes Measurement Information System (PROMIS). Quality of Life Research, 23, 1733-1741.
van der ArkL. A. (2007). Mokken scale analysis in R. Journal of Statistical Software, 20, 1-19.
61.
van WidenfeltB. M.TreffersP. D.de BeursE.SiebelinkB. M.KoudijsE. (2005). Translation and cross-cultural adaptation of assessment instruments used in psychological research with children and families. Clinical Child and Family Psychology Review, 8, 135-147.
62.
VilagutG.ForeroC. G.AdroherN. D.OlariuE.CellaD.AlonsoJ. (2015). Testing the PROMIS® Depression measures for monitoring depression in a clinical sample outside the US. Journal of Psychiatric Research, 68, 140-150.
63.
VoshaarM. A. O.PeterM.GlasC. A.VonkemanH. E.TaalE.KrishnanE.. . . van de LaarM. A. (2014). Calibration of the PROMIS physical function item bank in Dutch patients with rheumatoid arthritis. PLoS ONE, 9, e92367. doi:10.1371/journal.pone.0092367
64.
WahlI.RutsohnJ.CellaD.LöweB.RoseM.BrählerE.. . . SchaletB. (2015). Does anxiety mean the same in English and German language? Evaluation of the psychometric equivalence of the PROMIS® Anxiety item bank and its German translation. Journal of Psychosomatic Research, 78, 629-630.
65.
WainerH.DoransN.EignorD.FlaugherR.GreenB. F.MislevyR. J.SteinbergL. (2001). Computerized adaptive testing: A primer. Quality Life Research, 10, 733-734.
66.
WangT.VispoelW. P. (1998). Properties of ability estimation methods in computerized adaptive testing. Journal of Educational Measurement, 35, 109-135.
67.
YenW. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.
68.
ZweigM. H.CampbellG. (1993). Receiver-operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39, 561-577.
