Developing valid and reliable instruments is crucial, but costly and time-consuming in health care research and evaluation. The Food and Drug Administration (FDA) and the National Institutes of Health (NIH) have set up guidelines for developing patient-reported outcome (PRO) instruments. However, the guidelines are not applicable to cases of small sample sizes. Instead of using an exact estimation procedure to examine psychometric properties, the Bayesian Instrument Development (BID) method integrates expert data and participant data into a single seamless analysis. Using a novel set of priors, simulated data were used to compare BID to classical instrument development procedures and test the stability of BID. To display BID to non-statisticians, a graphical user interface (GUI) based on R and WINBUGS is developed and demonstrated with data on a small sample of heart failure patients. Costs were saved by eliminating the need for unnecessary continuation of data collection for larger samples as required by the classical instrument development approach.
BeckC. T.GableR. K. (2001). Item response theory in affective instrument development: an illustration. Journal of Nursing Measurement, 9, 5-22.
4.
BrowneW. J.DraperD. (2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1, 473-550.
5.
CellaD.RileyW.StoneA.RothrockN.ReeveB.YountS.. . . HaysR. D. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63, 1179-1194.
6.
ChalonerK. (1987). A Bayesian approach to the estimation of variance components for the unbalanced one-way random model. Technometrics, 29, 322-337.
7.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
8.
CrockerL.AlginaJ. (1986). Introduction to classical and modern test theory. Boston, MA: Harcourt Brace Jovanovich.
9.
CupertinoA. P.HuntJ. J.GajewskiB. J.JiangY.MarquisJ.FriedmannP. D.. . . RichterK. P. (2013). Substance abuse treatment, prevention, and policy. Manuscript in preparation.
10.
EfronB. (2010). The future of indirect evidence. Statistical Science, 25, 145-157.
11.
GajewskiB. J.CofflandV.BoyleD.BottM. J.PriceL.LeopoldJ.DuntonN. (2012). Assessing content validity through correlation and relevance tools: A Bayesian randomized equivalence experiment. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 8, 81-96.
12.
GajewskiB. J.HartS.BergquistS.DuntonN. (2007). Inter-rater reliability of pressure ulcer staging: Ordinal probit Bayesian hierarchical model that allows for uncertain rater response. Statistics in Medicine, 26, 4602-4618.
13.
GajewskiB. J.NicholsonN.WidenJ. E. (2009). Predicting hearing threshold in non-responsive subjects using a log-normal Bayesian linear model in the presence of left censored covariates. Statistics in Biopharmaceutical Research, 1, 137-148.
14.
GajewskiB. J.PriceL.CofflandV.BoyleD.BottM. J. (2013). Integrated analysis of content and construct validity of psychometric instruments. Quality & Quantity, 47, 57-78.
15.
GajewskiB. J.SedwickJ. D.AntonelliP. J. (2004). A log-normal distribution model of the effect of bacteria and ear fenestration on hearing loss: A Bayesian approach. Statistics in Medicine, 23, 493-508.
16.
GarthwaiteP. H.KadaneJ. B.O’HaganA. (2005). Statistical methods for eliciting probability distributions. Journal of the American Statistical Association, 100, 680-701.
17.
GreenS. B.YangY. (2009). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74, 121-135.
18.
GrosjeanP. (2012). SciViews: A GUI API for R. UMONS, Mons, Belgium [Computer software]. Retrieved from http://www.sciviews.org/SciViews-R
19.
LeeS. Y. (1981). A Bayesian approach to confirmatory factor analysis. Psychometrika, 46, 153-160.
20.
LeeS. Y. (2007). Structural equation modeling: A Bayesian approach. New York, NY: Wiley.
21.
LeeS. Y.ShiJ. Q. (2000). Joint Bayesian analysis of factor scores and structural parameters in the factor analysis model. Annals of the Institute of Statistical Mathematics, 52, 722-736.
22.
LeeS. Y.SongX. Y. (2004). Evaluation of the Bayesian and maximum likelihood approaches in analyzing structural equation models with small sample sizes. Multivariate Behavioral Research, 39, 653-686.
23.
LunnD. J.ThomasA.BestN.SpiegelhalterD. (2000). WINBUGS—A Bayesian modeling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325-337.
24.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13-103). New York, NY: American Council on Education.
25.
MichalakE. E.MurrayG. (2010). Development of the QoL.BD: A disorder-specific scale to assess quality of life in bipolar disorder. Bipolar Disorder, 12, 727-740.
26.
PawlowiczF.GajewskiB. J.CofflandV.BoyleD.BottM.DuntonN. (2012). Application of Bayesian methodology within an equivalence content validity study. Nursing Research, 61, 181-187.
27.
PolitD. F.BeckC. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29, 489-497.
28.
R Development Core Team. (2012). R: A language and environment for statistical computing [Computer software]. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.R-project.org/
RosseelY. (2012). lavaan: An R Package for structural equation modeling. Journal of Statistical Software, 48, 1-36.
31.
SamaniegoF. J.ReneauD. M. (1994). Toward a reconciliation of the Bayesian and frequentist approaches to point estimation. Journal of the American Statistical Association, 89, 947-957.
32.
SpeightJ.BarendseS. M. (2010). FDA guidance on patient reported outcomes. British Medical Journal, 340, c2921.
33.
ThomasJ. H.NanM. L. (2009). fgui: A method for automatically creating graphical user interfaces for command-line R packages. Journal of Statistical Software, 30, 1-14. Retrieved from http://www.jstatsoft.org/v30/i02/
34.
WestlandJ. C. (2010). Lower bounds on sample size in structural equation modeling. Electronic Commerce Research and Applications, 9, 476-487.
35.
WestlandJ. C. (2012). Erratum to “Lower bounds on sample size in structural equation modeling” [Electron. Commerce Res. Appl. 9 (6) (2010) 476–487]. Electronic Commerce Research and Applications, 11(4), 445.
36.
WilksS. S. (1962). Mathematical statistics. New York, NY: Wiley.
