Consequential validity (positive and negative consequences of test use) has long been considered a key component to assessment validity, yet explicit focus within measure development and evaluation research has been limited. In this paper, we offer a path forward using the Consequential Validity-Centered Measure Development (CVMD) framework. We argue that attending to consequential validity during the measure development process positions developers to proactively improve consequences of test use. CVMD is an integrated, mixed methods approach that includes 10 steps across four distinct phases of measure development. CVMD maintains all core elements of quantitatively focused measure development frameworks to ensure psychometric adequacy while also integrating qualitative methods to maximize usability, relevance, and positive consequences. We explain how the CVMD framework was operationalized in our development of the ESSY Whole Child Screener. In providing the CVMD framework, we aim to support assessment developers in centering consequential validity in the measure development process.
American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education (Joint Committee). (1966). Standards for educational and psychological tests and manuals. American Psychological Association.
2.
American Educational Research Association, and the American Psychological Association, and the National Council on Measurement in Education (Joint Committee). (2014). Standards for educational and psychological tests. American Educational Research Association.
3.
American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education (Joint Committee). (1999). Standards for educational and psychological testing. American Educational Research Association.
4.
BandalosD. L. (2018). Measurement theory and applications for the social sciences. The Guilford Press.
5.
BehizadehN.NeelyA. (2018). Testing injustice: Examining the consequential validity of edTPA. Equity & Excellence in Education, 51(3–4), 242–264. https://doi.org/10.1080/10665684.2019.1568927
6.
BoatengG. O.NeilandsT. B.FrongilloE. A.Melgar-QuinonezH. R.YoungS. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149. https://doi.org/10.3389/fpubh.2018.00149
7.
BraunV.ClarkeV. (2021). Thematic analysis: A practical guide. Sage.
8.
ByrneB. M. (2016). Adaptation of assessment scales in cross-national research: Issues, guidelines, and caveats. International Perspectives in Psychology, 5(1), 51–65. https://doi.org/10.1037/ipp0000042
9.
CaemmererJ. M.KoslouskiJ. B.BrieschA.ChafouleasS. M.ScudderA. M.MeloB. (2026). Advancing whole-child screening in schools: Initial validation of the ESSY screener [Manuscript submitted for publication]. Department of Educational Psychology, University of Connecticut.
10.
CizekG. J.BowenD.ChurchK. (2010). Sources of validity evidence for educational and psychological tests: A follow-up study. Educational and Psychological Measurement, 70(5), 732–743. https://doi.org/10.1177/0013164410379323
11.
CizekG. J.RosenbergS. L.KoonsH. H. (2008). Sources of validity evidence for educational and psychological tests. Educational and Psychological Measurement, 68(3), 397–412. https://doi.org/10.1177/0013164407310130
12.
CurtinM.FosseyE. (2007). Appraising the trustworthiness of qualitative studies: Guidelines for occupational therapists. Australian Occupational Therapy Journal, 54(2), 88–94. https://doi.org/10.1111/j.1440-1630.2007.00661.x
13.
EnoshG.TzafrirS. S.StolovyT. (2015). The development of client violence questionnaire (CVQ). Journal of Mixed Methods Research, 9(3), 273–290. https://doi.org/10.1177/1558689814525263
14.
FisherR. C.GerstnerJ. J.BandalosD. L. (2013, May). A review and analysis of scale development procedures in psychology and education [Poster Presentation]. Association for Psychological Science Annual Convention, Washington D.C.
15.
GoodwinL. D.LeechN. L. (2003). The meaning of validity in the new Standards for Educational and Psychological Testing: Implications for measurement courses. Measurement and Evaluation in Counseling and Development, 36(3), 181–191. https://doi.org/10.1080/07481756.2003.11909741
16.
GreenspoonP. J.SaklofskeD. H. (2001). Toward an integration of subjective well-being and psychopathology. Soc. Indic. Res, 54(1), 81–108. https://doi.org/10.1023/A:1007219227883
HermanK. C.BonifayW. (2023). Best practices for examining and reporting the social consequences of educational measures. School Psychology, 38(3), 129–136. https://doi.org/10.1037/spq0000550
19.
HermanK. C.ReinkeW. M.ThompsonA. M.HuangF.OwensS. (2023). Usability and social consequences of the early identification system as a universal screener for social, emotional, and behavioral risks. School Psychology, 38(3), 148–159. https://doi.org/10.1037/spq0000538
20.
HsiehH. F.ShannonS. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
21.
HubleyA. M.ZumboB. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103(2), 219–230. https://doi.org/10.1007/s11205-011-9843-4
JessenJ. D.OverbeckG.Koster-RasmussenR. (2023). Psychosocial consequences of weight screening of school-age children: A systematic review. Danish Medical Journal, 70(11), A09220534.
24.
JonesB. D. (2007). The unintended outcomes of high-stakes testing. Journal of Applied School Psychology, 23(2), 65–86. https://doi.org/10.1300/J370v23n02_05
25.
KaneM. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
26.
KoslouskiJ. B.ChafouleasS. M.BrieschA. M.CaemmererJ. M.PereiraB. (2024a). Developing a whole child school screening instrument: Evaluating perceived usability as an initial step in planning for consequential validity. School Mental Health, 16, 370–386. https://doi.org/10.1007/s12310-024-09670-w
27.
KoslouskiJ. B.ChafouleasS. M.BrieschA.CaemmererJ. M.PerryH.OasJ.XiongS.CharamutN. (2024b). School-based screening of social determinants of health: A scoping review. School Mental Health, 16, 1–14. https://doi.org/10.1007/s12310-023-09622-w
28.
LissitzR. W.SamuelsenK. (2007). Further clarification regarding validity and education. Educational Researcher, 36(8), 482–484. https://doi.org/10.3102/0013189X07311612
29.
LyonK.KoslouskiJ. B.ChafouleasS. M.BrieschA. M.CaemmererJ. M. (2025). Student perceptions of a whole child school screening instrument: An initial step in attending to consequential validity. School Mental Health, 17, 788–801. https://doi.org/10.1007/s12310-025-09782-x
30.
MattaM.MercerS. H.Keller-MargulisM. A. (2023). Implications of bias in automated writing quality scores for fair and equitable assessment decisions. School Psychology, 38(3), 173–181. https://doi.org/10.1037/spq0000517
31.
MessickS. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5–11. https://doi.org/10.3102/0013189x018002005
MessickS. (1998). Consequences of test interpretation and use: The fusion of validity and values in psychological assessment. ETS Research Report Series, 1998(2). https://doi.org/10.1002/j.2333-8504.1998.tb01797.x
34.
MillerF. G. (2023). Assessment in action: Toward a more complete and justice-oriented understanding of the social consequences of educational measures. School Psychology, 38(3), 192–198. https://doi.org/10.1037/spq0000551
35.
NielsenJ. (1994). Heuristic evaluation. In NielsenJ.MackR. L. (Eds.), Usability inspection methods (pp. 25–62). John Wiley & Sons.
36.
NormanG. (2015). The negative consequences of consequential validity. Advances in Health Sciences Education, 20(3), 575–579. https://doi.org/10.1007/s10459-015-9615-z
37.
OnwuegbuzieA. J.BustamanteR. M.NelsonJ. A. (2010). Mixed research as a tool for developing quantitative instruments. Journal of Mixed Methods Research, 4(1), 56–78. https://doi.org/10.1177/1558689809355805
38.
PernaL. W.ThomasS. L. (2007). Barriers to college opportunity: The unintended consequences of state-mandated testing. Educational Policy, 23(3), 451–479. https://doi.org/10.1177/0895904807312470
RiessmanC. K. (2008). Narrative methods for the human sciences. Sage Publications, Inc.
41.
SankofaN. L. (2022). Transformativist measurement development methodology: A mixed methods approach to scale construction. Journal of Mixed Methods Research, 16(3), 307–327. https://doi.org/10.1177/15586898211033698
SplettJ. W.BrannK. L.TrainorK. M.ShenZ. (2023). Examining utility and impact of social, emotional, and behavioral screening to identify and address needs. School Psychology, 38(3), 137–147. https://doi.org/10.1037/spq0000540
44.
UngarM.LiebenbergL. (2011). Assessing resilience across cultures using mixed methods: Construction of the child and youth resilience measure. Journal of Mixed Methods Research, 5(2), 126–149. https://doi.org/10.1177/1558689811400607
