Social validity is the extent to which procedures, goals, and outcomes are acceptable and important to end users and is not commonly addressed during assessment development. This article pilots a process of triangulating multiple methods to evaluate the social validity of a self-report assessment. These procedures were piloted using the Child Occupational Self-Assessment (COSA), a self-report of everyday activities for children with disabilities. This study collected COSA responses from 502 children with disabilities and conducted observations and interviews with 5 additional children with cerebral palsy. Data were aligned with social validity benchmark statements and integrated using data displays. Social validity was evaluated by examining congruencies and discrepancies in the results. Implications and the legitimacy of design decisions are discussed.
American Occupational Therapy Association. ( 2002). Occupational therapy practice framework: Domain and process . American Journal of Occupational Therapy, 56, 609-639.
2.
Betz, C.L., Redcay, G., & Tan, S. ( 2003). Self-reported health care self-care needs of transition age youth: A pilot study. Issues in Contemporary Pediatric Nursing, 26, 159-181.
3.
Bryman, A. ( 2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8-22.
4.
Cavet, J., & Sloper, P. ( 2004). Participation of disabled children in individual decisions about their lives and in public decisions about service development. Children & Society, 18, 278-290.
5.
Creswell, J.W., Plano Clark, V.L., Gutmann, M.L., & Hanson, W.E. ( 2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks, CA: SAGE.
6.
Dellinger, A.B., & Leech, N.L. ( 2007). Toward a unified validation framework in mixed methods research. Journal of Mixed Methods Research, 1(4), 309-332.
7.
Fasoli, L. ( 2003). Reading photographs of young children: looking at practices . Contemporary Issues in Early Childhood, 4, 32-47.
8.
Fawcett, S.B. ( 1991). Social validity: A note on methodology. Journal of Applied Behavior Analysis, 24, 235-239.
9.
Foster, S.L., & Mash, E.J. ( 1999). Assessing social validity in clinical treatment research issues and procedures. Journal of Consulting and Clinical Psychology, 67, 308-319.
10.
Gan, C., Campbell, K.A., Snider, A., Cohen, S., & Hubbard, J. ( 2008). Giving Youth a Voice (GYV): A measure of youth’s perceptions of the client-centredness of rehabilitation services. Canadian Journal of Occupational Therapy, 75, 96-104.
11.
Greene, J. ( 2008). Is mixed methods social inquiry a distinctive methodology ? Journal of Mixed Methods Research, 2(1), 7-22.
12.
Hemmingsson, H., Kottorp, A., & Bernspång, B. (2004). Validity of the school setting interview: an assessment of the student-environment fit. Scandinavian Journal of Occupational Therapy, 11, 171-178.
13.
Hitchcock, J.H., Nastasi, B.K., Dai, D.Y., Newman, J., Jayasena, A., Bernstein-Moore, R., . . . Varjas, K. ( 2005). Illustrating a mixed-method approach for validating culturally specific constructs. Journal of School Psychology, 43, 259-278.
14.
Jang, E.E., McDougall, D.E., Pollon, D., Herbert, M., & Russell, P. ( 2008). Integrative mixed methods data analytic strategies in research on school success in challenging circumstances. Journal of Mixed Methods Research, 2(3), 221-247.
15.
Johnson, R.B., & Onwuegbuzie, A.J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher , 33(7), 14-26.
16.
Keller, J., Kafkes, A., Basu, S., Federico, J., & Kielhofner, G. (2005). The Child Occupational Self Assessment (COSA) Version 2.1. Chicago, IL: MOHO Clearinghouse, Department of Occupational Therapy, College of Applied Health Sciences, University of Illinois at Chicago.
17.
King, G., Law, M., King, S., Hanna, S., Kertoy, M., Rosenbaum, P., & Young, N. ( 2004). Children’s Assessment of Participation and Enjoyment (CAPE) and Preferences for Activities of Children (PAC). San Antonio, TX: Harcourt Assessment.
18.
Kramer, J., & Hammel, J. (IN PRESS). "I do lots of things:" Children with cerebral palsy’s competence for everyday activities. International Journal of Disability, Development, and Education.
19.
Kramer, J., Smith, E.V., Jr., & Kielhofner, G. (2009). Rating scale use by children with disabilities on a self report of everyday activities. Archives of Physical Medicine and Rehabilitation, 90, 2047-2053.
20.
Lee, Y.J., & Greene, J. ( 2007). The predictive validity of an ESL placement test: A mixed methods approach. Journal of Mixed Methods Research, 1(4), 366-389.
21.
Masters, G.N. ( 1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
22.
McConney, A., Rudd, A., & Ayres, R. ( 2002). Getting to the bottom line: A method for synthesizing findings within mixed-methods program evaluations. American Journal of Evaluation, 23, 121-140.
23.
Messick, S. ( 1995). Validity of Psychological Assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
24.
Messick, S. ( 1998). Test validity: A matter of consequence. Social Indicators Research, 45, 35-44.
25.
Miles, M. B., & Huberman, A. B. (Eds.). (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: SAGE.
26.
Morgan, D.L. ( 2007). Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.
27.
Morse, J.M. ( 1998). Designing funded qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Collecting and interpreting qualitative materials (pp. 56-85). Thousand Oaks, CA: SAGE.
28.
Myford, C.M., & Mislevy, R.J. ( 1995). Monitoring and improving a portfolio assessment system . Princeton, NJ: Center for Performance Assessment.
29.
Onwuegbuzie, A.J., & Collins, K. ( 2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12, 281-316.
30.
Onwuegbuzie, A.J., & Johnson, R.B. ( 2006). The validity issue in mixed research. Research in the Schools, 13, 48-63.
Rebok, G., Riley, A., Forrest, C., Starfield, B., Green, B., Robertson, J., & Tambor, E. ( 2001). Elementary school-aged children’s reports of their health: A cognitive interviewing study. Quality of Life Research , 10, 59-70.
33.
Rost, J. ( 1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement , 14, 271-282.
34.
Smith, E.V., Jr., Conrad, K.M., Chang, K., & Piazza, J. ( 2002). An introduction to Rasch measurement for scale development and person assessment. Journal of Nursing Measurement, 10, 189-206.
35.
Strauss, A., & Corbin, J. ( 1998). Grounded theory methodology: An overview. In N. K. Denzin & Y. S. Lincoln (Eds.), Strategies of qualitative inquiry (pp. 158-183). Thousand Oaks, CA: SAGE.
36.
Sturgess, J., Rodger, S., & Ozanne, A. ( 2002). A review of the use of self-report assessment with young children. British Journal of Occupational Therapy, 65, 108-116.
37.
Taylor, S.J., & Bogdan, R. ( 1998). Introduction to qualitative research methods: A guidebook and resource (3rd ed.). New York, NY: Wiley.
38.
Teddlie, C., & Yu, F. ( 2007). Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research, 1(1), 77-100.
39.
Turner, C. ( 2003). Are you listening? What disabled children and young people in Wales think about the services they use. Cardiff, Wales : Barnardos, Children First, NCH.
40.
Valnes, B., Hare, R., Block, J., Branch, C., & Broshears, G. ( 2003). Future directions agenda for youth with disabilities, 2003. Portland, OR: National Youth Leadership Network.
41.
Wagner-Menghin, M.M. ( 2006). The Mixed-Rasch Model: An example for analyzing the meaning of response latencies in a personality questionnaire. Journal of Applied Measurement, 7(2), 225-237.
42.
Wolf, M.M. ( 1978). Social validity: The case for subjective measurement, or, how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214.
43.
World Health Organization. (2001). International classification of functioning, disability, and health. Geneva, Switzerland: Author.
44.
Wright, B.D., & Linacre, M. ( 1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 70, 857-860.
45.
Wright, B.D., & Stone, M.H. ( 1979). Best test design. Chicago, IL: MESA Press.
46.
Young, B., Rice, H., Dixon-Woods, M., Colver, A.F., & Parkinson, K.N. (2007). A qualitative study of the health-related quality of life of disabled children. Developmental Medicine & Child Neurology, 49, 660-665.
