A framework for quantitative measurement development, validation, and revision that incorporates both qualitative and quantitative methods is introduced. It extends and adapts Adcock and Collier’s work, and thus, facilitates understanding of quantitative measurement development, validation, and revision as an integrated and cyclical set of procedures best achieved through mixed methods research. It also offers a systematic guide concerning how these procedures may be undertaken through detailing key “stages,” “levels,” and practical “tasks.” A case study illustrates how qualitative and quantitative methods may be mixed through the use of the proposed framework in the cross-cultural content- and construct-related validation and subsequent revision of a quantitative measure. The contribution of this article to mixed methods research literature is briefly discussed.
AdcockR.CollierD. (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95, 529-546.
2.
AllenJ.WalshJ. A. (2000). A construct-based approach to equivalence: Methodologies for cross-cultural/multicultural personality assessment research. In DanaR. H. (Ed.), Handbook of cross-cultural and multicultural personality assessment (pp. 63-85). Mahwah, NJ: Lawrence Erlbaum.
3.
American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35, 33-40.
4.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
5.
BeereC. A. (1990). Gender roles: A handbook of tests and measures. New York, NY: Greenwood Press.
6.
Ben-PorathY. S. (1990). Cross-cultural assessment of personality: The case for replicatory factor analysis. In ButcherN.SpielbergerC. D. (Eds.), Advances in personality assessment (pp. 27-48). Hillsdale, NJ: Lawrence Erlbaum.
7.
BertrandJ. T.BrownJ. E.WardV. M. (1992). Techniques for analysing focus groups data. Evaluation Review, 16, 198-209.
8.
BrannenJ. (1992). Combining qualitative and quantitative approaches: An overview. In BrannenJ. (Ed.), Mixing methods: Qualitative and quantitative research (pp. 3-37). Aldershot, England: Avebury.
9.
BrannenJ. (2009). Mixed methods for novice researchers: Reflections and themes. International Journal of Multiple Research Approaches, 3, 8-12.
10.
BraunV.ClarkeV. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101.
11.
BrislinR. W. (2000). Back-translation. In KazdinA. E. (Ed.), Encyclopedia of psychology (pp. 359-360). Washington, DC: American Psychological Association.
12.
CampbellD. T.FiskeD. A. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
13.
CaracelliV. J.GreeneJ. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15, 195-207.
14.
CoaleyK. (2010). An introduction to psychological assessment and psychometrics. London, England: SAGE.
15.
CollinsK. M. T.OnwuegbuzieA. J.SuttonI. L. (2006). A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 4, 67-100.
16.
CreswellJ. W.Plano ClarkV. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE.
17.
DenzinN. K. (1970). The research act in sociology: The theoretical introduction to sociological methods. London, England: Butterworth.
18.
DenzinN. K. (1978). The logic of naturalistic inquiry. In DenzinN. K. (Ed.), Sociological methods: A sourcebook (pp. 6-29). New York, NY: McGraw-Hill.
19.
FloydF. J.WidamanK. E. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299.
20.
GergenK. J. (1985). The social constructionist movement in modern psychology. American Psychologist, 40, 266-275.
21.
GergenK. J. (2005). An invitation to social construction. London, England: SAGE.
22.
GreeneJ. C.CaracelliV. J.GrahamW. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255-274.
23.
GriffinC.PhoenixA. (1994). The relationship between qualitative and quantitative research: Lessons from feminist psychology. Journal of Community & Applied Social Psychology, 4, 287-298.
24.
HammersleyM. (1992). Deconstructing the qualitative-quantitative divide. In BrannenJ. (Ed.), Mixing methods: Qualitative and quantitative research (pp. 39-55). Aldershot, England: Avebury.
25.
HoffmanR. M. (2001). The measurement of masculinity and femininity: Historical perspective and implications for counselling. Journal of Counseling & Development, 79, 472-485.
26.
HoganT. P. (2007). Psychological testing: A practical introduction. New York, NY: John Wiley.
27.
HoganT. P.AgnelloJ. (2004). An empirical study of reporting practices concerning measurement validity. Educational and Psychological Measurement, 64, 802-812.
28.
JickT. D. (1983). Mixing qualitative and quantitative methods: Triangulation in action. In Van MaanenJ. (Ed.), Qualitative methodology (pp. 135-148). Beverley Hills, CA: SAGE.
29.
JohnsonR. B.OnwuegbuzieA. J.TurnerL. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133.
30.
KelleU.LaurieH. (1995). Computer use in qualitative research and issues of validity. In KelleU. (Ed.), Computer-aided qualitative data analysis (pp. 19-95). London, England: SAGE.
31.
LevantR. F.HirschL. S.CelentanoE.CozzaT. M.HillS.MacEachernM.. . . SchnedekerJ. (1992). The male role: An investigation of contemporary norms. Journal of Mental Health Counseling, 14, 325-337.
32.
LuytR. (2003). Rhetorical representations of masculinities in South Africa: Moving towards a material-discursive understanding of men. Journal of Community & Applied Social Psychology, 13, 46-69.
33.
LuytR. (2005). The Male Attitude Norms Inventory-II: A measure of masculinity ideology in South Africa. Men and Masculinities, 8, 208-229.
34.
LuytR. (in press). Constructing hegemonic masculinities in South Africa: The discourse and rhetoric of heteronormativity. Gender and Language.
35.
MessickS. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
36.
MorganD. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.
37.
MorganG. A.GlinerJ. A.HarmonR. J. (2001). Measurement validity. Journal of American Academy of Child & Adolescent Psychiatry, 40, 729-731.
38.
Nagy Hesse-BiberC. (2010). Mixed methods research: Merging theory with practice. London, England: Guilford Press.
NeumanW. L. (1997). Social research methods: Qualitative and quantitative approaches. Boston, MA: Allyn & Bacon.
41.
O’BrienK. (1993). Improving survey questionnaires through focus groups. In MorganD. L. (Ed.), Successful focus groups: Advancing the state of the art (pp. 105-117). Newbury Park, CA: SAGE.
42.
OnwuegbuzieA. J.BustamanteR. M.NelsonJ. A. (2010). Mixed research as a tool for developing quantitative instruments. Journal of Mixed Methods Research, 4(1), 56-78.
43.
PettM. A.LackeyN. R.SullivanJ. J. (2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. London, England: SAGE.
44.
PonterottoJ. G.GriegerI. (1999). Merging qualitative and quantitative perspectives in a research identity. In KopalaM.SuzukiL. A. (Eds.), Using qualitative methods in psychology (pp. 49-62). London, England: SAGE.
45.
RustJ.GolombokS. (2008). Modern psychometrics: The science of psychological assessment. London, England: Routledge.
46.
TashakkoriA.TeddlieC. (1998). Introduction to mixed method and mixed model studies in the social and behavioural sciences. In TashakkoriA.TeddlieC. (Eds.), Mixed methodology: Combining qualitative and quantitative approaches (pp. 3-19). Thousand Oaks, CA: SAGE.
47.
TashakkoriA.TeddlieC. (Eds.). (2003). Handbook of mixed methods in social & behavioral research. Thousand Oaks, CA: SAGE.
48.
TeddlieC.TashakkoriA. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. London, England: SAGE.
49.
ThompsonE. H.PleckJ. H. (1995). Masculinity ideologies: A review of research instrumentation on men and masculinities. In LevantR. F.PollackW. S. (Eds.), A new psychology of men (pp. 129-163). New York, NY: Basic Books.
50.
ThompsonE. H.PleckJ. H.FerreraD. L. (1992). Men and masculinities: Scales for masculinity ideology and masculinity-related constructs. Sex Roles, 27, 573-607.
51.
TranT. V. (2009). Developing cross-cultural measurement. Oxford, England: Oxford University Press.
52.
van de VijverF. J. R.PoortingaY. H. (2005). Conceptual and methodological issues in adapting tests. In HambletonR. K.MerendaP. F.SpielbergerC. D. (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 39-63). Mahwah, NJ: Lawrence Erlbaum.
53.
van de VijverF. J. R.TanzerN. K. (2004). Bias and equivalence in cross-cultural assessment: An overview. Revue Européenne de Psychologie Appliquée, 54, 119-135.
54.
WebbE. J.CampbellD. T.SchwartzR. D.SechrestL. (1966). Unobtrusive measures: Nonreactive research in the social sciences. Chicago, IL: Rand McNally.
55.
WolffB.KnodelJ.SittitraiW. (1993). Focus groups and surveys as complementary research methods. In MorganD. L. (Ed.), Successful focus groups: Advancing the state of the art (pp. 118-136). Thousand Oaks, CA: SAGE.
56.
YunJ.UlrichD. A. (2002). Estimating measurement validity: A tutorial. Adapted Physical Activity Quarterly, 19, 32-47.
