There is paucity of detailed information in the literature on how to develop procedures for estimating content validity to increase trustworthiness of assessment instruments. The authors present three unique examples, interpretation, and application of Tables of Specifications (ToS) for generating these estimates. The development and logic of the ToS requires presenting evidence that has transparency and creates trustworthiness in the validity estimates by maintaining an audit trail and using of triangulation, expert debriefing, and peer review. The argument presented is that content validity requires a mixed methods approach since data are developed through qualitative and quantitative methods that inform each other. This process is iterative and provides feedback on the effectiveness of the ToS through a process of determining consensus.
AryD.JacobsL.RazaviehA. (2005). Introduction to research in education (7th ed.). New York, NY: Wadsworth.
2.
BeggsD.LewisE. (1975). Measurement and evaluation in the schools. Boston, MA: Houghton Mifflin.
3.
BloomB. S. (1956). Taxonomy of educational objectives: The classification of educational goals. New York, NY: David McKay.
4.
BrodM.TeslerL.ChristensenT. (2009). Qualitative research and content validity: Developing best practices based on science and experience. Quality of Life Research, 18, 1263-1278.
5.
BryantA.CharmazK. (2007). Grounded theory in historical perspective: An epistemological account. In BryantA.CharmazK. (Eds.), The Sage handbook of grounded theory (pp. 31-74). Los Angeles, CA: Sage.
6.
CarminesE. G.ZellerR. A. (1991). Reliability and validity assessment. Newbury Park, CA: Sage.
7.
CeglowskiD. A. (2012, April). Aced out: Censorship of qualitative research in the age of scientifically based research. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.
8.
ChristT. W. (2012, April). Policy, scientifically based research, and randomized control trials: The gold standard?Paper presented at the annual meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.
9.
CookT. D. (2003). Why have educational evaluators chosen not to do randomized experiments?Annals of the American Academy of Political and Social Science, 589, 114-149.
10.
CronbachL. J. (1970). Essentials of psychological testing (3rd ed.). New York, NY: Harper & Row.
11.
CronbachL. J. (1971). Test validation. In ThorndikeR. L. (Ed.), Educational measurement (2nd ed., pp. 443-507). Washington, DC: American Council on Education.
12.
CullenA.CorynC.RughJ. (2011). The politics and consequences of including stakeholders in international development evaluation. American Journal of Evaluation, 32, 345-361.
GlaserB.StraussA. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine.
18.
GuionR. (1977). Content validity: The source of my discontent. Applied Psychological Measurement, 1, 1-10.
19.
HaynesS. N.RichardD. C. S.KubanyE. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238-247.
20.
HubleyA. M.ZumboB. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103, 219-230.
21.
JohnsonB.ChristensenL. (2004). Educational research: Quantitative, qualitative, and mixed approaches (2nd ed.). New York, NY: Pearson.
22.
KerlingerF. N.LeeH. B. (2000). Foundations of behavioral research (4th ed.). Fort Worth, TX: Harcourt College.
23.
KrathwohlD. R. (2009). Methods of educational and social science research: The logic of methods (3rd ed.). Long Grove, IL: Waveland Press.
24.
LawsheC. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563-575.
LimJ. (2010). The development of an instrument for K12 coordinators implementing curriculum videoconferencing and a model to predict utilization of videoconferencing. Berrien Springs, MI: Andrews University.
28.
LimJ.FreedS. (2009). We have the videoconference equipment installed, now what?The Qualitative Report, 14, 433-453. Retrieved from http://www.nova.edu/ssss/QR/QR14-3/lim.pdf
29.
LundyJ. J. (2012, April). Mixed methods approach to assuring content validity. Paper presented at the third annual Patient-Reported Outcome (PRO) Consortium Workshop, Silver Spring, MD.
30.
McNeilK. A.NewmanI.SteinhauserJ. (2005). How to be involved in program evaluation: What every administrator needs to know. Lanham, MD: Scarecrow Education.
31.
MessickS. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.
32.
MessickS. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
33.
Mosby. (2008). Mosby’s medical dictionary. St. Louis, MO: Elsevier.
34.
NewmanI.FryeB.BlumenfeldG.NewmanC. (1973). An introduction to the basic concepts of measurement and evaluation. Akron, OH: University of Akron.
35.
NewmanI.McNeilK. A. (1998). Conducting survey research in the social sciences. Lanham, MD: University Press of America.
36.
NewmanI.NewmanC.BrownR.McNeelyS. (2006). Conceptual statistics for beginners (3rd ed.). Lanham, MD: University Press of America.
37.
NewmanI.NewmanD.NewmanC. (2011). Writing research articles using mixed methods: Methodological considerations to help you get published. In RoccoT. S.HatcherT. (Eds.), The handbook of scholarly writing and publishing (pp. 191-208). San Francisco, CA: Jossey-Bass.
38.
NunnallyJ. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.
39.
NunnallyJ.BernsteinI. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
40.
PinedaM. F. (2011). The development and validation of an instrument to measure the attainment of competencies at an intercultural university in Mexico (Unpublished doctoral dissertation proposal), Florida International University, Miami.
41.
PolitD. F.BeckC. T. (2006). The content validity index: Are you sure you know what’s being reported? critique and recommendations. Research in Nursing & Health, 29, 489-497.
42.
PolitD. F.BeckC. T.OwenS. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30, 459-467.
43.
RidenourC.NewmanI. (2008). Mixed methods research: Exploring the interactive continuum. Carbondale, IL: Southern Illinois University Press.
44.
RubioD. M.Berg-WegerM.TebbS. S.LeeE. S.RauchS. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27, 94-104.
45.
SondergeldT. A. (2011, April). An illustrative example of why mixed-methods research is important in understanding the impact of comprehensive school reform on student outcomes. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
46.
StraussA.CorbinJ. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.
47.
StraussA.CorbinJ. (1994). Grounded theory methodology: An overview. In DenzinN. K.LincolnY. S. (Eds.), The handbook of qualitative research (pp. 273-285). Thousand Oaks, CA: Sage.
48.
TashakkoriA.NewmanI. (2010). Mixed methods: Integrating quantitative and qualitative approaches to research. In McGawB.BakerE.PetersonP. P. (Eds.), International encyclopedia of education (3rd ed., pp. 514-520). Oxford, England: Elsevier.
49.
TashakkoriA.TeddlieC. (2003). Handbook of mixed methods in social & behavioral research. Thousand Oaks, CA: Sage.
50.
TeddlieC.TashakkoriA. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Los Angeles, CA: Sage.
51.
TurnerB. A. (1981). Some practical aspects of qualitative data analysis: One way of organising the cognitive processes associated with the generation of grounded theory. Quality & Quantity, 15, 225-247.
52.
WilsonF. R.PanW.SchumskyD. A. (2012). Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development, 45, 197-210.
