Establishing Survey Validity and Reliability for American Indians Through “Think Aloud” and Test

Abstract

The purpose of this study was to use a mixed-methods approach to determine the validity and reliability of measurements used within an alcohol-exposed pregnancy prevention program for American Indian women. To develop validity, content experts provided input into the survey measures, and a “think aloud” methodology was conducted with 23 American Indian women. After revising the measurements based on this input, a test–retest was conducted with 79 American Indian women who were randomized to complete either the original measurements or the new, modified measurements. The test–retest revealed that some of the questions performed better for the modified version, whereas others appeared to be more reliable for the original version. The mixed-methods approach was a useful methodology for gathering feedback on survey measurements from American Indian participants and in indicating specific survey questions that needed to be modified for this population.

Keywords

aboriginal people,North America alcohol / alcoholism contraception reliability validity qualitative

Get full access to this article

View all access options for this article.

References

Abbott

P. J.

(2011). Screening American Indian/Alaska natives for alcohol abuse and dependence in medical settings. Current Drug Abuse Reviews, 4, 210–214.

Babor

Higgins-Biddle

J. C.

Saunders

J. B.

Monteiro

M. G.

(2001). AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for use in primary care (2nd ed.). Geneva, Switzerland: World Health Organization.

Bailey

B. A.

Sokol

R. J.

(2008). Pregnancy and alcohol use: Evidence and recommendations for prenatal care. Clinical Obstetrics and Gynecology, 51, 436–444.

Blood

Spratt

K. F.

(2007, April). Disagreement on agreement: Two alternative agreement coefficients. Paper presented at the SAS Global Forum, Orlando, Florida.

Breslin

F. C.

Sobell

L. C.

Sobell

M. B.

Agrawal

(2000). A comparison of a brief and long version of the Situational Confidence Questionnaire. Behaviour Research and Therapy, 38, 1211–1220.

Canales

Rakowski

(2006). Development of a culturally specific instrument for mammography screening: An example with American Indian women in Vermont. Journal of Nursing Measurement, 14, 99–115.

Chase

Reicks

Smith

Henry

Reimer

(2003). Use of the think-aloud method to identify factors influencing purchase of bread and cereals by low-income African American women and implications for whole-grain education. Journal of the American Dietetic Association, 103, 501–504.

Chen

J. H.

(2012). Maternal alcohol use during pregnancy, birth weight and early behavioral outcomes. Alcohol and Alcoholism, 47, 649–656.

Cicchetti

D. V.

Feinstein

A. R.

(1990). High agreement but low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology, 43, 551–558.

10.

Denzin

N. K.

Lincoln

Y. S.

(2000). Handbook of qualitative research (2nd ed.). Thousand Oaks, CA: SAGE.

11.

Donner

A. E. M.

(1987). Sample size requirements for reliability studies. Statistics in Medicine, 6, 441–448.

12.

Fahrenwald

Shangreaux

(2006). Physical activity behavior of American Indian mothers. Orthopaedic Nursing, 25, 22–29.

13.

Finer

L. B.

Zolna

M. R.

(2011). Unintended pregnancy in the United States: Incidence and disparities, 2006. Contraception, 84, 478–485.

14.

Fleiss

(1986). The design and analysis of clinical experiments. New York: John Wiley.

15.

Floyd

R. L.

Ebrahim

S. H.

Boyle

C. A.

(1999). Preventing alcohol-exposed pregnancies among women of childbearing age: The necessity of a preconceptional approach. Journal of Women’s Health & Gender-Based Medicine, 8, 733–736.

16.

Floyd

R. L.

Sobell

Velasquez

M. M.

Ingersoll

K. S.

Nettleman

M. D.

Sobell

. . . Nagaraja

(2007). Preventing alcohol-exposed pregnancies: A randomized controlled trial. American Journal of Preventive Medicine, 32, 1–10.

17.

Fonteyn

Fisher

(1995). Research corner. Use of think aloud method to study nurses’ reasoning and decision making in clinical practice settings. Journal of Neuroscience Nursing, 27, 124–128.

18.

Fonteyn

Kuipers

Grobe

S. J.

(1993). A description of think aloud method and protocol analysis. Qualitative Health Research, 3, 430–441.

19.

Gardin

F. A.

(2010). The “think-aloud” method to promote student modeling of expert thinking. Athletic Therapy Today, 15(4), 18–21.

20.

Gillam

S. L.

Fargo

Robertson

K. C.

(2009). Comprehension of expository text: Insights gained from think-aloud data. American Journal of Speech-Language Pathology, 18, 82–94.

21.

Göransson

K. E.

Ehnfors

Fonteyn

M. E.

Ehrenberg

(2008). Thinking strategies used by registered nurses during emergency department triage. Journal of Advanced Nursing, 61, 163–172.

22.

Grimley

D. M.

Prochaska

G. E.

Prochaska

J. O.

Velicer

W. F.

Galavotti

Cabral

R. J.

Lansky

(1996). Cross-validation of measures assessing decisional balance and self-efficacy for condom use. American Journal of Health Behavior, 20, 406–416.

23.

Gwet

(2001). Handbook of inter-rater reliability: How to measure the level of agreement between two or multiple raters. Gaithersburg, MD: StatAxis.

24.

Hanson

J. D.

Miller

A. L.

Winberg

Elliott

A. J.

(2013). Prevention of alcohol exposed pregnancies with non-pregnant American Indian women. American Journal of Health Promotion, 27, S66–S73.

25.

Henry

Reicks

Smith

Reimer

Atwell

Thomas

(2003). Identification of factors affecting purchasing and preparation of fruit and vegetables by stage of change for low-income African American mothers using the think-aloud method. Journal of the American Dietetic Association, 103, 1643–1646.

26.

Holmstrup

M. E.

Stearns-Bruening

Rozelle

(2013). Quantifying accurate calorie estimation using the “think aloud” method. Journal of Nutrition Education & Behavior, 45, 77–81.

27.

Ingersoll

K. S.

Floyd

Sobell

Velasquez

M. M.

, & Project CHOICES Research Group. (2003). Reducing the risk of alcohol-exposed pregnancies: A study of a motivational intervention in community settings. Pediatrics, 111, 1131–1135.

28.

Israel

B. A.

Parker

E. A.

Rowe

Salvatore

Minkler

Lopez

. . . Halstead

(2005). Community-based participatory research: Lessons learned from the Centers for Children’s Environmental Health and Disease Prevention Research. Environmental Health Perspectives, 113, 1463–1471.

29.

Jaspers

M. W. M.

Steen

van den Bos

Geenen

(2004). The think aloud method: A guide to user interface design. International Journal of Medical Informatics, 73, 781–795.

30.

Kesmodel

Wisborg

Olsen

S. F.

Henriksen

T. B.

Secher

N. J.

(2002). Moderate alcohol intake during pregnancy and the risk of stillbirth and death in the first year of life. American Journal of Epidemiology, 155, 305–312.

31.

Leonardson

G. R.

Kemper

Ness

F. K.

Koplin

B. A.

Daniels

M. C.

Leonardson

G. A.

(2005). Validity and reliability of the AUDIT and CAGE-AID in Northern Plains American Indians. Psychological Reports, 97, 161–166.

32.

Lupton

Burd

Harwood

(2004). Cost of fetal alcohol spectrum disorders. American Journal of Medical Genetics. Part C, Seminars in Medical Genetics, 127C, 42–50.

33.

MacNeela

Clinton

Place

Scott

Treacy

Hyde

Dowd

(2010). Psychosocial care in mental health nursing: A think aloud study. Journal of Advanced Nursing, 66, 1297–1307.

34.

May

P. A.

Miller

J. H.

Goodhart

K. A.

Maestas

O. R.

Buckley

Trujillo

P. M.

Gossage

J. P.

(2008). Enhanced case management to prevent fetal alcohol spectrum disorders in Northern Plains communities. Maternal and Child Health Journal, 12, 747–759.

35.

Nieman

L. Z.

Velasquez

M. M.

Groff

J. Y.

Cheng

Foxhall

L. E.

(2005). Implementation of a smoking cessation counseling module in a preceptorship program. Family Medicine, 37, 105–111.

36.

Pottier

Hardouin

Hodges

B. D.

Pistorius

Connault

Durant

. . . Planchon

(2010). Exploring how students think: A new method combining think-aloud and concept mapping protocols. Medical Education, 44, 926–935.

37.

Project CHOICES Research Group. (2002). Alcohol-exposed pregnancy: Characteristics associated with risk. American Journal of Preventive Medicine, 23, 166–173.

38.

Reicks

Smith

Henry

Reimer

Atwell

Thomas

(2003). Use of the think aloud method to examine fruit and vegetable purchasing behaviors among low-income African American women. Journal of Nutrition Education & Behavior, 35, 154–160.

39.

Sobell

Johnson

Velasquez

M. M.

von Sternberg

Nettleman

M. D.

Sidhu

(2003). Motivational intervention to reduce alcohol-exposed pregnancies—Florida, Texas, and Virginia, 1997-2001. Morbidity and Mortality Weekly Report, 52, 441–444.

40.

Trussell

(2011). Contraceptive failure in the United States. Contraception, 83, 397–404.

41.

Van Den Haak

M. J.

De Jong

M. D. T.

Schellens

P. J.

(2003). Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue. Behaviour & Information Technology, 22, 339–351.

42.

Velasquez

M. M.

Ingersoll

K. S.

Sobell

M. B.

Floyd

R. L.

Sobell

L. C.

Von Sternberg

(2010). A dual-focus motivational intervention to reduce the risk of alcohol-exposed pregnancy. Cognitive and Behavioral Practice, 17, 203–212.

Establishing Survey Validity and Reliability for American Indians Through “Think Aloud” and Test–Retest Methods

Abstract

Keywords

Get full access to this article

References