Extraneous Effects of Race,Gender,and Race-Gender Homo- and Heterophily Conditions on Data Quality

Abstract

This study comprehensively investigated the differences in response patterns of interview respondents by race, gender, and race-gender of both respondents and interviewers, to assess the impacts of response inconsistencies on data quality during survey interviews. The study focused only on Blacks and Whites in various interview phily matches. Interviewees (N = 491) responded to fully structured, closed-ended questions through direct interviews on support for affirmative action, and support for the 2009 America’s Affordable Health Choices Act as dependent variables. Findings showed various amounts of response differences to both dependent variables by differences in race, gender, and race-gender of respondents, vis-à-vis those of the interviewers’, thereby constituting various amounts of data inconsistencies. The effects of race, gender, and race-gender of both interviewers and respondents constitute potential nonrandom errors that must be controlled in interview survey research, otherwise, research findings and conclusions may diverge from true relationships between variables.

Keywords

race effect gender effect interviewer effect phily matches homophily heterophily

This study comprehensively analyzed differences in interview response patterns by race, gender, and race-gender (i.e., race plus gender) of respondents by differences in race, gender, and race-gender of interviewers to demonstrate the impacts of response inconsistencies on data quality obtained during survey interviews. For decades, several studies have identified interviewer effects as pervasive in data collection (D. W. Davis, 1997a, 1997b; Ellison, McFarland, & Krause, 2011; Krysan & Couper, 2003; Lievens & De Paepe, 2004; Tabane & Bouwer, 2006). According to Tabane and Bouwer (2006), interviewer effects occur because the data collection process is an interactional process involving power relationships, which favors the interviewer over the respondent. Consequent to interviewer effects, data generated in an interview survey might reflect various forms of inaccuracies and misinterpretations (D. W. Davis, 1997b).

To date, factors such as interviewer attitude (Durrant, Groves, Staetsky, & Steele, 2010), age (Collins & Butcher, 1983), and education (Durrant et al., 2010) had been studied relative to interviewer effects on data quality. The most commonly studied interviewer effects, however, have been race (D. W. Davis, 1997a, 1997b; D. W. Davis & Silver, 2003; Ellison et al., 2011; Januszka, Lora, Wollard, & Rocco, 2007; Krysan & Couper, 2003; Lange, 2002; Springman, Wherry, & Notaro, 2006; Webster, 1996) and gender (R. E. Davis, Couper, Janz, Caldwell, & Resnicow, 2010; Dykema, Diloreto, Price, White & Schaeffer, 2012; Flores-Macias & Lawson, 2008; Liu & Stainback, 2013; Weinreb, 2006) of interviewer.

Race and Gender Effects

Overwhelmingly, studies on race-of-interviewer effects have been predominant on Black¹ respondents (D. W. Davis, 1997a, 1997b; Ellison et al., 2011; Lievens & De Paepe, 2004),with a result pattern that showed that Blacks altered their responses to interview items based on the race of their interviewers (Ellison et al., 2011; Webster, 1996). For example, Blacks admitted to White¹ interviewers than to Black interviewers, that they (Blacks) lacked power to change things or make a political difference through voting. They (Blacks) also over reported having voted in an election when interviewed by other Blacks than when interviewed by Whites (D. W. Davis, 1997a). With specific reference to older people, Ellison et al. (2011) found that older Blacks claimed higher levels of non-organizational religious practices when interviewed by Whites than by Blacks. On race sensitive issues such as discrimination, Krysan and Couper (2003) indicated that Blacks significantly denied racial discrimination against themselves when the interviewer was White but not when the interviewer was a fellow Black.

Like Blacks, White respondents are also influenced by race of the interviewer. Studies on White respondents (R. E. Davis et al., 2010; Finkel, Guterbock, & Borg, 1991; Gong & Aadland, 2011; Krysan & Couper, 2003) agreed that Whites gave more liberal responses on race-related questions when interviewed by Blacks than by Whites. Whites also appear to acquiesce to race of interviewer on racially sensitive issues (D. W. Davis, 1997a). Whites interviewed by Blacks were reluctant to oppose racially centered policies, and Whites also minimized their tendencies for discrimination against Blacks when the interviewer was Black but not when the interviewer was White (D. W. Davis, 1997a; Krysan & Couper, 2003). In addition, when the interviewer was White rather than Black, White respondents strongly expressed trust of all Whites and preference for an all-White community (Krysan & Couper, 2003).

Patterns of gender-of-interviewer effects appear inconsistent within and across studies. Flores-Macias and Lawson (2008), for example, found gender-of-interviewer effects in bivariate analysis in their Mexico City data, that men interviewed by women identified women’s rights as a “very urgent” priority for the next president by 10 percentage points more than men interviewed by men. Also, men interviewed by men were more likely to favor criminalizing abortion in the case of rape, than men interviewed by women. In the same study, however, Flores-Macias and Lawson (2008) found women to be equally progressive on abortion questions between men and women interviewers, but they (women respondents) were less progressive when interviewed by men on women’s rights questions. Also, in their Mexico national sample, Flores-Macias and Lawson indicated that there were virtually no differences in attitudes toward abortion among male respondents by gender of interviewer, but women appear to become more progressive on abortion when interviewed by men than by women. Like Flores-Macias and Lawson (2008), Liu and Stainback (2013) also found mixed results in their study of gender-of-interviewer effects. They found respondents to be more pro-marriage when interviewed by women than by men interviewers, but they also indicated that the likelihood of gender-of-interviewer effects was inconsistent across their statistical models.

Other studies (Huddy et al., 1997; Kane & Macaulay, 1993) found gender effects to be significant such that respondents acquiesced to the gender of the interviewer. Gong and Aadland (2011) also found gender effects wherein respondents demonstrated a significantly higher willingness to pay for a curbside recycling program when interviewed by women than when interviewed by men. Weinreb (2006), however, found that women responded similarly to women stranger-interviewers as to men interviewers, but they (women respondents) gave different answers to women-insider interviewers. Weinreb’s findings stressed that, except for the “insider situation” in which the woman interviewer was a recognized member of a community, women respondents did not show a general pattern of difference in response to interview questions based on gender of interviewer (Weinreb, 2006).

Objective

While many studies had been done on interviewer effects, most have been done on race-of-interviewer effects with a predominant focus on Black respondents (D. W. Davis, 1997a, 1997b; Ellison et al., 2011; Lange, 2002). Fewer studies (e.g., Finkel, Guterbock, Marian, & Borg, 1991; Gong & Aadland, 2011; Krysan & Couper, 2003) appeared to have included White respondents. Unlike race-of-interviewer effects, studies on gender-of-interviewer are much fewer, despite the call by some studies (Oppenheim, 1992; Warren, 1988) for more rigorous analyses of gender-of-interviewer influences on response behavior. Also, previous studies have predominantly analyzed either the race or gender-of-interviewer effects, but not both in the same study, and studies that analyzed both race and gender effects in the same study (e.g., Gong & Aadland, 2011) did not comprehensively analyze all race-gender phily conditions in their analyses. That is, a comprehensive analysis of differences in response patterns by the combination of race, gender, and race-gender of respondents, vis-à-vis those of the interviewer, have been largely absent in previous studies. Based on the shortcomings of previous studies, the objective of this study was to comprehensively test for differences in interview response in all possible phily matches involving race, gender, and race-gender of both respondent and interviewer, between Blacks and Whites, within the context of the same single study.

Theoretical Guide: Inconsequentiality Versus Consequentiality of Interviewer Effects

This study is guided by two competing theoretical assumptions: the inconsequentiality versus consequentiality of interviewer effects on data quality. Many studies on interviewer effects (D. W. Davis, 1997a, 1997b; R. E. Davis et al., 2010; Gong & Aadland, 2011; Krysan & Couper, 2003; Lievens & De Paepe, 2004; Springman et al., 2006; Tabane & Bouwer, 2006) have concluded that interview responses showed a structured (i.e., nonrandom) relationship between interviewer and respondent demographic characteristics, especially when a question was race or gender sensitive. Some other studies (Allen, Dawson, & Brown, 1989; Reese & Brown, 1995) have, however, concluded that interviewer effects were inconsequential to data quality. This position was earlier claimed by Schuman and Converse (1971) that differences observed because of demographic characteristics of an interviewer were just differences, rather than an indication of greater or lesser accuracy of response. Schuman and Converse contended that different answers to different racial categories of interviewers did not demonstrate evidence of data distortion or proof that the answers given to one category were more valid than those given to another. D. W. Davis (1997a), however, argued that to ignore interviewer effects, such as the influence of race and gender on data quality, was to ignore a legitimate source of variance, capable of biasing data quality and statistical analysis. In addition, to ignore the race-of-interviewer effects on respondents is to fail to be sensitive to everyday issues of race relations (D. W. Davis, 1997a).

Distorted data could result from both mono- and cross-cultural conditions of data collection between interviewer and respondent. In mono-cultural situations, people tend to open up to each other and talk more freely about sensitive issues than in cross-cultural conditions (Ghane, Kolk, & Emmelkamp, 2010; Tabane & Bouwer, 2006). But, when extreme demographic similarities exist between an interviewer and an interviewee, the problem of potential consensus (Webster, 1996), persuasion, and respondents’ tendency to align their views with those of the interviewer, become highly realistic (Anderson & Alpert, 1974), and capable of validating the discrete interaction distortion hypothesis (Webster, 1996). This hypothesis states that “a person is highly likely to distort his/her answers when positive attraction and the subsequent desire to impress exist in a temporary discrete interaction between two parties” (Webster, 1996, p. 64). Cross-cultural interviewer-respondent conditions tend to produce distorted data mainly because such conditions are often marred by language differences, lack of understanding of the other, and socio-political insensitivity (Tabane & Bouwer, 2006).

To measure the likelihood of data distortion due to the race and gender of both interviewers and respondents through response inconsistences requires analyses of interview responses under all possible phily conditions (homophilous and heterophilous) between interviewer and interviewee. Using race and gender of both interviewer and respondent for phily matches, interview responses under all possible phily matches (see Table 1) between interviewers and respondents need to be analyzed, to determine the extent to which the competing ideas of whether interviewer effects are inconsequential or not to data quality are supported based on mono- or cross-cultural conditions. While analysis of cultural difference is beyond the scope of this study, a comprehensive analysis of demographic homo- and heterophily matches is adequate for studying interviewer effects under mono- and cross-cultural conditions between interviewer and respondent. The assumption from our phily analyses, hence, are that interviewer effects will be regarded as consequential to data quality if response patterns during homophilous conditions are significantly different from heterophilous matches between interviewers and respondents. Conversely, interviewer effects will be regarded as inconsequential to data quality if patterns of interviewer response are similar in both homo- and heterophilous conditions.

Table 1.

Independent Variables: Complete Phily Matches for Race, Gender, and Race-Gender of Interviewer and Respondent.

Phily types		Respondent	Interviewer
Homophilous matches		Race homophily
	1	White	White
	2	Black	Black
		Gender homophily
	3	Man	Man
	4	Woman	Woman
		Race-gender homophily
	5	White men	White men
	6	White women	White women
	7	Black men	Black men
	8	Black women	Black women
Heterophilous matches		Race heterophily
	1	Black	White
	2	White	Black
		Gender heterophily
	3	Women	Men
	4	Men	Women
		Race-gender heterophily
	5	Black men	White men
	6	Black men	White women
	7	Black men	Black women
	8	Black women	White men
	9	Black women	White women
	10	Black women	Black men
	11	White women	Black men
	12	White women	Black women
	13	White women	White men
	14	White men	Black men
	15	White men	Black women
	16	White men	White women

Method

Independent Variable

To assess differences in interview response by race, gender, and race-gender of respondents vis-à-vis race, gender, and race-gender of interviewers, this study generated 24 phily matches (8 homophilies and 16 heterophilies) between respondents and interviewers as independent variables (see Table 1). All 24 phily matches, as independent variables, were used to conduct a comprehensive analysis of interview response inconsistencies, of Black and White, and men and women respondents, vis-à-vis Black and White, and men and women interviewers on response patterns to two selected dependent variables.

Dependent Variables

The influences of race, gender, and race-gender phily matches were tested on two separate dependent variables; support for affirmative action (AA) practice, and support for America’s Affordable Health Choices Act (AHC) of 2009. These two government policies were selected as dependent variables because they readily generate public opinion, and they allowed us the opportunity to analyze phily effects on one policy which explicitly made provisions for race and gender in its letters (AA), and one that neither explicitly nor implicitly made references to race and gender (AHC). This will help to affirm or deny the commonality of race and gender effects on both race and gender sensitive and neutral topics.

Support for AA

The selection of support for AA practice as a dependent variable was based on literature that affirmed the language of AA as explicitly race and gender specific (U.S. Commission on Civil Rights, 1981) and literature that has concluded that race of interviewer significantly shaped interviewees’ responses on race-sensitive items (Alderfer & Tucker, 1996; D. W. Davis, 1997a, 1997b). Support for AA has also been established to be racially sensitive and gender patterned. Minorities, especially Blacks, for example, are more likely to support AA than non-Hispanic Whites, and women are more likely to favor the policy than men (Konrad & Spitz, 2003; Oyinlade, 2013; Park, 2009; Smith, 1998). It was, therefore, anticipated in this study that interview response to this variable would be shaped by race and gender phily matching of both interviewer and interviewee. The measurement of this variable (support for AA) is explained under “measures.”

Support for AHC of 2009: H.R. 3200

Since first introduced by the Obama Administration (later amended as H.R. 3962-Affordable Health Care for America Act, and signed into law as H.R. 3590: Patient Protection and Affordable Care Act; Congress.Gov, 2009), AHC has been heavily debated in the U.S. Congress and the American popular media. The Act was written to make health care affordable for all Americans, and its text was race and gender neutral (Congress.Gov, 2009). The central theme in the debate on AHC, arguably, has been racial and gender neutral, but mainly political (Democrats vs. Republicans; Liberals vs. Conservatives, etc.). Because of its popularity in the media, and the political tones (rather than racial or gendered) of the debates, we anticipated that respondents would have opinions on the bill, and their opinions might be racially and gender neutral. This assumption was not borne out of naivety for the general potential influences of race and gender on social issues in the United States. We are well aware that the characterization of AHC as “Obamacare” may be laden with negative racial undertones, but the policy, however, provided us an opportunity for phily comparisons with AA policy, a much more confirmed racially and gender sensitive social policy. The following research question was answered to meet the objective of this study:

Research Question: How were the levels of support for AA and AHC different by race, gender, and race-gender philies? That is, how were the levels of support for AA and AHC different by race, gender, and race-gender of respondents, vis-à-vis race, gender, and race-gender of interviewers?

Answers to this question would show patterns of change (i.e., inconsistencies) in respondents’ answers to interview questions on both AA and AHC by race, gender, and race-gender of respondents relative to race, gender, and race-gender of interviewers.

Measures

A separate scale was constructed to measure support for AA and support for AHC. AA describes any measure, beyond simple termination of a discriminatory practice, adopted to correct or compensate for past or present discrimination, or to prevent discrimination from recurring in the future (Office of Federal Contracts Compliance Programs [OFCCP], 2002; U.S. Commission on Civil Rights, 1977). Items used to measure support for AA in this study (see Table 2), were adapted from Parra (1991). We defined AHC as a program intended to ensure that every American had an affordable health care insurance through an employer or through government. We were unable to secure any existing scale that measured support for AHC, so we constructed one based on general knowledge of the bill, as commonly expressed in popular media. We used general media knowledge about the bill because we assumed that most people in the public would not have read the actual scripts of the bill, and that they would have formed opinions on the bill mainly based on information (accurate or not) obtained from the media. We justified this approach to scale construction because we deemed it sufficient for the objective of this study, which was the measurement of change in answers to interview questions based on race and gender phily matches.

Table 2.

Correlation Matrix and Factor Analysis for the Scales of Dependent Variables.

	Item-by-item correlation matrix						Factor analysis
Items	Item 1	Item 2	Item 3	Item 4	Item 5	Item 6	Component 1
Scale: Support for Affirmative Action
Item 1							.852
Item 2	.478						.690
Item 3	.773	.523					.880
Item 4	.704	.557	.802				.884
Item 5	.715	.495	.686	.690			.839
Item 6	.543	.554	.626	.638	.544		.779
Item 7	.688	.493	.700	.702	.708	.645	.856
Item 1. Affirmative action is good in general. Item 2. Minority job applicants should be given special treatment in the hiring process. Item 3. Businesses should use affirmative action to ensure fairness in employment. Item 4. Affirmative action results in better utilization of human potentials in society. Item 5. Affirmative action is good for addressing continuing discrimination against minorities. Item 6. Affirmative action should be used to correct past injustices. Item 7. Affirmative action is good for bridging the gap among all races.
Scale: Support for Health Choices Act
Item 1							.686
Item 2	.865						.894
Item 3	.601	.640					.811
Item 4	.734	.772	.757				.902
Item 5	.588	.639	.662	.760			.810
Item 6	.711	.716	.610	.706	.641		.869
Item 7	.721	.744	.646	.707	.621	.850	.878
Item 1. Government should pay for health care for everyone Item 2. Government should provide health insurance for everyone Item 3. Government should provide health insurance to compete with private insurance Item 4. Government should provide health insurance that covers prescription medicine Item 5. Government should provide health insurance that covers pre-existing conditions Item 6. Government health insurance is in our best interest Item 7. Government health care coverage is the best health care reform for the country.

Each scale was summated rating Likert-type, and contained seven fully structured, closed-ended items, with response options ranging from 6 (strongly agree) to 1 (strongly disagree). Higher values indicated greater support for each variable. Closed ended questions were used so as to generate data with quantitative properties for statistical analysis.

Reliability statistics yielded Cronbach’s alpha = .923 for the scale of support for AA and .942 for the scale of support for health care reform (see Table 2). Factor analysis, using principal component extraction method, showed a strong internal consistency under one component for the items on each scale. Factor loading coefficients for all AA items ranged from .690 to .884, and they ranged from .686 to .902 for all items of AHC (see Table 2 for complete information on each scale). Other items on the questionnaire were nominal questions on the race (Black or White) and gender (man or woman) of both the interviewer and respondents.

Data

Twenty-nine university students (Whites = 62% [n = 18], Blacks = 38% [n = 11], men = 55% [n = 16], women = 45% [n = 13], White men = 34% [n = 10], White women = 28 % [n = 8], Black men = 21% [n = 6], Black women = 17 % [n = 5]) served as survey interviewers for this research. To enhance the consistence in their performance, all the interviewers were assembled and trained in interviewing data collection. The training included guidelines for politeness when requesting someone to be interviewed, and readily granting right of non-participation to anyone not willing to participate in the study. Interviewers were also instructed to promise anonymity to all willing participants. During each interview, the interviewers were instructed to only read interview questions verbatim as structured on the questionnaire, and to record respondents’ answers exactly as stated by respondents, and in conformity with the closed-ended response format of each question. Interviewers were instructed not to explain, interpret, or discuss any questionnaire item to research participants. Interviewers could only repeat (reread) questions, if research participants wanted further information on a questionnaire item.

Interview sites were public places with high foot traffic in the city of Omaha, Nebraska. These locations were the four major city malls as well as strip malls, big department and drug stores such as Wal-Mart, Target, Shopko, and Walgreens. The malls and stores spanned across all regions (North, South, East, and West) in the city, and they drew patrons from all areas of the city. The patrons of these locations were solicited by our trained interviewers for voluntary participation in this study. The interviewers went in self-selected teams of three or four people to each location to provide support for one another as might be necessary. We also anticipated that the mere presence of other interviewers (i.e., team members) at a location, at the same time, would encourage each interviewer to perform as instructed during training. Each team conducted interviews in, at least, two time periods (morning, afternoon or evening) in, at least, two different sites, during an 8-week period in fall 2010.

Consistent with the objective of this study, all 29 interviewers were either Black or White, and they were instructed to interview only Black and White voluntary participants. Of 580 questionnaires distributed to the 29 interviewers, a total of 522 questionnaires (90%) were completed. Thirty-one completed questionnaires of respondents who identified themselves as neither Black, nor White were discarded for not being the target population for this study. This left a useful return rate of 85% (491 questionnaires). Of the 491 useful interviews, 69% (n = 338) were completed by White interviewers, while Black interviewers completed 31% (n = 153) of the interviews. Men and women interviewers completed 63% (n = 310) and 37% (n = 181), respectively, of the interviews. Counting by race-gender, White men, White women, Black men and Black women interviewers completed 43% (n = 209), 26% (n = 129), 21% (n = 101), and 11% (n = 52), respectively, of useful interviews. The participants were mainly White (68%, n = 329) and men (54%, n = 261). White men constituted 35% of all respondents, whereas White women, Black men, and Black women constituted 34%, 19%, and 13%, respectively, of all 491 respondents.

Tests and Findings

ANOVA coefficients with the Bonferroni/Dun post hoc comparisons, at alpha = .05, were used to measure significant differences in support levels for AA and AHC under various phily matches of race, gender, and race-gender of respondents and interviewers. ANOVA comparisons indicated specific conditions (phily matches) under which interview responses changed for any specific respondent category, and post hoc results showed the extent of significant change in interview responses between any two categories of respondents vis-à-vis their respective categories of interviewers. Findings for each test are reported under each phily area as follows:

Race Phily

Interview response differences under four possible race-phily conditions (two homophilies and two heterophilies) were analyzed (Table 3). ANOVA coefficient showed that the Black homophily (Black respondent/Black interviewer) produced the highest levels of support for both AA (b = 3.891) and AHC (b = 3.837), but when interviewed by Whites (race heterophily), Black respondents lowered their support for both dependent variables (AA, b = 3.191; AHC, b = 2.484). Unlike Blacks, White respondents demonstrated negative support for both dependent variables, especially in the White homophily (White respondent/White interviewer) condition (AA, b = −4.443; AHC, b = −3.193). When interviewed by Blacks (race heterophily), however, Whites were less negative in their support for both dependent variables (AA, b = −2.640; AHC, b −3.128).

Table 3.

Differential Levels of Support for Affirmative Action and America’s Affordable Health Choices Act by Phily Conditions.

ANOVA coefficient
Respondent	Interviewer	n	M	b	t value	p value
Support for affirmative action
Race phily
Intercept				25.821	61.564	.000
White	White	238	21.4	−4.443	−7.948	.000
White	Black	83	23.2	−2.640	−3.505	.000
Black	White	84	29.0	3.191	4.254	.000
Black	Black	66	29.7	3.891
Main effect: df = (3, 467) 470, F = 30.422, λ = 91.265, p = .0000
Gender phily
Intercept				23.913	58.073	.000
Men	Men	172	24.3	.366	.591	.555
Women	Men	126	26.7	2.817	4.149	.000
Women	Women	91	24.0	.109	.144	.886
Men	Women	82	20.6
Main effect: df = (3, 467) 470, F = 8.437, λ = 25.312, p = .0000
Support for the Affordable Health Choices Act
Race philies
Intercept				28.948	59.551	.000
White	White	241	25.76	−3.193	−4.932	.000
White	Black	83	25.82	−3.128	−3.128	.000
Black	White	88	31.43	2.484	2.895	.004
Black	Black	65	32.79	3.837
Main effect: df = (3, 473) 476, F = 15.557, λ = 46.672, p = .0000
Gender philies
Intercept				27.390	58.928	.000
Men	Men	174	28.58	1.185	1.696	.091
Women	Men	129	29.49	2.099	2.749	.006
Women	Women	90	26.73	−.656	−.762	.447
Men	Women	84	24.76	−2.628
Main effect: df = (3, 473) 476, F = 14.260, λ = 4.753, p = .0028

The Bonferroni/Dun post hoc race-phily analyses (α = .0083 for α = .05) showed that the strongest significant difference in support for AA and AHC was the homophily comparison. The comparison revealed that White homophily supported the two dependent variables significantly lesser than the Black homophily (M difference = 8 points for AA and 7 points for AHC). Except for the White homophily versus White-respondent/Black-interviewer heterophily and the Black-respondent/White-interviewer heterophily versus Black homophily comparisons that were not significantly different in their support for both dependent variables, all other race-phily comparisons showed significant differences in support of both AA and AHC (see Table 4 for full result).

Table 4.

Post Hoc Results for Race and Gender Phily Comparisons for Support for Affirmative Action (AA) and America’s Affordable Health Choices Act (AHC) Using the Bonferroni/Dunn Significance α = .0083 (for α = .05).

	M difference	Critical difference	p value
Affirmative action
Race philly comparisons
White resp/White inter vs. White resp/Black inter	−1.803	2.723	.080
White resp/White inter vs. Black resp/White inter	−7.634	2.711	.000*
White resp/White inter vs. Black resp/Black inter^a	−8.334	2.971	.000*
White resp/Black inter vs. Black resp/White inter	−5.831	3.306	.000*
White resp/Black inter vs. Black resp/Black inter	−6.531	3.523	.000*
Black resp/White inter vs. Black resp/Black inter	−0.700	3.513	.598
Gender phily comparisons
Men resp/men inter vs. women resp/men inter	−2.451	2.663	.015
Men resp/men inter vs. women resp/women inter^a	0.257	2.944	.817
Men resp/men inter vs. men resp/women inter	3.657	3.047	.002*
Women resp/men inter vs. women resp/women inter	2.708	3.124	.022
Women resp/men inter vs. men resp/women inter	6.108	3.222	.000*
Women resp/women inter vs. men resp/women inter	3.400	3.458	.010
Affordable health choices act
Race philly comparisons
White resp/White inter vs. White resp/Black inter	−.064	3.164	.957
White resp/White inter vs. Black resp/White inter	−5.677	3.096	.000*
White resp/White inter vs. Black resp/Black inter^a	−7.029	3.474	.000*
White resp/Black inter vs. Black resp/White inter	−5.613	3.804	.000*
White resp/Black inter vs. Black resp/Black inter	−6.965	4.117	.000*
Black resp/White inter vs. Black resp/Black inter	−1.353	4.066	.379
Gender phily comparisons
Men resp/men inter vs. women resp/men inter	−0.914	2.995	.419
Men resp/men inter vs. women resp/women inter^a	1.841	3.347	.146
Men resp/men inter vs. men resp/women inter	3.813	3.425	.003*
Women resp/men inter vs. women resp/women inter	2.755	3.540	.040
Women resp/men inter vs. man resp/women inter	4.726	3.614	.000*
Women resp/women inter vs. man resp/women inter	1.971	3.911	.182

Homophily comparisons, resp = respondent, inter = interviewer.

Gender Phily

Like race-phily analyses, four possible gender phily conditions (two homophilies and two heterophilies) were also analyzed for shifts in support levels for both AA and AHC (Table 3). ANOVA coefficient showed that gender phily was a significant factor of support for both AA (b = 2.817) and AHC (b = 2.099) only when women were interviewed by men (gender heterophily). All other gender philies did not produce significant shifts in support for both dependent variables.

The Bonferroni/Dun post hoc gender phily analyses (α = .0083 for α = .05) showed that the largest significant difference in support for both AA and AHC was in the women-respondent/men-interviewer versus men-respondent/women-interviewer heterophily comparison. The former phily outscored the latter by 6.1 points for AA and 4.7 points for AHC. Results also showed that the men homophily gave greater support for AA and AHC (M difference = 3.65 and 3.81, respectively) than the men who were interviewed by women (men respondent/women interviewer heterophily comparisons). See Table 4 for result details on post hoc gender comparisons.

Race-Gender Phily

Interview response patterns under all possible matches (4 homophily and 12 heterophily) were analyzed for race-gender differences in support for both dependent variables (Table 5). ANOVA coefficients showed that White men displayed negative attitude toward both programs, regardless of the race-gender of their interviewers. Statistically significant results indicated that White men were most negative about both AA and AHC when interviewed by White women (AA, b = −8.161; AHC, b = −6.462), but they (White men) lessened their negative toward AA in the White men homophily (b = −5.808) and toward AHC when interviewed by Black men (b = −4.305). In all other results involving White men as respondents, the shifts in their support for both dependent variables were not statistically significant.

Table 5.

ANOVA Coefficients for Differential Support for Affirmative Action and the America’s Affordable Health Choices Act by Race-Gender Phily Conditions.

Phily conditions
Respondent	Interviewer	n	M	b	t value	p value
Affirmative action support
Intercept				25.613	56.473	.000
White men^a	White men	72	19.8	−5.808	6.020	.000
White men	White women	42	17.5	−8.161	−6.780	.000
White men	Black men	36	23.3	−2.280	−1.772	.080
White men	Black women	10	23.8	−1.813	−.778	.437
White women^a	White women	54	23.0	−2.632	−2.430	.016
White women	White men	66	24.1	−1.477	−1.479	.140
White women	Black men	25	23.7	−1.893	−1.250	.212
White women	Black women	14	21.8	−3.827	−1.930	.054
Black men^a	Black men	30	29.9	4.254	3.049	.002
Black men	Black women	9	26.8	1.165	.475	.635
Black men	White men	32	30.3	4.731	3.490	.000
Black men	White women	19	22.8	−2.824	−1.643	.101
Black women	White men	26	32.3	6.695	4.500	.000
Black women	White women	7	27.6	1.958	.707	.480
Black women	Black men	10	35.2	9.587	4.116	.000
Black women^a	Black women	16	27.9	2.324
Main effect: df = (15, 452) 467, F = 10.046, λ = 150.689, p = .0000
Affordable Health Choices Act support
Intercept				26.676	52.795	.000
White men^a	White men	74	26.6	−2.095	−1.846	.066
White men	White women	42	22.2	−6.462	−4.518	.000
White men	Black men	35	24.4	−4.305	−2.781	.006
White men	Black women	11	27.4	−1.313	−.497	.620
White women^a	White women	55	26.0	−2.676	−2.095	.037
White women	White men	66	27.0	−1.722	−1.450	.148
White women	Black men	25	26.2	−2.476	−1.376	.169
White women	Black women	14	28.2	−.462	−.196	.845
Black men^a	Black men	31	32.4	3.679	2.253	.025
Black men	Black women	9	32.7	3.990	1.372	.171
Black men	White men	32	34.3	5.605	3.481	.001
Black men	White women	20	24.7	−3.976	−1.995	.047
Black women	White men	29	34.8	6.117	3.636	.000
Black women	White women	6	23.7	−5.010	−1.414	.158
Black women	Black men	10	37.6	8.924	3.227	.001
Black women^a	Black women	14	30.9	2.181
Main effect: df = (15,457) 472, F = 5.657, λ = 84.849, p = .0000

Homophilies.

Like White men, White women as respondents also demonstrated a pattern of negative attitude toward both government programs, regardless of the race-gender of their interviewers. The only significantly different response to both dependent variables, however, was the White women homophily (AA, b = −2.632; AHC, b = −2.676). Also, the answers given by White women to Black women interviewers on support for AA approached statistical significance (b = −3.827, p = .054). All other phily matches involving White women as respondents did not produce statistically significant shifts in their support for both programs.

Results for Black men as respondents showed that Black men, as a pattern, responded positively toward both dependent variables. ANOVA coefficients revealed that Black men most positively supported both AA and AHC when interviewed by White men (AA, b = 4.731; AHC, b = 5.605), but they (Black men) slightly lowered their support for both dependent variables in the Black men homophily (AA, b = 4.254; AHC, b = 3.679). Results also showed that when interviewed by White women, Black men displayed significant negative support for AHC (b = −3.976).

Like Black men, Black women were predominantly positive in their response to both dependent variables. For both AA and AHC, Black women were most significantly positive when interviewed by Black men (AA, b = 9.587; AHC, b = 8.924), followed by a lowered, but positive support, when interviewed by White men (AA, b = 6.695; AHC, b = 6.117). All other phily matches involving Black women as respondents failed to show significant shifts in response to both dependent variables by Black women.

The Bonferroni/Dun post hoc analysis (α = .0004 for α = .05) calculated 120 possible separate post hoc race-gender phily comparisons for each dependent variable. Of the 120 phily comparisons for each dependent variable, approximately 23% (n = 27) and 15% (n = 18) of phily comparisons for AA and AHC, respectively, indicated significant differences in interview response. All phily comparisons that indicated significant differences in support for both dependent variables are displayed in Table 6.

Table 6.

Post Hoc Results for Race-gender/Race-gender Phily Comparisons for Support for Affirmative Action (AA) and America’s Affordable Health Choices Act (AHC).

	Affirmative action		Health choices act
Race-gender phily comparison	M difference	Critical difference	M difference	Critical difference
WTM resp/WTM inter vs. BKM resp/WTM inter	−10.538	5.835	−7.700	6.896
WTM resp/WTM inter vs. BKW resp/WTM inter	−12.502	6.284	−8.212	7.140
WTM resp/WTM inter vs. BKM resp/BKM inter^a	−10.061	5.968
WTM resp/WTM inter vs. BKW resp/BKM inter	−15.394	9.268	−11.019	10.981
WTM resp/WTM inter vs. BKW resp/BKW inter^a	−8.132	7.591
WTM resp/WTW inter vs. BKM resp/BKM inter	−12.414	6.565	−10.141	7.717
WTM resp/WTW inter vs. BKW resp/BKM inter	−17.748	9.664	−15.386	11.468
WTM resp/WTW inter vs. BKW resp/BKW inter	−10.485	8.068
WTW resp/BKM inter vs. BKW resp/BKM inter	−11.867	9.817	−13.229	11.687
WTW resp/BKM inter vs. BKW resp/BKM inter	−11.480	10.276
WTW resp/WTW inter vs. BKM resp/BKM inter^a	−6.885	6.254
WTW resp/WTW inter vs. BKW resp/BKM inter	−12.219	9.455	−11.600	11.205
WTW resp/WTM inter vs. BKM resp/WTM inter	−6.207	5.916	−7.327	7.021
WTW resp/WTM inter vs. BKW resp/WTM inter	−8.171	6.359	−7.839	7.261
WTW resp/WTM inter vs. BKW resp/BKM inter	−11.064	9.320
WTW resp/WTM inter vs. WTM resp/WTW inter	6.684	5.421
BKM resp/WTW inter vs. BKW resp/BKM inter	−12.411	10.730	−12.900	12.623
BKM resp/WTM inter vs. WTM resp/WTW inter	12.891	6.444	12.067	7.648
BKM resp/WTM inter vs. WTW resp/WTW inter	7.362	6.127	8.281	7.246
BKM resp/WTM inter vs. BKW resp/WTW inter			9.581	9.290
BKM resp/WTM inter vs. WTM resp/BKM inter	7.010	6.672	9.910	7.972
BKW resp/WTM inter vs. WTM resp/WTW inter	14.855	6.853	12.579	7.869
BKW resp/WTM inter vs. WTW resp/WTW inter	9.326	6.556	8.793	7.480
BKW resp/WTM inter vs. BKM resp/WTW inter	9.518	8.289	10.093	9.473
BKW resp/WTM inter vs. WTM resp/BKM inter	8.974	7.068	10.422	8.184
BKW resp/WTM inter vs. WTW resp/BKM inter	8.588	7.693
BKW resp/WTM inter vs. WTW resp/BKW inter	10.522	9.104
BKW resp/BKM inter vs. WTW resp/BKW inter	13.414	11.371

Note. All reported values are significant at Bonferroni/Dunn significance α = .0004 (for α = .05).

Homophily comparisons, WTM = White men, WTW = White women, BKM = Black men, BKW = Black women, resp = respondent, inter = interviewer.

Post hoc comparison showed that White men interviewed by White women were least likely to support both dependent variables when compared with Black women interviewed by Black men (17.7 points lesser in support of AA, and 15.4 points lesser in support of AHC). The least difference in phily comparisons for White men respondents for AA showed that the White men homophily was 8.1 points lesser in support for AA than the Black women homophily. For support of AHC, the least comparison difference involving White men showed that White men homophily was 7.7 points lesser than Black men interviewed by White men.

Like White men, all race-gender phily comparisons involving White women as respondents showed White women supporting both AA and AHC lesser than Black men and Black women as respondents. Post hoc comparisons showed that White women were least likely to support AA when the White women homophily was compared with Black women interviewed by Black men (12.2 points lesser for White women than Black women respondents). White women were also least likely to support AHC when White women interviewed by Black men were compared with Black women interviewed by Black men (13.2 points differential). The closest gap in comparison of any two phily matches in support of both AA and AHC occurred when White women interviewed by White men were compared with Black men interviewed by White men (AA = 6.2 points differential, AHC = 7.3 points differential).

Race-gender post hoc comparisons involving Black men showed that Black men interviewed by White men, supported AA by 12.89 points more than White men interviewed by White women. When interviewed by White women, however, Black men supported AA and AHC by 12.4 points and 12.9 points, respectively, which was lesser than Black women interviewed by Black men. The closest gap in support for both AA and AHC among all significant phily comparisons involving Black men as respondents occurred when Black men interviewed by White men, supported AA by 7.0 points more than White men interviewed by Black men. For support of AHC, Black men interviewed by White men scored 8.28 points more than the White women homophily.

The pattern of post hoc interview response analyses involving Black women showed that Black women interviewed by White men scored 14.85 points and 12.57 points higher support for AA and AHC, respectively, than White men interviewed by White women. Black women interviewed by Black men also displayed 13.4 points greater support for AA than White women interviewed by Black women. The closest gap between any two phily comparisons for AA was the 8.58 points higher support for AA by Black women interviewed by White men over White women interviewed by Black men. Similarly, the closest gap in phily comparisons was the 8.79 points higher support for AHC by Black women interviewed by White men over the White women homophily. See details of all differences in interview responses to both AA and AHC by race-gender phily post hoc comparisons in Table 6.

Conclusion

This study comprehensively analyzed, beyond the parameters covered by previous studies on the topic, the likelihood that interviewer effects will be consequential (or not) to data quality based on homo- and heterophily interview conditions between interviewers and respondents. By using ANOVA tests, in this study, we sought to examine the extent to which interview responses would differ by race, gender, and race-gender of interviewers and the phily conditions under which such differences would occur to determine consequentiality of race, gender, and race-gender effects on data quality.

ANOVA comparisons were preferred for analysis over ordinary least squares (OLS), particularly, hierarchical regression used in similar studies (e.g., Gong & Aadland, 2011) because of the challenges posed by the number of comparisons necessitated in this study. For one, because of the comprehensiveness of race, gender, and race-gender variables in 24 different phily conditions we analyzed for response differences, the nature of the group differences in our analyses amounted to a 2 × 2 × 2 × 2 design with a four-way interaction. Interpreting four-way interactions are notoriously complicated, which often pose additional challenges in the presentation of findings for clarity. Also, the 2 × 2 × 2 × 2 design complicates the creation of interaction terms in OLS regression because the z scores amount to −1 and 1, and multiplying these (using coded contrasts) provides little clarity. Even with a concerted attempt to analyze the data using multilevel modeling by nesting the respondents into individual groups, the problem of limited number of degrees of freedom hindered such analysis in OLS, and hence, making the 2 × 2 × 2 × 2 ANOVA the best analysis for our data and research questions. Another hindrance to multilevel nesting of interviewees in interviewers is that our data were not recorded to allow linking particular set of interviews with particular interviewers due to over conformity to confidentiality and anonymity rights of both interviewers and respondents. The data collection method was however, deliberate because the objective of this study was to test for mean differences in interview response by phily matches. Our research objective, therefore, justifies our choice of data collection process and analysis.

Our findings across all phily comparisons indicated a general pattern of nonrandom (i.e., structured) interview response, hence, confirming consequentiality of interviewer effects on data quality. Our findings demonstrated a pattern of race, gender, and race-gender of interviewer effects based on patterns homo- and heterophilous matches between interviewer and respondents. For example, the White homophily showed a higher negative value for both AA and AHC than the White respondent/Black interviewer heterophily (see Table 3). Similarly, the Black homophily showed a higher support for both dependent variables than the Black respondent/White interviewer heterophily. Post hoc comparisons also confirmed that the largest difference in support for both dependent variables was between the White homophily and the Black homophily, and other differences in phily comparisons became smaller as the comparisons became more heterophilous. These findings support earlier findings that respondents tended to acquiesce to the race of interviewer (D. W. Davis, 1997a, 1997b; Ellison et al., 2011; Krysan & Couper, 2003), and Whites tended to be more liberal and politically correct (R. E. Davis et al., 2010; Finkel, Guterbock, & Borg, 1991; Gong & Aadland, 2011; Krysan & Couper, 2003), as well as reluctant to oppose racially centered policy when interviewed by Blacks (Krysan & Couper, 2003). Our finding that gender difference in support for both dependent variables was significant only when women were interviewed by men (Tables 3 and 4), indicated that men had significant effects on how women answered questions, but not vice versa.

Our race-gender phily analyses specified particular patterns of race-gender influences of both respondents and interviewers on interview responses. We found that White men gave significantly different answers (higher negative response) to AA when interviewed by White women and fellow White men, but no significant difference was found when White men were interviewed by Black men and Black women. White men also gave significantly different answers to AHC when interviewed by White women (higher negative response) and Black men (tempered negative response) but gave no significantly different responses when interviewed by White men and Black women. White women gave significantly different (negative) answers to AA and AHC only to White women interviewers, but they gave no significantly different responses to other race-gender categories of interviewers. Results also revealed that Black men gave significantly different answers (positive response) to Black men and White men interviewers but gave no significantly different responses to White women and Black women interviewers on support for AA. For AHC, Black men gave significantly different answers (positive response) to Black men, White men and White women (negative response) interviewers but gave no significantly different response to Black women interviewers. Last, Black women gave significantly different (positive) answers to both dependent variables only when interviewed by White men and Black men.

This study made some important contributions to existing literature on the effects of extraneous variables on data quality. Unlike previous studies that focused mainly on interviewer effects, this study paid attention to both interviewer and respondent’s characteristics. Also, unlike most previous studies that focused only on one interviewer effect characteristic (e.g., race only, gender only, etc.) this study was based on multiple characteristics (race [Black and White], gender, and race-gender) of both respondents and interviewers within the context of a single study. This study, therefore, gives a more comprehensive analyses of race, gender, and race-gender effects on data quality in one single study than found in many (if not any) studies on the same topic. Our analysis of race-gender effects (which were mostly absent in previous studies) are especially important because every individual is a combination of both race and gender, and neither is separable from the other in real-life interactions. In addition, post hoc values showed specific differences in interview response between any two phily categories, thereby providing knowledge on specific directions and magnitude of response difference in interview response based on phily matches. We add that this study also provide empirical confirmation to speculations in popular media that support for the America’s Health Choices Act of 2009 was race sensitive, even though the language of the bill was race neutral, unlike the AA policy.

Overall, this study reaffirms the significance of the need to pay attention to race, gender, and race-gender effects on data quality during data collection. Both data quality and research integrity are at stake to the extent that race and gender of respondents and interviewers influence data quality when research participants conform their answers to accommodate the interviewer, thereby exaggerating or tempering the relationships between variables (D. W. Davis, 1997b; Sherman, 2002). Given that findings of social science research carry significant implications for practice and generalizations about society, it is important for data collectors to devise methods to assess and nullify race and gender effects in their data and analyses.

Limitations

We recognize the following limitations to the findings of this study:

There was an imbalance in the racial and gender compositions of the survey interviewers, and a much larger imbalance in the racial and gender compositions of the respondents by each interviewer category. There were fewer Black respondents than White respondents, and the average numbers of cross-race interviews per person were much smaller than for same-race interviews for White interviewers. Black interviewers had a much higher per capita cross-race interviews (though collectively they completed much lesser amount of interviews) than White interviewers. Black women interviewers and Black women respondents were also much fewer than other race-gender combinations of interviewers and respondents. The reasons (beyond the scope of this study) for the imbalance in cross-race interviews were unclear, but we suspected that the imbalance might have potentially been a function of lingering uncomfortable race relations sentiments in the city. Because of traditional tension and discomfort in cross-race relations in the city (and the United States in general), White interviewers in this study might have been less inclined to readily approach Blacks for interview; hence, they (White interviewers) turned inwardly and interviewed more Whites. For Back interviewers, it might have been a bit easier for them to solicit Whites for interviews because they have always lived in a world dominated by Whites, and might have therefore, developed a greater level of comfort than Whites in engaging in cross-racial activities. Another explanation for the imbalance in cross-race interviews might be that because the interview was conducted in Omaha, Nebraska, a predominantly White city, Blacks would have more opportunities to interview Whites (because of the much larger White population) than vice versa. This means that the imbalance in cross-race interview could have been more of a function of structural factors of population distribution than tensions of race relations.

It was mentioned in literature review that some studies had linked variables like age and education to differences in interview response. The focus in this study was strictly on race and gender phily analyses beyond the scope of any previous study, hence, we did not control for the potential influences of other possible variables. While the omission of other variables in our analyses might be a source of limitation to our results, it conversely helped to make our analyses streamlined, manageable, and focused on our research objective.

Multilevel nesting of interviewees in interviewers would have been ideal for data analyses had data been collected in such a way to permit such analyses. Nesting requires linking a particular interviewer to a particular set of interviewees. An overly strict adherence to the privacy and anonymity rights of both interviewers and interviewees prevented necessary identification of particular set of interviews with particular interviewer to enable nesting. Data were, however, collected consistently with the objective of testing mean differences in interview response by phily matches using ANOVA.

The Bonferroni correction, though necessary because of multiple post hoc tests to reduce the likelihood that any set of group comparisons would produce a Type I error, is overly strict and might have hidden some potential significant group differences in our post hoc comparisons.

Footnotes

Acknowledgements

The authors wish to express gratitude to Ms. Patrice Henderson, former sociology student at the University of Nebraska at Omaha, and all the student interviewers for their generous assistance and contributions to the success of this study. We also express deep gratitude to Dr. Jonathan Santos, Department of Psychology, University of Nebraska, Omaha, for his advice and review of the statistical portion of this study.

Authors’ Note

Mr. Losen was an undergraduate student and a mentee of the first author at the University of Nebraska at Omaha.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research and/or authorship of this article.

Notes

Author Biographies

A. Olu Oyinlade is an industrial/organizational sociologist, applied demographer, and professor of sociology at the University of Nebraska, Omaha. His research and consulting activities are in organization behavior and development with a special focus on employee performance improvement, change management, leadership assessment, program evaluation, and diversity planning and management.

Alex Losen earned a BS with a focus in Organizational Sociology from the University of Nebraska, Omaha. He is finishing a management training program with a large corporation, Consolidated Electrical Distributors, as well as remains an independent scholar.

References

Alderfer

C. P.

Tucker

R. C.

(1996). A field experiment for studying race relations embedded in organizations. Journal of Organizational Behavior, 17, 43-57.

Allen

Dawson

Brown

(1989). A schema-based approach to modeling an African American racial belief system. American Political Science Review, 83, 421-441.

Anderson

W. T.

Jr. Alpert

M. I.

(1974). Optimal heterophily: Aid to effective communication. Public Personnel Management, 3, 279-288.

Collins

Butcher

(1983). Interviewer and clustering effects in an attitude survey. Journal of the Market Research Society, 25, 39-58.

Congress.Gov. (2009). America’s Affordable Health Choices Act of 2009: HR3200. Retrieved from http://beta.congress.gov/bill/111th-congress/house-bill/3200

Davis

D. W.

(1997a). The direction of race of interviewer effects among African-Americans: Donning the black mask. American Journal of Political Science, 41, 309-322.

Davis

D. W.

(1997b). Nonrandom measurement error and race of interviewer effects among African Americans. Public Opinion Quarterly, 61, 183-207.

Davis

D. W.

Silver

B. D.

(2003). Stereotype threat and race of interviewer effects in a survey on political knowledge. American Journal of Political Science, 47, 33-45.

Davis

R. E.

Couper

M. P.

Janz

N. K.

Caldwell

C. H.

Resnicow

(2010). Interviewer effects in public health surveys. Health Education Research, 25, 14-26.

10.

Durrant

G. B.

Groves

R. M.

Staetsky

Steele

(2010). Effects of interviewer attitudes and behaviors on refusal in household survey. Public Opinion Quarterly, 74, 1-36.

11.

Dykema

Diloreto

Price

J. L.

White

Schaeffer

N. C.

(2012). ACASI gender-of-interviewer voice effects on reports to questions about sensitive behaviors among young adults. Public Opinion Quarterly, 76, 311-325.

12.

Ellison

C. G.

McFarland

M. J.

Krause

(2011). Measuring religiousness among African Americans: Exploring race-of-interviewer effects. Review of Religious Research, 53, 65-84.

13.

Finkel

S. E.

Guterbock

T. M.

Borg

M. J.

(1991). Race-of-interviewer effects in a pre-election poll: Virginia 1989. Public Opinion Quarterly, 55, 313-330.

14.

Flores-Macias

Lawson

(2008). Effects of interviewer gender on survey responses: Findings from a household survey in Mexico. International Journal of Public Opinion Research, 20, 100-110,

15.

Ghane

Kolk

A. M.

Emmelkamp

P. M. G.

(2010). Assessment of exploratory model of mental illness: Effects of patient and interviewer characteristics. Social Psychiatric Epidemiology, 45, 175-182.

16.

Gong

Aadland

(2011). Interview effects in an environment valuation telephone survey. Environmental & Resource Economics, 49, 47-64.

17.

Huddy

Billig

Bracciodieta

Hoeffler

Moynihan

P. J.

Pugliani

(1997). The effects of the interviewer gender on the survey response. Political Behavior, 19, 197-220.

18.

Januszka

Lora

Wollard

Rocco

(2007). Investigating race and ethnicity on data collection and analysis. In Nielsen

S. M.

Plakhotnik

M. S.

(Eds.), Proceedings of the Sixth Annual College of Education Research Conference: Urban and International Education Section (pp. 36-42). Miami: Florida International University. Retrieved from http://digitalcommons.fiu.edu/cgi/viewcontent.cgi?article=1258&context=sferc

19.

Kane

E. W.

Macaulay

J. A.

(1993). Interviewer gender and gender attitudes. Public Opinion Quarterly, 57, 1-28.

20.

Konrad

A. M.

Spitz

(2003). Explaining demographic group differences in affirmative action attitudes. Journal of Applied Social Psychology, 33, 1618-1642.

21.

Krysan

Couper

M. P.

(2003). Race in the live and the virtual interview: Racial deference, social desirability, and activation effects in attitude surveys. Social Psychology Quarterly, 66, 364-383.

22.

Lange

J. W.

(2002). Methodological concerns for non-Hispanic investigators conducting research with Hispanic Americans. Researching in Nursing & Health, 25, 411-419.

23.

Lievens

De Paepe

(2004). An empirical investigation of interviewer-related factors that discourage the use of high structure interviews. Journal of Organizational Behavior, 25, 29-46.

24.

Liu

Stainback

(2013). Interviewer gender effects on survey responses to marriage-related question. Public Opinion Quarterly, 77, 606-618.

25.

Office of Federal Contracts Compliance Programs. (2002). Facts on Executive Order 11246-Affirmative Action. U.S. Department of Labor. Retrieved from http://www.dol.gov/ofccp/regs/compliance/aa.htm

26.

Oppenheim

A. N.

(1992). Questionnaire design, interviewing and attitude measurement. London, England: Printer Publishers.

27.

Oyinlade

A. O.

(2013). Affirmative action support in an organization: A test of three demographic models. SAGE Open. doi:10.1177/2158244013516156

28.

Park

J. J.

(2009). Taking race into account: Charting students attitudes towards affirmative action. Research in Higher Education, 50, 670-690.

29.

Parra

L. F.

(1991). Development of an Attitudes Toward Affirmative Action Scale (MA thesis). Department of Psychology, Bowling Green University, OH.

30.

Reese

L. A.

Brown

R. E.

(1995). The effects of religious messages on racial identity and system blame among African Americas. Journal of Politics, 57, 24-43.

31.

Schuman

Converse

J. M.

(1971). The effects of black and white interviewers on black responses in 1968. Public Opinion Quarterly, 35, 44-68.

32.

Sherman

(2002). The subjective experience of race and gender in qualitative research. The American Behavioral Scientist, 45, 1247-1253.

33.

Smith

W. A.

(1998). Gender and racial/ethnic differences in the affirmative action attitudes of US college students. Journal of Negro Education, 67, 127-141.

34.

Springman

Wherry

Notaro

P. C.

(2006). The effects of interviewer race and child race on sexual abuse disclosures in forensic interviews. Journal of Child Sexual Abuse, 15, 99-116,

35.

Tabane

R. J.

Bouwer

(2006). The influence of cross-cultural interviewing on the generation of data. South African Journal of Education, 26, 553-564.

36.

U.S. Commission on Civil Rights. (1977). Statement on affirmative action. Washington, DC: U.S. Government Printing Office.

37.

U.S. Commission on Civil Rights. (1981). Affirmative action in the 1980s: Dismantling the process of discrimination: A proposed statement of the United States Commission on Civil Rights (Clearinghouse Publication 65, U.S. Commission on Civil Rights). Washington, DC: U.S. Government Printing Office.

38.

Warren

C. A. B.

(1988). Gender issues in field research (Qualitative Research Methods Series). Thousand Oaks, CA: Sage.

39.

Webster

(1996). Hispanic and Anglo interviewer and respondent ethnicity and gender: The impact on survey response quality. Journal of Marketing Research, 34, 62-72.

40.

Weinreb

(2006). The limitations of stranger-interviews in rural Kenya. American Sociological Review, 71, 1014-1039.