Sage Journals: Discover world-class research

Abstract

The present study describes a culturally relevant approach to introductory psychology textbook selection for students attending a historically Black college/university (HBCU). The following multistage procedure was used: (1) a survey of HBCU psychology departments was conducted to ascertain how they selected their introductory psychology textbooks; (2) a review of the literature on introductory psychology textbook selection was performed to identify criteria; and (3) rating criteria were developed; used in the actual selection of a textbook for HBCU students; and were subjected to psychometric analysis. The psychometric analyses of 240 ratings (10 criteria X 4 raters X 6 books) involved both reliability assessment of the scale with the selection criteria as items and interrater reliability in evaluating introductory psychology textbooks. The survey of HBCU faculty yielded no consensus or systematic process from the respondents. Both internal consistency reliability and agreement among faculty were adequate on the rating instrument for the total scores and subscale scores of Quality of Learning Resources and Learning and Comprehension. However, the subscale scores for Cultural Relevance yielded mixed results by type of reliability: Interrater agreement was adequate but internal consistency reliability was poor. Future research using multiple methodologies must address the relevancy of cultural diversity for introductory psychology courses

Keywords

Cultural diversity general psychology historically Black college/university psychometric analysis textbook section

Cultural diversity is considered a major “cross-cutting theme” in the new proposed model for the teaching of introductory psychology, and multiculturalism is subsumed under the fourth pillar of psychological science addressing social and personality dimensions of human behavior (Gurung et al., 2016). Introductory psychology courses are ideal to promote understanding, tolerance, and appreciation for the lives and experiences of different ethnic and racial groups in a diverse society. Estimates indicate between one to two million students are enrolled in introductory psychology courses in the United States during the year (Habarth, Hansell, & Grove, 2011; Gurung et al., 2016). Introductory psychology is the second most popular course in many college curricula serving as both the foundation for the major and a prerequisite for a liberal arts undergraduate education (Gurung et al., 2016). In addition, Landrum et al. (2010) wrote,

Psychology educators need to ensure that students are acquiring sensitivity for and appreciation of sociocultural and international diversity for interpersonal success and possess the ability to function effectively in a global society. Psychology students need to know that, regardless of the boundaries between countries and cultures, exposure to different perspectives helps individuals to understand multiple approaches to investigating and explaining behavior, sometimes depending on cultural context.

(p. 153)

From the perspective of student learning, the textbook is one of the major components of introductory psychology courses (Gurung, Daniel, & Landrum, 2012; Gurung, Landrum, & Daniel, 2012). Thus, as Lonner (2016) asserted, one way to gauge how well cross-cultural research is integrated and diffused to general psychology students is to assess how it is handled in introductory psychology textbooks. An analysis of the cultural content of 35 introductory psychology textbooks, and a survey of 30 cross-cultural experts about what material should be included, converged to suggest that cultural diversity is underrepresented in these books (Lonner, 2016). Griggs’s (2014) updated objective analysis of 13 full-length introductory psychology textbooks revealed that diversity in terms of ethnicity or race was the only “nontraditional” topic without a chapter devoted to it. Thus a gap exists between theory and practice with regard to cultural content when it comes to the textbooks for introductory psychology courses. One could argue, however, that other strategies to promote cultural diversity compensate for its underrepresentation in introductory psychology textbooks.

Unfortunately, the literature on the teaching and learning of psychology also demonstrates that cultural diversity is not sufficiently diffused in introductory psychology through other mechanisms such as course offerings (Fuentes & Shannon, 2016), faculty lectures (Prieto et al., 2009), or published research on the teaching of psychology (Ocampo et al., 2003). A noteworthy omission in a survey of the membership of the Society for the Teaching of Psychology was that textbooks were not listed among the resources used to infuse diversity issues into the teaching of psychology (Prieto et al., 2009). Interestingly, Landrum and Hormel (2002) found that the longer an instructor had been teaching introductory psychology they were likely neither to consider cultural diversity an important attribute in the selection of a textbook nor to believe that it facilitated student learning. Thus the current literature highlights the need for increased attention to issues of ethnic and racial diversity in introductory psychology, especially the selection of textbooks.

Besides the fact that such efforts are consistent with the new teaching model of psychology (Gurung et al., 2016), the growing number of students of color in college populations reminds teachers of psychology about the importance of cultural diversity (Fuentes & Shannon, 2016; Prieto et al., 2009). The relevance of culture is particularly important for universities with populations that are predominantly students of color. As pointed out by Landrum et al. (2010), the cultural context may help individuals understand multiple approaches to investigating and explaining human behavior. We believe this to be true for teachers of psychology as well as their students. Students in psychology departments of the historically Black colleges and universities (HBCUs), which have been the focus of a recent White House initiative, represent the diversity of college campuses whose needs should be considered (Gasman, 2013). The HBCU context provides African American students with a social and academic environment that facilitates learning through cultural identification.

Consistent with this view, a longitudinal study indicated that students in science, technology, engineering, and mathematics (STEM) in HBCU environments, compared to predominantly White institutions (PWIs), tend to be more persistent in pursuit of their majors (Woodcock, Hernandez, Estrada, & Schultz, 2012). Such findings have implications for the teaching of psychology, which is also a STEM discipline (Gurung et al., 2016). Fuentes and Shannon (2016) recommended that psychology instructors consider the “changing demographics” of students in college settings and the impact of stereotype threat or identity threat on classroom dynamics. The concerns about racial stereotypes are substantially reduced among African American students attending HBCUs relative to PWIs (Chavous, Harris, Rivas, Helaire, & Green, 2004). Moreover, Smith and Hopkins (2004) suggested, after their failure to replicate the stereotype threat effects in a HBCU environment, “an experimental stereotype threat situation was not powerful enough to offset the students’ level of comfort at this university” (p. 319). Whaley (2012) demonstrated that cultural identification with the course material enhanced the learning of abnormal psychology among HBCU students. In conjunction, these findings suggest that the cultural relevance of introductory psychology courses may be especially significant in the HBCU environment. These findings point in the direction of making it a pedagogical priority to select a general psychology textbook with which African American students attending HBCUs could identify.

Thus, given the estimates of introductory psychology course enrollment and the shifting forces of society toward the accommodation of a more culturally diverse student body, the methods for the selection and adoption of textbooks for introductory psychology courses need to be reflective and relevant to this multicultural reality, particularly for students attending HBCUs. It was Wolfle’s (1942) assertion that the selection and adoption of the introductory psychology textbook is the “single most important Supplement” to the faculty member teaching the course. It is important to note that the selection of the appropriate textbook for undergraduate students is a challenge in other disciplines as well as being a general issue in college teaching (see Berry, Cook, Hill, & Stevens, 2010; Garinger, 2002). We recognize the fact that psychology and physical sciences can learn from each other in terms of teaching and learning the respective disciplines (see Hake, 2015). However, the issue of culture may be more central to psychology as a social science than other STEM disciplines.

Research on introductory psychology textbooks generically may also inform a culturally relevant textbook selection process. There is not an abundance of studies on introductory psychology textbook selection, so the development of an approach to the identification of an appropriate textbook for students attending an HBCU may evolve from examining research on the analysis of contents of such books, faculty rankings of the most important attributes relevant to selection criteria, and research on students’ perceptions and use of textbooks. The most common objective approach to research on introductory psychology textbooks is content analysis (e.g., Chatman & Goetz, 1985; Griggs, 2014; Griggs, Bujak-Johnson, & Proctor, 2004; Griggs & Jackson, 2013; Harari & Jacobson, 1984; Lonner, 2016; Weiten, 1988Weiten & Wight, 1992). As mentioned earlier, content analyses by Griggs (2014) and Lonner (2016) simply underscored the inadequate representation of cultural diversity in introductory psychology textbooks.

Another approach to content analysis is the utilization of selection criteria established a priori to identify which are most relevant from a list of attributes in the selection of introductory psychology textbooks (Altman, Erickson, & Pena-Sheff, 2006; Landrum & Hormel, 2002). Past research indicated that what participants rated highly as important in selecting and adopting an introductory psychology text included the following: accuracy; readability; organization; currency of research; diversity; level of rigor; support from ancillary package; critical thinking; cost; and quality (or inclusion) of a glossary (Altman et al., 2006; Landrum & Hormel, 2002). These particular attributes can contribute to the development of selection criteria in identifying an introductory psychology textbook, but they need to be supplemented with culturally relevant components.

Research on students’ perceptions and use of introductory psychology textbooks has yielded two instruments to be used to understand the features most relevant to student learning. The instruments are the Textbook Assessment and Usage Scale (TAUS; Gurung & Martin, 2011) and the Collegiate Student Assessment of Textbooks (CSAT; Landrum, Gurung, & Spann, 2012). The TAUS assesses students’ perceptions of the main characteristics of the textbook – “figures”, “tables”, “photographs”, “research examples”, “everyday examples”, “pedagogical aids”, “visual appeal”, and “writing quality”. The CSAT is an attitudinal and behavioral measure of students’ views of textbooks in terms of “intent to purchase”, “likeability”, and “use”. Factor analysis of the CSAT revealed that students intend to purchase the textbook for “learning and understanding”, “class requirement”, and “cost”; their liking of the textbook depends on “practical application to students’ lives and convenience”, “accessibility”, and “graphs and tables”; their use of the textbook is determined by “study aids”, “instructor’s use of the textbook”, and “ease of use” (Landrum et al., 2012). Moreover, Landrum et al. (2012) found moderate correlations between the TAUS and CSAT for subscale scores and overall scores. Indeed these dimensions, factors, and characteristics reflected in students’ perceptions of introductory psychology textbooks are relevant to selection criteria, but they too do not address cultural relevance of the content.

The most relevant findings from research with the TAUS or CSAT on students’ perception and use of introductory psychology textbooks are: (1) students do discriminate, albeit modestly, in their perceptions of textbooks; (2) students’ evaluations of textbooks are related to their amount of reading; and (3) there is little correlation between students’ views of textbooks and their course performance (Gurung & Landrum, 2012; Gurung, Daniel, & Landrum, 2012; Gurung, Landrum, & Daniel, 2012; Gurung & Martin, 2011; Landrum et al., 2012). Although, Landrum et al. (2012) suggested that the effect on course performance may be indirect due to their finding of a significant correlation between the percent of reading completed and selected academic outcomes. Taken together with research on cultural diversity, these different areas of research on introductory psychology textbooks support the current research effort and provide some guidance in terms of criteria selection to be discussed later.

The issue of cultural diversity not being adequately addressed is a major concern for many universities in general, and psychology departments in particular (Boysen, 2011). Clearly with Gurung et al.’s (2016) new proposed model of teaching introductory psychology, cultural diversity should be a major consideration in the textbook selection process. The purpose of the present study is to describe a culturally relevant approach to the introductory psychology textbook selection process for an HBCU student population. The current approach builds upon the previously mentioned research but adds a diversity component including a survey of HBCU psychology departments and the inclusion of a criterion addressing diversity in textbook selection. This is a preliminary investigation to examine the feasibility of using this systematic approach to select an introductory psychology textbook for HBCU students.

Method

Overview

During the fall 2011 semester, the department chair appointed three faculty members at an HBCU to a textbook selection committee to begin the process of selecting a new book for the general psychology course. The committee developed an empirical approach to textbook selection for the introductory psychology course. The empirical approach consisted of a multistage procedure that ultimately leads to the development of a rating system and assessment of its psychometric quality: (1) a survey of HBCU psychology departments was conducted to ascertain the strategies that they use to select textbooks for their introductory psychology courses; (2) a review of the research on introductory psychology textbooks to extrapolate the most relevant selection criteria; and (3) a rating instrument was developed and used in the actual selection of a textbook, the procedure was replicated during the fall 2014 semester, the ratings combined across semesters and subjected to psychometric analyses.

A Survey of HBCU Psychology Departments

Initially we contacted colleagues at other HBCUs to determine whether they have recommendations on textbooks relevant to our particular student population or have a systematic process of textbook selection that our department could use. From November 2011 to February 2012, committee members spoke with several of our colleagues. They initially compiled a list of the contact information for all of the four-year HBCUs and began calling each institution to speak with someone who was currently teaching the general psychology course. There were at least three attempts made to reach someone at each institution on the list. We asked each respondent the following questions: (1) What General Psychology textbook is your department currently using? (2) How long has your department used the aforementioned textbook? (3) What criteria were used in the selection process of the textbook you are currently utilizing? (4) What was the faculty feedback of the textbook? (5) What was the student feedback of the textbook? (6) What text would you not recommend and why? From the phone calls we were able to speak with 54 of 86 our colleagues, giving us a response rate of 63%. Survey respondents made several recommendations of introductory psychology textbooks, but none had a systematic process of selecting a book. The committee decided to create a systematic objective process with a set of criteria empirically derived from the relevant literature and professional experience.

Instrument Development

Criteria Selection

During the spring 2012 semester, we developed 21 criteria, derived mainly from the content analysis research studies, that we believed were important components for introductory psychology textbook selection (Altman et al., 2006; Landrum & Hormel, 2002). We also added criteria that we considered applicable to the HBCU context. The criteria on the list included: cost; research (past and present); illustration; overall relevance; blending of theory with real life applications/examples; readability; access to test banks; chapter summary or summarization; highlighting of key figures and research in the field; online interactive/student online learning accessibility; glossary; relevance to American Psychological Association (APA) learning goals; relevance to the minority student; multicultural relevance; explanation of concepts; critical thinking skill development; organization; supplementary materials; current or past HBCU usage; publishers’ representative presentation and/or accessibility to service; and compatibility with blackboard. Each of the textbooks were rated on the 21 items by selecting for each item whether the book “meets criteria” (1), “undecided” (2), or “doesn’t meet criteria” (3).

Revised Criteria and Scoring

In the summer of 2012, we consolidated the 21 item criteria list into 10 items. The decision to reduce the number of criteria was based on observations that some of the items on the original list appeared redundant or could be subsumed by criteria for other items. The elimination of criteria in the process of reduction and finalizing the list of criteria was done by committee consensus. The revised list consisted of cost, blending of theory with real life application/examples, readability, supplementary materials, highlighting of key figures and research in the field, glossary, relevant to the minority student, explanation of concepts, organization and critical thinking skill development. Each textbook to be rated was placed on an Excel spreadsheet in rows of the first column and the 10 criteria in columns 2–11 above the books. The committee also decided to change from a 3-point rating scale to a 5-point scale to increase the precision of the rating system. The ratings scale scores ranged from 1 to 5 with: 1 = “worst or does not meet criteria”; 2 = “moderately disagree that text does not meet criteria”; 3 = “neutral”; 4 = “moderately agree that text meets criteria”; and 5 = “best or meets criteria.”

Evaluation and Selection of a Textbook

Faculty Raters

Four faculty raters participated in the textbook selection process across two different semesters in groups of three. Each of these individuals taught the general psychology course, but they varied in terms of their personal and professional backgrounds with the exception of all being African American. There were two males and two females. Educational backgrounds included three doctoral level psychologists (2 clinical and 1 counseling) and one master’s degree in general psychology. Teaching experience ranged from two years to over 20 years.

Book Selection

Each committee member relied on their teaching experience to choose two textbooks that they thought would be good choices for our department and contacted book publishers to receive four copies of each text. The three committee members and the department chair each received copies of all six books during the fall 2012 selection. From that list the top three books that met most of the criteria among all three raters were selected for further consideration. In the early part of the fall 2012 semester, each committee member rated all three finalist textbooks and the points were calculated to derive an average score for each textbook. In the end, the committee chose the textbook with the highest score as the new textbook for the general psychology class. We replicated this procedure in the fall 2014, comparing the 2012 course book to four new introductory psychology textbooks, to identify a new book for the general psychology course. One rater was replaced by a new faculty member in the fall 2014. The process generated 90 ratings (10 criteria X 3 books X 3 faculty raters) in the fall 2012 and 150 ratings (10 criteria X 5 books X 3 faculty raters) in the fall 2014. A comparison of ratings across the two semesters yielded no statistically significant differences, so they were combined for a total of 240 ratings which were subjected to psychometric analyses.

Psychometric Evaluation of the Rating System

The psychometric evaluation consisted of two types of reliability. The first evaluation focused on the reliability of the criteria. The second evaluation examined the agreement among raters.

Reliability of Criteria

A two-step procedure is applied to 240 ratings derived from four raters by seven books by10 criteria or items of the instrument across two semesters taking into account rater replacement. Only two of the four raters evaluated all seven books: Each of the other two raters evaluated the number of books that were available during the semester when they served on the committee. The first step is an exploratory factor analysis which allows us to determine whether the construct, which we will label “textbook suitability for HBCU students,” is unidimensional or multidimensional. The next step is an estimation of internal consistency reliability using alpha coefficient (Cronbach, 1951). Peterson (1994) demonstrated empirically that most alpha values fall between 0.70 and 0.82, so we consider a coefficient alpha of 0.70 or higher as acceptable internal consistency reliability.

Interrater Reliability

Agreement among raters is estimated using the intraclass correlation coefficient (ICC; Shrout & Fleiss, 1979). Two-way analysis of variance (ANOVA) is first conducted to derive the mean square values associated with the sources of variance needed to compute the ICC. The 7 (Book) X 4 (Faculty Rater) ANOVA has both books and faculty raters as between-subjects variables in a main effects model. In other words, books are the target and faculty raters are the judges. Because two raters and one book were evaluated in two different semesters, there may be some shared variance in effects due to raters and to books. Because there was a two year interval between introductory textbook selections, it is also quite plausible that the ratings are independent of each with no shared variance. The ratings of highest rated book in fall 2012 and its rating in fall 2014 by the same two faculty raters were examined via correlational analyses. The version of ICC (1, k) is selected to assess interrater agreement because of the multiple raters (k = 4) and the underlying assumption that effects due to raters, to the interaction between raters and books, and to random error are inseparable (Shrout & Fleiss, 1979). Shrout and Fleiss (1979) demonstrated that, all things being equal, ICC (1, k) tends to yield a more conservative estimate of interrater reliability than the versions of ICC (2, k) and ICC (3, k). The formula is adapted for the current analysis as follows:

ICC (1, k) = BMS - WMS / BMS, where BMS = {MS}_{(Book)} and WMS = {MS}_{(WithinBookError)}

Results

Descriptive Analyses

The 10 criteria that constituted the final list for the rating instrument were the following: cost; blending of theory with real life applications; readability; supplementary materials; highlighting of key figures; glossary; relevant to the minority student; explanation of concepts; organization; and critical thinking skill development. Because “cost” was assigned a rating of “3” in all instances, it showed zero variance in faculty ratings. The criterion “cost” was not included in subsequent reliability analyses. The number of faculty ratings was reduced to 216 ratings with the removal of the criterion of “cost.” Table 1 presents the descriptive statistics and intercorrelations for the criteria ratings. An absolute value greater than 2.00 for skewness or kurtosis is an indication of violation of assumptions of normality. As can be seen in Table 1, all criteria or rating scores did meet assumptions of normality. Scores on the item “blending of theory and real-life applications” were significantly correlated with those on the criteria of “explanation of concepts” and “critical thinking skills development.” The criterion “readability” scores were also significantly associated with those of “supplemental materials,” “glossary,” “relevant to the minority student,” “explanation of concepts,” and “organization.” The item “supplemental materials” yielded scores also significantly associated with scores on the items “glossary” and “explanation of concepts”. Criterion scores for “relevant to the minority student” and for “explanation of concepts” were significantly associated. Finally, it is also noteworthy that the scores for the item “critical thinking skills development” have no significant correlations with 88% of the remaining criteria scores.

Table 1.

Descriptive Statistics and Intercorrelations for Criteria Scores

Criteria	Mean	Standard Deviation	Skewness	Kurtosis	1	2	3	4	5	6	7	8	9
1. Blending of Theory and Real-Life Applications	3.79	0.79	–0.21	–0.12	—
2. Readability	4.04	0.62	–0.02	–0.11	0.29	—
3. Supplemental Materials	3.79	0.79	–0.81	0.95	0.14	0.47*	—
4. Highlighting of Key Figures	3.63	0.65	–0.51	0.45	0.44*	0.15	0.01	—
5. Glossary	3.54	0.78	–0.15	–0.12	0.12	0.49*	0.77‡	-0.01	—
6. Relevant to the Minority Student	3.54	0.72	0.22	–0.08	0.13	0.43*	0.29	0.27	.23	—
7. Explanation of Concepts	3.79	0.78	0.40	–1.20	0.50*	0.56†	0.43*	0.10	.34	.44*	—
8. Organization	3.83	0.76	–0.35	0.20	0.38	0.47*	0.23	0.31	.09	.09	.38	—
9. Critical Thinking Skills Development	3.79	0.78	0.41	–0.54	0.64‡	0.37	0.12	0.11	.12	.13	.38	.33	—

*p < .05; † p < .01; ‡ p < .001.

Reliability of Criteria

The 216 criteria rating scores were subjected to a principal components analysis with varimax rotation. The coefficients were sorted by size and any criterion with a value below 0.50 was eliminated for a given factor. Three factors were retained, because the eigenvalue dropped below 1.0 beyond that point. The three-factor solution accounted for 69% of the variance in the criteria rating scores of introductory psychology textbooks. The first factor in the original (unrotated) solution included all criteria except “highlighting of key figures.” The rotated three-factor solution is presented in Table 2. Factor 1 in the rotated solution included: “supplemental materials”; “glossary”; “readability”; and “explanation of concepts.” This factor is labeled Quality of Learning Resources. Factor 2 consisted of: “blending of theory and real-life applications”; “critical thinking skills development”; “organization”; and “explanation of concepts.” The second factor is labeled Learning and Comprehension. It is evident that “explanation of concepts” cross-loaded on both factor 1 and factor 2, but it is retained on factor 1 because of its slightly larger coefficient. Factor 3 was the most stable consisting of “highlighting of key figures” and “relevant to the minority student” in both the original and rotated solutions. The last factor is labeled Cultural Relevance. Finally, the Cronbach’s alpha for the scales derived from Quality of Learning Resources, Learning and Comprehension, and Cultural Relevance were 0.81, 0.75 and 0.42, respectively, indicating adequate internal consistency reliability for the first two factors but not the third factor. The total scale also yielded adequate internal consistency reliability, α = 79 with item-total correlations ranging from r = 0.38 for “relevant to the minority student” to r = 0.68 for “readability.”

Table 2.

Three-Factor Rotated Solution from Principal Component Analysis

Criteria	Factor
Criteria	1	2	3
Supplemental Materials	0.877
Glossary	0.873
Readability	0.653
Explanation of Concepts	0.537	0.524
Blending of Theory and Real-Life Applications		0.849
Critical Thinking Skills Development		0.848
Organization		0.613
Highlighting of Key Figures			0.803
Relevant to the Minority Student			0.708

Interrater Reliability

The 7(Book) X 4 (Faculty Rater) ANOVA yielded a significant model effect, R²= 0.27, F (9, 215) = 8.41, p < 0.001. To obtain the total within-book error variance, a one-way ANOVA was conducted with book as the only between-subjects variable, R²= 0.15, F (6, 209) = 6.29, p < 0.001, Table 3 presents a summary of the ANOVA models combined. Following Shrout and Fleiss’s (1979) approach, Table 3 provides the ANOVA results to compute ICC (1, k). It was hypothesized that this ICC model is most appropriate because of shared variance due to books and raters. Correlational analyses supported this hypothesis with a moderate correlation for Book 1 across semesters, r = 0.32, and large coefficient for Rater 1, r = 0.58 but not Rater 2, r = 0.29. Using the data from Table 3, we find that there is very good agreement among faculty raters, ICC (1, 4) = 0.841. For the sake of comparison, ICC (2, 4) = 0.854 and ICC (3, 4) = 0.859 were computed and also shown to be very good. A 7 (Book) X 4 (Faculty Raters) X 9 (Criteria) model, R²= 0.31, F (17, 198) = 5.28, p < 0.001, revealed a nonsignificant main effect for criteria, F (8, 198) = 1.56, p = ns, As a consequence, accounting for variance due to selection criteria did not change the level of agreement among raters, ICC (1, 4) = 0.841, We also conducted a one-way multivariate analysis of variance with factor scores for Quality of Learning Resources, Learning and Comprehension, and Cultural Relevance as the dependent variables and books and raters as between-subjects, independent variables. Significant multivariate effects were evident for Quality of Learning Resources, Learning and Comprehension, and Cultural Relevance. Table 4 presents the results of univariate tests for each dependent variable. Interrater agreement was very good for Quality of Learning Resources, ICC (1, 4) = 0.827 and Cultural Relevance, ICC (1, 4) = 0.895; and it was excellent for Learning and Comprehension, ICC (1, 4) = 0.984.

Table 3.

Analysis of Variance Results for Faculty Ratings of Selection Criteria

Source^a	Type III SS	df	MS	F	p
Intercept	2224.588	1	2224.588	5498.608	0.001
Books	17.440	6	2.91	6.29	0.001
Within Book Error	96.519	209	0.46
Raters	13.18	3	4.39	10.86	0.001
Error	83.34	206	0.41

The main effects for books was obtained from a single variable model with a lower

MS_(Book) value resulting in a more conservative intraclass correlation coefficient.

Table 4.

Multivariate Analysis of Variance Results for Scores from Three Factors Underlying Selection Criteria

Source^a	Type III SS	df	MS	F	p
Quality of Learning Resources
Intercept	2941.55	1	2941.55	9018.84	0.001
Books	11.46	6	1.91	5.86	0.001
Within Book Error	68.17	209	0.33
Raters	45.21	3	15.07	135.19	0.001
Error	22.96	206	0.11
Learning and Comprehension
Intercept	2844.12	1	2844.12	25985.58	0.001
Books	42.28	6	7.05	64.385	0.001
Within Book Error	22.88	209	0.11
Raters	4.32	3	1.44	16.00	0.001
Error	18.55	206	0.09
Cultural Relevance
Intercept	2578.88	1	2578.88	11141.81	0.000
Books	13.13	6	2.19	9.45	0.000
Within Book Error	48.38	209	0.23
Raters	24.77	3	8.26	72.068	0.000
Error	23.60	206	0.12

The main effects for books was obtained from a single variable model with a lower

MS_(Book) value resulting in a more conservative intraclass correlation coefficient.

Discussion

Recent empirical research has underscored the need for cross-cultural research and documented the importance of cultural diversity in the teaching and learning of psychology (Boysen, 2011; Case, 2007; Fuentes & Shannon, 2016; Kernahan & Davis, 2010; Kite & Whitely, 2012; Lonner, 2016; Prieto et al., 2009; Whaley, 2012; Warren, 2006). We supplemented our textbook selection procedure, informed by several areas of research on introductory psychology textbooks, with a cultural diversity component including a survey of HBCU psychology departments and one item among the criteria. In our survey of other HBCUs, we found that none had a systematic approach to the textbook selection process. This approach is akin to Lonner’s (2016) study of cross-cultural experts to identify cultural content to be included in introductory psychology textbooks. We believed that we could benefit from our HBCU colleagues’ knowledge and experience in selecting an introductory text book with which their students identified.

Unfortunately, there was no consensus among respondents from other HBCUs to questions about the selection of introductory psychology textbooks. Thus this first stage of the process can be eliminated, because it did not yield any information to facilitate the selection of an introductory psychology textbook for HBCU students. In fact, this first stage was not repeated for the introductory psychology textbook selection process in the fall 2014 semester. The findings suggest that even HBCU faculty do not adequately address issues of cultural diversity in the textbook selection process. One implication of this finding is that HBCU faculty should become more involved in research to address cultural diversity issues for their general psychology students.

The selection of criteria for the development of the rating instrument relied heavily on the broad literature on introductory psychology textbooks. The influence is evident in the factor structure underlying the rating instrument. Exploratory factor analysis of the textbook ratings with the selection criteria produced a three-factor solution. Varimax rotation of the original factors yielded the following factors: Quality of Learning Resources; Learning and Comprehension; and Cultural Relevance. Our exploratory factor analysis was more focused in that criteria were preselected. Criteria selection was informed by the highest-ranking attributes by teachers of introductory psychology reported by Landrum and Hormel (2002) and Altman et al. (2006). Thus similarities between the pattern of highest loadings for coefficients from the current exploratory factor analysis and top rankings of the selection criteria by a national sample of introductory psychology instructors provide some external validity for our findings.

The fact that HBCU psychology faculty may value the same attributes as essential to the teaching of introductory psychology as their counterparts at PWIs is not surprising. Most HBCU faculty were trained at PWIs often times in APA-approved psychology programs. Only a handful of HBCUs have doctoral programs in psychology which also follow APA-guidelines in the teaching of psychology. In fact, there is only one psychology department at a HBCU in the entire country at Florida A & M University with bachelor’s and master’s degree programs with curricula emphasizing Black psychology. Individual differences in a HBCU faculty’s level of cultural identification and cultural competence should be taken into consideration in future research on the teaching and learning of introductory psychology, particularly the textbook selection process.

It is interesting to note that the criterion “relevant to the minority student” tied with “glossary” for the lowest rating of all the criteria (see Table 1). Zero-order correlations revealed that this criterion “relevant to the minority student” was only significantly related to “readability”. This correlation suggests that our faculty raters consider cultural identification an important factor in HBCU student performance. This assumption is supported by previous research on the teaching and learning of psychology in the HBCU context (Whaley, 2012). Table 1 also revealed that the criterion of “readability” was scored the highest by faculty raters. It was the criterion that was correlated with more of the remaining criteria than any other, suggesting that it is considered a central feature of introductory psychology textbooks. Intercorrelations of “readability,” “supplemental materials,” “glossary,” and “explanation of concepts” comprised the factor Quality of Learning Resources from the principal component analysis. The importance placed on the criterion of readability by faculty raters is supported by other research. Quality of learning resources in general and readability in particular could enhance student learning. Consistent with this assertion, Landrum et al. (2012) found “percentage of reading completed” was positively correlated with students’ quiz scores, total course points, and final course grades: They also reported that percentage of textbook read was significantly correlated with subscales of the CSAT and TAUS tapping such dimensions as “ease of use” which also corresponds to our Quality of Learning Resources factor.

Another noteworthy finding was zero variability for the criterion of cost. Faculty raters were 100% neutral on this criterion with a score of “3” across all books and semesters. Students often complain about the cost of introductory psychology textbooks and are not even willing to pay more for a book that they like (Gurung & Landrum, 2012). Moreover, there is evidence that the cost of introductory psychology textbooks is not associated with students’ perceptions of the quality of the book and is negatively correlated with their reading habits and learning orientation (Landrum et al., 2012). It appears that faculty may be ambivalent about the issue of textbook costs, because they may sympathize with students regarding the expense but appreciate the importance of reading for student academic success. This is one possible interpretation of faculty remaining 100% neutral on the issue of textbook costs. The national survey of teachers of introductory psychology ranked textbook cost in the top 25% of important attributes related to textbook selection (Landrum & Hormel, 2002). Future research could clarify this issue by using a faculty equivalent of the CSAT to identify their specific views on textbook costs among HBCU faculty.

The objective selection of criteria from the literature yielded another factor from principal components analysis consistent with research on textbooks for the teaching and learning of introductory psychology. These findings reflect what Campbell (1996) referred to as the overlap between measurement validity and experimental design validity. In other words, our results demonstrate both construct validity and external validity. Learning and Comprehension was another factor generated from faculty ratings of the criteria in the introductory psychology textbook selection process. This factor is similar to the Learning and Understanding subscale of the CSAT. It is important to note that the CSAT is a student-focused instrument, while our findings reflect faculty perspectives. One implication of the similarity between factors underlying faculty ratings and the CSAT is that HBCU faculty are attuned to students’ needs, To test this hypothesis, future researchers should administer this instrument to HBCU students to determine whether it would replicate these factors. It would also be important to correlate the factor scores with the relevant CSAT or TAUS subscales to establish unequivocal construct validity.

The exploratory factor analysis yielded a final factor which we labeled Cultural Relevance with adequate interrater agreement but low internal consistency reliability. These mixed reliability findings are consistent with literature on the teaching and learning of introductory psychology. Cultural diversity was rated moderately low in importance among the rankings of the national sample of teachers of introductory psychology reported by Landrum and Hormel (2002). We noted earlier that that there is a gap between theory and practice in the value placed on cultural diversity in the literature on the teaching and learning of introductory psychology (see Fuentes & Shannon, 2016; Griggs, 2014; Gurung et al., 2016; Landrum et al., 2010; Landrum & Hormel, 2002; Lonner, 2016; Ocampo et al., 2003; Prieto et al., 2009). Our findings also yielded limited information on cultural diversity.

Internal consistency reliability was uneven for the factor scores with Quality of Learning Resources and Learning and Comprehension being adequate and Cultural Relevance showing a low alpha coefficient. Estimates of interrater agreement for factor scores also produced uneven findings. Intraclass correlation coefficients yielded the highest correlation for the Learning and Comprehension, followed by Cultural Relevance, and Quality of Learning Resources. Unlike internal consistency reliability, interrater agreement was acceptable across all of the factors despite the differences. The two types of reliability showed divergent patterns across the underlying factors. Thus we recommend that the entire scale be used in the selection of a textbook for introductory psychology courses.

One possible reason for the low to moderate effects in the current study is that our operationalization of cultural diversity was not adequate. Perhaps more items addressing cultural issues among selection criteria or the assessment of other mechanisms of diffusion at HBCUs may yield different outcomes than similar studies at PWIs. Nevertheless, diffusion of cultural diversity in introductory psychology continues to an important goal. Fuentes and Shannon (2016) argued that including cultural diversity in introductory psychology courses “may initiate student awareness of differences and possibly serve as a catalyst for further interest or study” (p. 200). Future research should devote more attention to cultural diversity in introductory psychology courses in general, and textbooks for those courses in particular.

Some additional considerations for future study include a larger pool of raters and the need for student input in the evaluation of introductory psychology textbooks. The methodology should also be tested with other subject domains and different undergraduate settings. For example, the multisite design used by Gurung et al. (2012) can be adapted to have several HBCUs rate a common set of introductory psychology textbooks which would address the need for more raters, expand to different settings, and increase the involvement of other HBCU faculty in culturally relevant research on introductory psychology textbook selection. Student input is recognized as a valuable resource in the evaluation process for introductory psychology courses. This is true of college textbooks in general. Toward this end, Berry et al. (2010) identified factors that promote and prevent textbook use among undergraduate students. Several studies gather student input via surveys on various aspects of “use” which have implications for faculty in the textbook selection process (Clump, Bauer, & Bradley, 2004; Gurung & Martin, 2011; Sheppard, Grace, & Kock, 2008; Sikorski et al., 2002; Taylor, 2011). One of the principal findings from such research efforts is that students often do not read, or even purchase the text, which limits the utility of their input in the textbook selection process.

On the other hand, innovative methods can be used to get students’ perspectives such as having them report the percentage of the textbook that they read (Landrum et al., 2012); or focusing on select chapters (Gurung & Landrum, 2012, Study 2). A popular approach is students’ evaluation of specific features like pedagogical aids or topics covered (Gurung, 2004; McCann, Immel, Kadah-Ammeter, & Adelson, 2016; Weiten, Deguara, Rehmke, & Sewell, 1999). These alternative strategies also have limitations, but such issues are beyond the scope of the present discussion. Future research should also include a survey of HBCU students regarding the selected textbook, because the ultimate goal is to serve the best interest of students. We also encourage researchers to employ the aforementioned innovative methods to study African American students’ perceptions of introductory psychology textbooks in both HBCU and PWI environments.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Arthur L. Whaley is an independent scholar who has recently served as a research/evaluation consultant for community-based programs serving Black youth in New York City. He has held several administrative, faculty, and staff positions at various universities over the past 30 years. He received his PhD in clinical psychology from Rutgers University and completed a DrPH in epidemiology from the Mailman School of Public Health, Columbia University. His teaching interests include abnormal psychology, biopsychology cross-cultural psychology, statistics, and epidemiology.

William A. L. Clay, III is recently retired as the interim chair and associate professor in the Department of Psychology and Philosophy, Texas Southern University. His area of specialization is forensic psychology. Dr. Clay is deeply devoted to the development of a psychological model to aid and assist with the selection and adoption of an introductory psychology textbook for students attending historically Black colleges/universities. His teaching areas were introductory psychology, child development, and personality psychology.

Dominique Broussard is an assistant professor in the Department of Psychology and Philosophy at Texas Southern University (TSU) where she teaches both undergraduate and graduate students. She earned her master’s of education in community counseling and her PhD in counseling psychology both from the University of Georgia. Dr. Broussard is currently working with other TSU colleagues on a Substance Abuse and Mental Health Services Administration suicide prevention grant. Her current research includes the diversity and disparities associated with sexual health, mental health and the teaching of psychology.

References

Altman

Erickson

Pena-Shaff

J. B.

(2006) An inclusive process for departmental textbook selection. Teaching of Psychology 33: 228–231.

Berry

Cook

Hill

Stevens

(2010) An exploratory analysis of textbook usage and study habits: Misperceptions and barriers to success. College Teaching 59(1): 31–39.

Boysen

(2011) Diversity topics covered in teaching of psychology courses. Teaching of Psychology 38(2): 89–93.

Campbell

D. T.

(1996) Unresolved issues in measurement validity: An autobiographic overview. Psychological Assessment 8: 363–368.

Case

(2007) Raising white privilege awareness and reducing racial prejudice: Assessing diversity course effectiveness. Teaching of Psychology 34: 231–235.

Chatman

Goetz

(1985) Improving textbook selection. Teaching of Psychology 12: 150–152.

Chavous

T. M.

Harris

Rivas

Helaire

Green

(2004) Racial stereotypes and gender in context: African Americans at predominantly Black and predominantly White colleges. Sex Roles 51(1–2): 1–16.

Clump

Bauer

Bradley

(2004) The extent that psychology students read textbooks: A multiple class analysis of reading across the psychology curriculum. Journal of Instructional Psychology 31: 227–232.

Cronbach

L. J.

(1951) Coefficient alpha and the internal structure of tests. Psychometrika 16: 297–334.

10.

Fuentes

M. A.

Shannon

C. R.

(2016) The state of multiculturalism and diversity in undergraduate psychology training. Teaching of Psychology 43: 197–203. doi: 10.1177/0098628316649315.

11.

Garinger, D. (2002). Textbook selection for the ESL classroom. Center for Applied Linguistics Digest. Retrieved from http://mcael.org/uploads/File/provider_library/Textbook_Eval_CAL.pdf.

12.

Gasman, M. (2013). The changing face of historically black colleges and universities. Retrieved from http://www.gse.upenn.edu/pdf/cmsi/Changing_Face_HBCUs.pdf.

13.

Griggs

R. A.

(2014) Topical coverage in introductory textbooks from the 1980s through the 2000s. Teaching of Psychology 41(1): 5–10. doi: 10.1177/0098628313514171.

14.

Griggs

R. A.

Bujak- Johnson

Proctor

D. L.

(2004) Using common vocabulary in text selection and teaching the introductory course. Teaching of Psychology 31: 265–269.

15.

Griggs

R. A.

Jackson

S. L.

(2013) Introductory psychology textbooks: An objective analysis update. Teaching of Psychology 40: 163–168.

16.

Gurung

R. A. R.

(2004) Pedagogical aids: Learning enhancers or dangerous detours? Teaching of Psychology 31: 164–166.

17.

Gurung

R. A. R.

Daniel

D. B.

Landrum

R. E.

(2012) A multisite study of learning in introductory psychology courses. Teaching of Psychology 39: 170–175.

18.

Gurung

R. A.

Hackathorn

Enns

Frantz

Cacioppo

J. T.

Loop

Freeman

J. E.

(2016) Strengthening introductory psychology: A new model for teaching the introductory course. American Psychologist 71: 112–124.

19.

Gurung

Landrum

R. E.

(2012) Comparing student perceptions of textbooks: Does liking influence learning? International Journal of Teaching and Learning in Higher Education 24: 144–150.

20.

Gurung

R. A.

Landrum

R. E.

Daniel

D. B.

(2012) Textbook use and learning: A North American perspective. Psychology Learning & Teaching 11(1): 87–98.

21.

Gurung

R. A. R.

Martin

R. C.

(2011) Predicting textbook reading: The Textbook Assessment and Usage Scale. Teaching of Psychology 38(1): 22–28.

22.

Habarth

Hansell

Grove

(2011) How accurately do introductory psychology textbooks present psychoanalytic theory? Teaching of Psychology 38(1): 16–21.

23.

Hake

R. R.

(2015) What might psychologists learn from scholarship of teaching and learning in physics? Scholarship of Teaching and Learning in Psychology 1(1): 100–106.

24.

Harari

Jacobson

(1984) Teaching of psychology in the 1980s: A content analysis of leading psychology textbooks. Teaching of Psychology 11: 236–237.

25.

Kernahan

Davis

(2010) What are the long term effects of learning about racism? Teaching of Psychology 37(1): 41–45.

26.

Kite

M. E.

Whitely

B. E.

(2012) Ethnic and nationality stereotypes in everyday language. Teaching of Psychology 39(1): 54–56.

27.

Landrum

R. E.

Beins

B. C.

Bhalla

Brakke

Briihl

D. S.

Curl-Langager

R. M.

Van Kirk

J. J.

(2010) Desired outcomes of an undergraduate education in psychology from departmental, student, and societal perspectives. In: Halpern

D. F.

(ed.) Undergraduate education in psychology: A blueprint for the future of the discipline, Washington, DC: American Psychological Association, pp. 145–160.

28.

Landrum

R. E.

Hormel

(2002) Textbook selection: Balance between the pedagogy, the publisher, and the student. Teaching of Psychology 29: 245–250.

29.

Landrum

R. E.

Gurung

R. A.

Spann

(2012) Assessments of textbook usage and the relationship to student course performance. College Teaching 60(1): 17–24.

30.

Lonner

(2016) The Introductory psychology text and cross-cultural psychology: Beyond Ekman, Whorf, and Biased I.Q. Tests. Online Readings in Psychology and Culture 11(1): doi: /10.9707/2307-0919.1145.

31.

McCann

L. I.

Immel

K. R.

Kadah-Ammeter

T. L.

Adelson

S. K.

(2016) The importance and interest of introductory psychology textbook topics: Student opinions at technical college, 2-, and 4-year institutions. Teaching of Psychology 43: 215–220. doi: 1177/ 0098628316649477.

32.

Ocampo

Prieto

L. R.

Whittlesey

Connor

Janco-Gidley

Mannix

Sare

(2003) Diversity research in Teaching of Psychology: Summary and recommendations. Teaching of Psychology 30(1): 5–18.

33.

Prieto

L. R.

Whittlesey

Herbert

Ocampo

Schomburg

(2009) Dealing with diversity issues in the classroom: A survey of the STP membership. Teaching of Psychology 36(2): 77–83.

34.

Peterson

R. A.

(1994) A meta-analysis of Cronbach’s coefficient alpha. Journal of Consumer Research 21: 381–391.

35.

Shepperd

J. A.

Grace

J. L.

Kock

E. J.

(2008) Evaluating the electronic textbook: Is it time to dispense with the paper text. Teaching of Psychology 35(1): 2–5.

36.

Shrout

P. E.

Fleiss

J. L.

(1979) Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin 86: 420–428.

37.

Sikorski

J. F.

Rich

Saville

B. K.

Bukist

Drogan

Davis

S. F.

(2002) Student use of introductory texts: Comparative survey findings from two universities. Teaching of Psychology 29: 312–313.

38.

Smith

C. E.

Hopkins

(2004) Mitigating the impact of stereotypes on academic performance: The effects of cultural identity and attributions for success among African American college students. Western Journal of Black Studies 28(1): 312–321.

39.

Taylor

A. K.

(2011) Students learn equally well from digital as from paperbound texts. Teaching of Psychology 38: 278–281.

40.

Warren

C. S.

(2006) Incorporating multiculturalism into undergraduate psychology courses: Three simple active learning activities. Teaching of Psychology 33: 105–109.

41.

Whaley

A. L.

(2012) Effects of ethnic/racial identification on case-based learning outcomes in a psychology course at a historically Black college/university (HBCU). Journal of Cognitive Education and Psychology 11(1): 79–93.

42.

Weiten

(1988) Objective features of the introductory psychology as related to professor’s impressions. Teaching of Psychology 15(1): 10–16.

43.

Weiten

Deguara

Rehmke

Sewell

(1999) University, community college, and high students’ evaluations of textbook pedagogical aids. Teaching of Psychology 26(1): 19–21.

44.

Weiten

Wight

R. D.

(1992) Portraits of a discipline: An examination of introductory psychology textbooks in America. In: Puente

A. E.

Matthews

J. R.

Brewer

C. L.

(eds) Teaching psychology in America: A history, Washington, DC: American Psychological Association, pp. 453–504.

45.

Wolfle

(1942) The first course in psychology. Psychological Bulletin 39: 685–712.

46.

Woodcock

Hernandez

P. R.

Estrada

Schultz

(2012) The consequences of chronic stereotype threat: Domain disidentification and abandonment. Journal of Personality and Social Psychology 103: 635–646.

Cultural Diversity in Introductory Psychology Textbook Selection: The Case for Historically Black Colleges/Universities (HBCUs)

Abstract

Keywords

Method

Overview

A Survey of HBCU Psychology Departments

Instrument Development

Criteria Selection

Revised Criteria and Scoring

Evaluation and Selection of a Textbook

Faculty Raters

Book Selection

Psychometric Evaluation of the Rating System

Reliability of Criteria

Interrater Reliability

Results

Descriptive Analyses

Reliability of Criteria

Interrater Reliability

Discussion

Footnotes

Declaration of Conflicting Interests

Funding

Author biographies

References