Sage Journals: Discover world-class research

Abstract

This study reports on the development and validation of the Civic Engagement Scale (CES). This scale is developed to be easily administered and useful to educators who are seeking to measure the attitudes and behaviors that have been affected by a service-learning experience. This instrument was administered as a validation study in a purposive sample of social work and education majors at three universities (N = 513) with a return of 354 (69%). After the reliability and validity analysis was completed, the Attitude subscale was left with eight items and a Cronbach’s alpha level of .91. The Behavior subscale was left with six items and a Cronbach’s alpha level of .85. Principal component analysis indicated a two-dimensional scale with high loadings on both factors (mean factor loading for the attitude factor = .79, and mean factor loading for the behavior factor = .77). These results indicate that the CES is strong enough to recommend its use in educational settings. Preliminary use has demonstrated that this scale will be useful to researchers seeking to better understand the relationship of attitudes and behaviors with civic engagement in the service-learning setting. The primary limitations of this research are that the sample was limited to social work and education majors who were primarily White (n = 312, 88.1%) and female (n = 294, 83.1%). Therefore, further research would be needed to generalize this research to other populations.

Keywords

civic engagement scales validity reliability service-learning

Introduction

During the 1920s and 1930s, our country witnessed changes that led to individuals moving from a generation of civic responsibility to one of individuals who have been less committed to their communities (Putnam, 1995, 1996, 2000). This generation has remained less active in their communities with no membership and participation in civic activities (Putnam, 1995, 2000). The Great Depression had a traumatic impact on civic involvement in the 1930s. Many groups experienced a drastic drop in membership and volunteers during this era underlying “the effects of acute economic distress on civic engagement” (Putnam, 2000, p. 54). World War II created a burst of patriotism that resulted in vital growth in community involvement for two decades. However, that growth was followed by a slump in community involvement in the 1960s that has never fully recovered. The 1950s, 1960s, and 1970s witnessed turbulence and change in communities and on college campuses. During this era, the civil rights became a political issue, as there was an increasing focus on poverty and “the nation’s social problems” (Stanton, Giles, & Cruz, 1999). These years were difficult for those who were on the margins of society due to their race, gender, social class, or social orientation (Putnam, 2000).

Through the years, we have witnessed an evolution of our society into one that views problems as private. In this view, we assume that people have the resources they need to solve their own problems. The push to view problems as private encourages individuals to consider charity rather than commitment to deal with social problems (Lisman, 1998). Boyte (1991) contends that since World War II, youth have become less and less connected to public affairs. Some research has indicated that many college students are choosing paths that are disconnected from “civic life, voting, politics, governments, and social problems” (Hollander, 1999, p. v.; Putnam, 1995). Waldstein and Reiher (2001) contend that changes within our society have limited the way individuals form attachments to the community at large.

Some authors have suggested that a decrease in civic engagement leads to a decrease in social capital (Coleman, 1988; Hyman, 2002; Lin, 2001; Portes, 1998; Putnam, 1995). Coleman (1988) defined social capital as being embedded within relationships for the purpose of facilitating social action, and Hyman (2002) contended that social capital exists within social relationships and is purposeful. Lin (2001) defined social capital as an initial investment in relationships, with an intended outcome, that is, obtaining education for the intended outcome of increased income. Putnam (1995) held that social capital reinforces the networks and norms of relationships for the purpose of building social trust. Therefore, a positive consequence of civic participation and social capital may be seen as an increased level of trust within the community.

All of the authors included in this discussion have agreed that social capital exists and is embedded within relationships. Therefore, if individuals are engaging less in their communities, we could infer that there is a loss of relationships.

Concerns that the younger generations may be less inclined to participate and engage in their communities have created an interest in the way young people are socialized and learn to engage in their communities (Boyte, 1991; Campbell, 2000; Flanagan, 2003). Some scholars have noted that the overall decline in civic life and an overall lack of participation in community service may contribute to the declining interest in community engagement (Barber, 1992; Putnam, 1995). These concerns have served as an impetus for conversations regarding the manner in which students are taught to engage in the communities in which they live (Battistoni, 1997; Lisman, 1998).

Dewey (1899/1959), an early proponent of connecting academics and the community, believed that students should be taught about the community by saturating them with “the spirit of service” and providing them with the “instruments of . . . self-direction” (p. 49). Dewey, in Democracy and Education (1916/1966), wrote that classroom learning should be applied to the context of the community. He believed that education should consist of a combination and balance of formal and informal learning that occurs in school learning and through concrete experiences in the community (Hepburn, 1997).

To improve and enhance an individual’s civic responsibility, one must experience some level of civic learning. Ehrlich (1997) contended that “Civic learning—in the sense of how a community works and how to help it work better and academic learning are mutually reinforcing” (p. 61). He further suggested that that this form of learning is based upon moral learning, which he has defined as “reinforcing the elements of character that leads to ethical actions. These elements include: respect for the autonomy and dignity of others, compassion and kindness, honesty and integrity, and a commitment to equity and fairness” (p. 61). Similarly, some authors have found that such learning through service learning or community service activities serves to enable students as they embrace active citizenship within their own communities (Coles, 1993; Williams & Gilchrist, 2004). Purposeful community service exercises and assignments are an intervention that can be used to teach and train young adults the concepts of building relationships within their respective communities. Pateman (1970) contended that individuals learn to participate through participation. Purposeful involvement in community service activities provides students with the opportunity to participate with support and supervision through which they can reflect and respond to their community experiences.

Such practice can create what Battistoni (1997) identified as a civic outcome, in which individuals learn how to become partners in their communities, working with others to solve community problems. Such an outcome would be related to most universities’ goal to instill in students the desire to become purposefully engaged in their communities—not only to provide relief work but also to become active participants in problem-solving processes.

Assessment is one method that can be used to measure or evaluate a student’s potential to become civically engaged as a result of the educational process. This article introduces a multidimensional assessment tool called the Civic Engagement Scale (CES) that will enable educators to assess the attitudes and behaviors of students who have had a community service experience. The major focus of this article is to present the initial findings related to the psychometric properties of the scale.

The CES Method

Initial Development

For purposes of this study, the definition of civic engagement created by Thomas Ehrlich (1997) was utilized and expanded upon. Civic engagement has been defined as the process of believing that one can and should make a difference in enhancing his or her community. To enhance the community, one requires possessing the knowledge, skills, and values necessary to make a difference. The possession and demonstration of that knowledge, skills, and values are expressed through attitudes and/or behaviors. The CES has been created to measure two specific aspects of engagement: attitudes and behaviors.

For the scale, civic attitudes have been defined as the personal beliefs and feelings that individuals have about their own involvement in their community and their perceived ability to make a difference in that community. Civic behaviors have been defined as the actions that people take to actively attempt to engage and make a difference in their community.

This study began with a pilot version of the scale consisting of 11 attributes each on attitude and behavior. Attributes were chosen based on a literature review. The attributes were then rewritten as statements to form a scale. The scale items were chosen for their perceived ability to measure an individual’s changes in attitudes and behaviors. Next, the scale was given to a panel of college students of bachelor’s level for a review. Minor revisions were made to the scale following this review but no items were deleted. The scale was then given to seven faculty members for a review. These two groups of reviewers provided information regarding the content, comprehensiveness, and language, and provided suggestions to the developers (DeVellis, 1991; Springer, Abell, & Nugent, 2002).

The initial scale consisted of two constructs: attitudes and behaviors with 11 items each. The goal of the initial study was to test the psychometric characteristics of the scale, Civic Engagement. The results were used for an item analysis that provided information on the reliability and validity of the scale (Faul & van Zyl, 2004; Spector, 1992).

The respondents were asked to indicate their level of agreement or disagreement with the 11 items of the attitude dimension and to rate their level of participation as never to always on the behavior dimension of the scale. Their level of agreement was measurement on a 7-point Likert (1932) scale (1 = disagree, and 7 = agree) and their level of participation was also measured on a 7-point Likert-type scale (1 = never, and 7 = always).

Pilot Study

The scale was piloted during the spring semester at three institutions of higher education in the United States: one large public state urban university in the midwest and two private faith-based liberal arts universities, one urban located in the midwest and one in a suburban area in the southeast. In total, 513 surveys were administered as a purposive sample with a return of 354 (69%). A purposive sample is “a nonprobability sampling procedure in which researcher participants with particular characteristics are purposely selected for inclusion in a research sample” (Unrau, Krysik, & Grinnell, 1997, p. 254). The students were chosen according to the criterion that they were all college students of bachelor’s or master’s level. The purpose of the pilot study was to test the psychometric characteristics of the scale for validation and to determine the reliability and validity of the scale.

A preamble consent form was utilized to provide all participants with a full explanation of the study and the potential risks and benefits of their participation. Respondents were notified that their confidentiality would be protected to the extent that is provided by the law and that their responses would remain anonymous and any data reported would be aggregated and not reported individually. Those who chose to participate were asked to complete a research package that included basic demographic information, the CES, and the Community Service Attitudes Scale (CSAS; Shiarella, McCarthy, & Tucker, 2000). The CSAS was chosen for this study due to its established reliability and validity and the level on which it related to the constructs being measured in this study. The CSAS was used primarily for the purpose of construct validation.

Respondents

The majority of the respondents were female (n = 294, 83.1%) and White (n = 312, 88.1%); the age of the respondents ranged from 17 to 63 years (M = 28.42, SD = 9.58); a little more than half of the sample identified themselves as single, never married (n = 188, 53.9%), with just over a third being married (n = 133, 38.1%). The majority of respondents reported having no children (n = 197, 55.6%) with a mean number of .76 and a median of .00. Slightly more than half of the students were undergraduate (59%), with most of them being education majors (n = 159, 50.6%). The graduate students were all social work majors (n = 127, 40.4%). The mean years of education completed ranged between 12 and 25 with a mean of 17.18 (SD = 2.06). The majority of respondents reported having previous volunteer experience (n = 297, 83.9%), 43 reported no previous volunteer experience (n = 43, 12.1%), and 14 did not report whether they had been involved in previous volunteer experience (n = 14, 4%).

Results

Reliability

To begin the validation phase, the instrument was examined for reliability. The coefficient alpha was used to determine the reliability estimate. To create a scale that can be easily used, it was important to create an instrument with as few items as possible with the highest alpha coefficient possible. The attitude component was left with eight items from the original 11 items with a Cronbach’s alpha level of .91. The behavior component was left with six items from the original 11 items with a Cronbach’s alpha level of .85. The final items are presented in the appendix. These findings supported the internal consistency of the subscales. The strongest attitude items were “I am committed to serve in my community” and “I believe that all citizens have a responsibility to their community.” The strongest behavior items were “I help members of my community” and “I stay informed of events in my community.”

Content and Factorial Validity of the CES

For factorial validity, principal component analysis (PCA) with a varimax rotation was used to examine the factorial structure of the scale. The items from each factor were then submitted to principal axis factoring with varimax rotation. All items that loaded less than .45 in the initial analysis were deleted and the analysis was redone. PCA indicated a two-dimensional scale with a high loading on both factors (mean factor loading for the attitude factor = .79, and mean factor loading for the behavior factor = .77). These results indicate evidence of acceptable factorial validity.

To demonstrate content validity, an instrument must demonstrate that it is a valid measure of the construct that it was intended to measure (Faul & Hudson, 1997). To demonstrate content validity, the components of the total domain must be identified, and it must be demonstrated that the items clearly represent the components (Singleton & Straits, 1999). An item analysis was conducted for the CES to determine whether its items significantly contributed to the total score. Each item on the CES was correlated with its own total score and with the subscale (Table 1). The mean of all of these correlations of the Attitude subscale, after all the unwanted item self-correlations were removed, provided a content validity coefficient of .71. The mean of all these correlations of the Behavior subscale, after all the unwanted item self-correlations were removed, provided a content validity coefficient of .65.

Table 1.

Item-Total Correlations for the CES.

Attitude factor
Attitude item	CES^a
1. I feel responsible for my community.	0.69
2. I believe I should make a difference in my community.	0.73
3. I believe that I have a responsibility to help the poor and the hungry.	0.67
4. I am committed to serve in my community.	0.74
5. I believe that all citizens have a responsibility to their community.	0.76
6. I believe that it is important to be informed of community issues.	0.72
7. I believe that it is important to volunteer.	0.73
8. I believe that it is important to financially support charitable organizations.	0.64
Mean	0.71
Behavior factor
Behavior item	CES^a
1. I am involved in structured volunteer position(s) in the community.	0.62
2. When working with others, I make positive changes in the community.	0.66
3. I help members of my community.	0.76
4. I stay informed of events in my community.	0.68
5. I participate in discussions that raise issues of social responsibility.	0.55
6. I contribute to charitable organizations within the community.	0.61
Mean	0.65

Note. CES = Civic Engagement Scale.

These correlations are based on corrected item-total correlations.

Construct Validity of the CES

Construct validity, as defined by Campbell and Stanley (1963), consists of convergent and discriminant validity. To begin, to demonstrate convergent validity, the CES must correlate moderately with measures that are believed to be related to the construct. In the initial item analysis, the CES correlated well with its own total subscale score, which provides supporting evidence for convergent validity at the item level of analysis.

To demonstrate discriminant validity on the scale level, the CES must correlate poorly with variables that are believed to be unrelated to the construct. On the subscale level of analysis, it was hypothesized that the scores of the CES would not correlate well with demographic variables such as gender, number of children, and age. There is no theoretical basis for believing that the variables of gender, number of children, or age would influence the attitudes and behaviors of civic engagement. The correlations and their means are shown in Table 2 for the background measures and the total score of the attitude and behavior dimensions of the scale, which indicates an average correlation of .06 for attitudes and .17 for behaviors. These findings provide support for the beginning evidence of discriminant validity at the subscale level of analysis for the CES.

Table 2.

Correlations Among Background Measures and the Attitudes and Behaviors of the CES.

Criterion measure	Attitudes	Behaviors
Age	.05	.24
Gender	.10	.05
Number of children	.02	.21
Mean correlation	.06	.17

To further test for convergent validity, on the scale level, it was hypothesized that some factors of the CSAS would correlate with the CES. The CSAS consists of eight factors: normative helping, connectedness, costs, awareness, benefits, seriousness, career benefits, and intentions. It was hypothesized that the subscale of civic attitudes would correlate well with the subscales of normative helping and connectedness while civic behaviors would correlate well with the subscale intentions. The subscale of attitudes was correlated moderately with the subscales of normative helping (.66) and connectedness (.68). The Civic Behavior subscale also showed a moderate, but weaker than expected, correlation with the Intentions subscale (.47). These results can be seen as the beginning evidence for convergent construct validity.

Table 3 shows the correlations between the subscales of the CES and the CSAS. It is clear from the table that although it is difficult to conclude with predictions on the interrelationship between the different subscales, the correlations shown in the table provide the beginning evidence of the adequacy of the subscales relative to convergent validity.

Table 3.

Correlation Matrix for Subscale Scores Displaying the Subscales for the CES and the CSAS.

	atttot	behtot	csaout	csafeel	csaatit	helpatt	connect	costs	aware	benefit	serious	career	intent
atttot	1.00	0.61	0.15	0.66	0.58	0.66	0.68	−0.02	0.52	0.43	0.44	0.21	0.51
behtot		1.00	0.16	0.55	0.48	0.54	0.55	0.01	0.44	0.40	0.35	0.18	0.47
csaout			1.00	0.26	0.18	0.28	0.22	0.88	0.26	0.45	0.15	0.50	0.02
csafeel				1.00	0.82	0.99	0.92	−0.03	0.78	0.62	0.64	0.39	0.72
csaatit					1.00	0.82	0.87	−0.03	0.79	0.48	0.90	0.31	0.65
helpatt						1.00	0.90	−0.01	0.77	0.64	0.65	0.41	0.67
connect							1.00	−0.02	0.74	0.53	0.69	0.34	0.66
costs								1.00	0.04	0.01	−0.02	0.12	−0.19
aware									1.00	0.53	0.56	0.31	0.50
benefit										1.00	0.34	0.55	0.45
serious											1.00	0.26	0.46
career												1.00	0.24
intent													1.00

Note. CES = Civic Engagement Scale; CSAS = Community Service Attitudes Scale.

atttot and behtot were for CES; csaout, csafeel, csaatit, helpatt, connect, costs, aware, benefit, serious, career, and intent were for CSAS.

Discussion

The findings of this study indicate that the CES consists of two dimensions, attitudes and behaviors, and has good reliability. Good content validity has been substantiated and the data support the construct validity of the CES. A limitation of the study is that the sample was fairly homogeneous in that all were college students with similar fields of study. This scale was validated primarily with students studying to be service providers in their communities, that is, social work and education majors. This instrument wound need further research and testing to further validate and generalize how this scale would perform with other samples in other nonservice-related fields.

The CES has been constructed to be used in college settings and every effort has been made to make the instrument as easy to administer as possible. While this study focused on certain groups of students, the instrument could be used in different ways. This instrument could be utilized in a pretest/posttest situation where there is a question about a change in a student’s attitudes and behaviors before and after a class or semester experience and accumulation of knowledge. It is however acknowledged that the specificity of the measure was not tested; therefore, it will be important to analyze the specificity of the measure in a prepost test research study. The instrument could be utilized to assess the difference in attitudes and behaviors of students who have had a service-learning experience during the course of the semester as opposed to those who have not had that type of experience.

A limitation of this scale is that it is created to measure two dimensions of civic engagement. To get a more in-depth view of how students are affected by a service-learning experience, it may be helpful to measure other possible outcomes of that experience. The potential outcomes of service-learning may be as different as the service-learning programs themselves. Therefore, the burden lies with the researchers to identify the type of program outcomes they are measuring and to choose instruments that will measure their targeted outcomes. Another limitation is that this instrument is a self-report measure. The items are clear and the intent of the measurement is obvious. Therefore, respondents could easily choose responses that would make them appear to have stronger attitudes and behaviors that would indicate a higher level of civic engagement.

In spite of these limitations, the evidence from this study provides a good basis for recommending the use of the CES in educational and research settings of those interested in the impact of service learning on civic engagement.

Conclusion

It is believed that this study indicates that the CES can provide useful information about individuals’ attitudes and behaviors of engagement in their community. Such information could be used in educational settings where service learning is being used to assess a student’s attitudes and behaviors of civic engagement following the service-learning experience.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Amy Doolittle is an associate professor of Social Work at the University of Tennessee–Chattanooga.

Anna C. Faul is a professor at Kent School of Social Work, University of Louisville, Louisville, KY & research fellow at the Department of Social Work, University of the Free State, South Africa.

References

Barber

B. R.

(1992). An aristocracy of everyone: The politics of education and the future of America. New York, NY: Ballantine Books.

Battistoni

R. M.

(1997). Service learning and democratic citizenship. Theory Into Practice, 36, 150-156.

Boyte

H. C.

(1991). Community service and civic education. Phi Delta Kappan, 72, 765-767.

Campbell

D. E.

(2000). Social capital and service learning. PS: Political Science and Politics, 33, 641-645.

Campbell

D. T.

Stanley

J. C.

(1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.

Coleman

J. S.

(1988). Social capital in the creation of human capital. The American Journal of Sociology, 94, S94-S120.

Coles

(1993). The call of service: A witness to idealism. New York, NY: Houghton Mifflin.

DeVellis

R. F.

(1991). Scale development: Theory and applications (Applied Social Research Methods Series, No. 26). Newbury Park, CA: Sage.

Dewey

(1959). Dewey on education. New York, NY: Teachers College Press. (Original work published 1899)

10.

Dewey

(1966). Democracy and education. New York, NY: Macmillan. (Original work published 1916)

11.

Ehrlich

(1997). Civic learning: Democracy and education revisited. Educational Record, 78(3/4), 57-65.

12.

Faul

A. C.

van Zyl

M. A.

(2004). Constructing and validating a specific multi-item assessment or evaluation tool. In Roberts

A. R.

Yeager

K. R.

(Eds.), Evidence-based practice manual: Research and outcome measure in health and human services (pp. 564-581). New York, NY: Oxford University Press.

13.

Faul

A. C.

Hudson

W. W.

(1997). The index of drug involvement: A partial validation. Social Work, 42, 565-572.

14.

Flanagan

(2003). Developmental roots of political engagement. PS: Political Science & Politics, 36, 257-261.

15.

Hepburn

M. A.

(1997). Service learning in civic education: A concept with long, sturdy roots. Theory into Practice, 36(3), 136-142.

16.

Hollander

E. L.

(1999). Foreward. In Bringle

R. G.

Games

Malloy

E. A.

(Eds.), Colleges and universities as citizens (pp. v-viii). Boston, MA: Allyn & Bacon.

17.

Hyman

J. B.

(2002). Exploring social capital and civic engagement to create a framework for community building. Applied Developmental Science, 6(4), 196-202.

18.

Likert

(1932). The method of constructing an attitude scale. Archives of Psychology, 140, 44-53.

19.

Lin

(2001). Building a network theory of social capital. In Lin

Cook

Burt

R. S.

(Eds.) Social capital: Theory and research (pp. 3-29). Hawthorne, NY: Aldine De Gruyter.

20.

Lisman

C. D.

(1998). Toward a civil society: Civic literacy and service learning. Westport, CT: Bergin & Garvey.

21.

Pateman

(1970). Participation and democratic theory. England: Cambridge University Press.

22.

Portes

(1998). Social capital: Its origins and applications in modern sociology. Annual Review in Sociology, 24, 1-24.

23.

Putnam

R. D.

(1995). Bowling alone: America’s declining social capital. Journal of Democracy, 6(1), 65-78.

24.

Putnam

R. D.

(1996). The strange disappearance of civic America. The American Prospect, 24, 34-48.

25.

Putnam

R. D.

(2000). Bowling alone: The collapse and revival of American community. New York, NY: Simon & Schuster.

26.

Shiarella

A. H.

McCarthy

A. M.

Tucker

M. L.

(2000). Development and construct validity of scores on the Community Service Attitudes Scale. Educational and Psychological Measurement, 60, 286-300.

27.

Singleton

R. A.

Straits

B. C.

(1999). Approaches to social research (3rd ed.). New York, NY: Oxford University Press.

28.

Spector

P. E.

(1992). Summated rating scale construction: An introduction (Sage University Paper Series on Qualitative Applications in the Social Sciences, No. 07-082). Beverly Hills, CA: Sage.

29.

Springer

D. W.

Abell

Nugent

W. R.

(2002). Creating and validating rapid assessment instruments for practice and research: Part 2. Research on Social Work Practice, 12, 768-795.

30.

Stanton

T. K.

Giles

D. E.

Cruz

N. I.

(1999). Service-learning: A movement’s pioneers reflect on its origins, practice, and future. San Francisco, CA: Jossey-Bass.

31.

Unrau

Y. A.

Krysik

J. L.

Grinnell

R. M.

(1997). Social work research & evaluation: Quantitative and qualitative approaches (5th ed.). Itasca, IL: F. E. Peacock.

32.

Waldstein

F. A.

Reiher

T. C.

(2001). Service-learning and students’ personal and civic development. Journal of Experiential Education, 24(1), 7-13.

33.

Williams

B. N.

Gilchrist

L. Z.

(2004). Civic learning via service-learning: A proposed framework and methodology to linking servant leadership theory to the contemporary practice of community policing. Global Virtue Ethics Review, 5(3), 80-94.