Evaluation of Evaluation Studies Using Qualitative Research Methods in the Social Work Literature (1990-2003): Evidence That Constitutes a Wake-Up Call
Restricted accessResearch articleFirst published online May, 2005
Evaluation of Evaluation Studies Using Qualitative Research Methods in the Social Work Literature (1990-2003): Evidence That Constitutes a Wake-Up Call
Objective: This study examines the quality of evaluation studies using qualitative research methods in the social work literature in terms of a number of criteria commonly adopted in the field of qualitative research. Method: Using qualitative and evaluation as search terms, relevant qualitative evaluation studies from 1990 to 2003 indexed by Social Work Abstracts were examined, and their quality was evaluated. Results: The review shows that the quality of published evaluation studies using qualitative research methods in the social work field is not high and that many of the reviewed studies are not sensitive to the following issues: philosophical base of the study, auditability, bias, truth value, consistency, and critical interpretations of the data. Conclusions: Social workers using findings arising from published evaluation studies using qualitative research methods in social work should be cautious and social workers conducting qualitative evaluation studies should be sensitive to the issue of quality. Adequate training for social workers on qualitative evaluation should also be carried out.
Anastas, J. W. (2004). Quality in qualitative evaluation: Issues and possible answers. Research in Social Work Practice, 14, 57-65.
2.
Atherton, C. R., & Bolland, K. A. (2002). Postmodernism: A dangerous illusion for social work. International Social Work, 45, 421-433.
3.
Batchelor, J., Gould, N., & Wright, J. (1999). Family centers: A focus for the children in need debate. Child and Family Social Work, 4, 197-208.
4.
Bloland, H. G. (1995). Postmodernism and higher education. Journal of Higher Education, 66, 521-559.
5.
Bronstein, L. R., & Kelly, T. B. (1998). A multidimensional approach to evaluating school-linked services: A school of social work and county public school partnership. Social Work in Education, 20, 152-164.
6.
Burnette, D. (1998). Teaching qualitative research: A compendium of model syllabi. Alexandria, VA: Council on SocialWork Education.
7.
Cigno, K.,& Gore, J. (1999). Aseamless service: Meeting the needs of children with disabilities through a multi-agency approach. Child and Family Social Work, 4, 325-335.
8.
Davis, D., Ray, J.,& Sayles, C. (1995). Ropes course training for youth in a rural setting: “At first I thought it was going to be boring....” Child and Adolescent Social Work Journal, 12, 445-463.
9.
De Anda, D. (2001). A qualitative evaluation of a mentor program for at-risk youth: The participants’ perspective. Child and Adolescent Social Work Journal, 18, 97-117.
10.
Dellgran, S., & Hojer, P. (2001). Mainstream is contextual: Swedish social work research dissertations and theses. Social Work Research, 25, 243-252.
11.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (2000). Handbook of qualitative research. Thousand Oaks, CA: Sage.
12.
Derezotes, D. (1995). Evaluation of the Late Nite Basketball Project. Child and Adolescent Social Work Journal, 12, 33-50.
13.
Drisko, J. W. (1997). Strengthening qualitative studies and reports: Standards to promote academic integrity. Journal of Social Work Education, 33, 185-197.
14.
Düvell, F., & Jordan, B. (2001). “How low can you go?” Dilemmas of socialwork with asylum seekers in London. Journal of SocialWork Research and Evaluation, 2, 189-205.
15.
Elks, M. A., & Kirkhart, K. E. (1993). Evaluating effectiveness from the practitioner perspective. Social Work, 38, 554-563.
16.
Erera, P. I. (1997). Empathy training for helping professionals: Model and evaluation. Journal of Social Work Education, 33, 245-260.
17.
Fischer, J. (1973). Is casework effective?Social Work, 18, 5-20.
18.
Gambrill, E. (1999). Evidence-based practice: An alternative to authority-based practice. Families in Society, 80, 341-350.
19.
Gardner, F. (2003). Critical reflection in community-based evaluation. Qualitative Social Work, 2, 197-212.
20.
Gellis, Z. D. (2001). Using a participatory research approach to mobilize immigrant minority family caregivers. Journal of Social Work Research, 2, 267-282.
21.
Goicoechea-Balbona, A., Barnaby, C., Ellis, I., & Foxworth, V. (2000). AIDS: The development of a gender appropriate research intervention. Social Work in Health Care, 30, 19-37.
22.
Gomory, T. (2000a). Critical realism (Gomory's blurry theory) or positivism (Thyer's theoretical myopia): Which is the prescription for social work research?Journal of Social Work Education, 37, 67-78.
23.
Gomory, T. (2000b). A fallibilistic response to Thyer's theory of theory-free empirical research in social work practice. Journal of Social Work Education, 37, 26-50.
24.
Guba, E. G., & Lincoln, Y. S. (1981). Effective evaluation. San Francisco: Jossey-Bass.
25.
Guba, E. G.,& Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In N. K. Denzin& Y. S. Lincoln (Eds.), Handbook of qualitative research(pp. 105-117). Thousand Oaks, CA; Sage.
26.
Hammersley, M. (1990). Reading ethnographic research: A critical guide. London: Longman.
27.
Hammersley, M. (1992). What's wrong with ethnography: Methodological explorations. London: Routledge.
28.
Heath, D. T. (1998). Qualitative analysis of private mediation: Benefits for families in public child welfare agencies. Children and Youth Services Review, 20, 605-627.
29.
Hill, M., Dillane, J., Bannister, J.,& Scott, S. (2002). Everybody needs good neighbours: An evaluation of an intensive project for families facing eviction. Child and Family Social Work, 7, 79-89.
30.
Huberman, A. M., & Miles, M. B. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: Sage.
31.
Illback, R. J.,& Kalafat, J. (1995). Initial evaluation of a school-based integrated service program: Kentucky Family Resource and Youth Services Centers. Special Services in the Schools, 10, 139-163.
32.
Janesick, V. J. (1998). The dance of qualitative research design. In N. K. Denzin & Y. S. Lincoln (Eds.), Strategies of qualitative inquiry(pp. 35-55). Thousand Oaks, CA: Sage.
33.
Lather, P. (1986). Issues of validity in openly ideological research: Between a rock and a soft place. Interchange, 17, 63-84.
34.
LeCompte, M. D., & Goetz, J. P. (1982). Problems of reliability and validity in ethnographic research. Review of Educational Research, 52, 31-60.
35.
Ligon, J., Markward, M. J., & Yegidis, B. L. (1999). Comparing student evaluations of distance learning and standard classroom courses in graduate social work education. Journal of Teaching in Social Work, 19, 21-29.
36.
Lincoln, Y. S. (1998). From understanding to action: Newimperatives, new criteria, new methods for interpretive researchers. Theory and Research in Social Education, 26, 12-29.
37.
Lincoln, Y. S.,& Guba, E. G. (2000). Paradigmatic controversies, contradictions and emerging confluences. In N. K. Denzin& Y. S. Lincoln (Eds.), Handbook of qualitative research(pp. 163-188). Thousand Oaks, CA: Sage.
38.
Mowbray, C. T., & Bybee, D. (1998). The importance of context in understanding homelessness and mental illness: Lessons learned from a research demonstration project. Research on Social Work Practice, 8, 172-199.
39.
Netting, F. E., & Williams, F. G. (2000). Expanding the boundaries of primary care for elderly people. Health and Social Work, 25, 233-242.
40.
Padgett, D. K. (1998). Qualitative methods in social work research: Challenge and rewards. Thousand Oaks, CA: Sage.
41.
Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: Sage.
42.
Patton, M. Q. (2002). Two decades of developments in qualitative inquiry. Qualitative Social Work, 1, 261-283.
43.
Platt. D. (2001). Refocusing children's services: Evaluation of an initial assessment process. Child andFamily SocialWork, 6, 139-148.
44.
Proctor, E. K. (1996). Research and research training in social work: Climate, connections, and competencies. Research on SocialWork Practice, 6, 366-378.
45.
Rehr, H.,& Epstein, I. (1993). Evaluating the Mount Sinai Leadership Enhancement Program: Adevelopmental perspective. SocialWork in Health Care, 18, 79-99.
46.
Reid, W. J.,& Hanrahan, P. (1982). Recent evaluations of social work: Grounds for optimism. Social Work, 27, 328-340.
47.
Ringma, C., & Brown, C. (1991). Hermeneutics and the social sciences: An evaluation of the function of hermeneutics in a consumer disability study. Journal of Sociology and Social Welfare, 18, 57-73.
48.
Rivard, J. C., Johnsen, M. C., Morrissey, J. P.,& Starrett, B. E. (1999). The dynamics of interagency collaboration: Howlinkages develop for child welfare and juvenile justice sectors in a system of care demonstration. Journal of Social Service Research, 25, 61-82.
49.
Rizzo, T. A., Corsaro, W. A., & Bates, J. E. (1992). Ethnographic methods and interpretive analysis: Expanding the methodological options of psychologists. Developmental Review, 12, 101-123.
50.
Rodwell, M. K., & Woody, D. (1994). Constructive evaluation: The policy/practice context. In E. Sherman & W. J. Reid (Eds.), Qualitative research in social work(pp. 315-327). New York: Columbia University Press.
51.
Rosenau, P. M. (1992). Post-modernism and the social sciences insights, inroads and intrusions. Princeton, NJ: Princeton University Press.
52.
Rubin, A. (1985). Practice effectiveness: More grounds for optimism. Social Work, 30, 469-476.
53.
Rubin, A. (2000). Editorial: Socialwork research at the turn of the millennium: Progress and challenges. Research on Social Work Practice, 10, 9-14.
54.
Ryan, K. (1998). Advantages and challenges of using inclusive evaluation approaches in evaluation practice. American Journal of Evaluation, 19, 101-122.
55.
Salcido, R. M., & Cota, V. (1995). Cross-cultural training for child welfare workers when working with Mexican-American clients. Journal of Continuing Social Work Education, 6, 39-46.
56.
Sandelowski, M. (1986). The problem of vigor in qualitative research. Advances in Nursing Science, 8, 27-37.
57.
Schwandt, T. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2, 58-72.
58.
Schwandt, T. A. (2000). Three epistemological stances for qualitative inquiry: Interpretivism, hermeneutics, and social constructionism. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research(pp. 189-213). Thousand Oaks, CA: Sage.
59.
Seale, C. (1999). The quality of qualitative research. Thousand Oaks, CA: Sage.
60.
Seale, C. (2002). Quality issues in qualitative inquiry. Qualitative Social Work, 1, 97-110.
61.
Segal, S. P. (1972). Research on the outcomes of social work interventions: A review of the literature. Journal of Health and Social Behavior, 13, 3-17.
62.
Smith, J. K., & Deemer, D. K. (2000). The problem of criteria in the age of relativism. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research(pp. 877-896). Thousand Oaks, CA; Sage.
63.
Sokal, A., & Bricmont, J. (1998). Fashionable nonsense: Postmodern intellectuals’ sense of science. New York: Picador USA.
64.
Steier, F. (Ed.). (1991). Research and reflexivity. London: Sage.
65.
Stout, K. D. (1991). A continuum of male controls and violence against women: A teaching model. Journal of Social Work Education, 27, 305-319.
66.
Sun, A. (1998). Organizing an international continuing education program: A preliminary study based on participants’ perspectives. Journal of Continuing Social Work Education, 6, 23-29.
67.
Thyer, B. A. (1989). First principles of practice research. British Journal of Social Work, 19, 309-323.
68.
Thyer, B. A. (1991). Guidelines for evaluating outcome studies on socialwork practice. Research on SocialWork Practice, 1, 76-91.
69.
Tschudi, F. (1989). Do qualitative and quantitative methods require different approaches to validity? In S. Kvale (Ed.), Issues of validity in qualitative research(pp. 109-134). Lund, Sweden: Studentlitteratur.
70.
Vera, M. I. (1990). Effects of divorce groups on individual adjustment: A multiple methodology approach. Social Work Research and Abstracts, 26, 11-20.
71.
Walsh-Bowers, R., & Basso, R. (1999). Improving early adolescents’ peer relations through classroom creative drama: An integrated approach. Social Work in Education, 21, 23-32.
72.
Weissman, H. H. (1981). Teaching qualitative research methods. In S. Briar, H. Weissman, & A. Rubin (Eds.), Research utilization in social work education(pp. 59-65). New York: Council on Social Work Education.
73.
Whipple, E. E., Grettenberger, S. E., & Flynn, M. S. (1997). Doctoral research education in a social context: Development and implementation of a model outreach course. Journal of Teaching in Social Work, 14, 3-26.
