Course enhancement conversations: A holistic and collaborative evaluation approach to quality improvement in higher education

Abstract

This practice article describes the use of collaborative evaluation for institutional improvement leading to continuous learning at a regional university in Australia. The University of Southern Queensland’s Academic Quality Framework (2019–2022) was developed in response to external drivers to improve practices relating to the comprehensive and systematic analysis of academic data. One aspect of the response was to introduce ‘course enhancement conversations’, which were a collaboration between the central Academic Quality Unit and academic staff, including course teams and learning and teaching leaders within schools and faculties. A feature of these conversations was the use of sentinel rather than performance indicators of course (not teaching) quality to prompt the holding of conversations. Conversations were conducted in a spirit of constructive collaboration, where the shared goal was to support course teams to deliver an outstanding student experience. Through adopting a collaborative approach to evaluation, Academic Quality Unit staff were evaluation champions who gradually acculturated academic staff and learning and teaching leaders to evaluative thinking and data-informed decision-making. Issues brought to light during these conversations have informed changes in practice at the university level, including the development of a new evaluation procedure to embed best-practice monitoring and evaluation across all levels of curriculum delivery.

Keywords

internal evaluation academic quality student evaluations capacity building collaborative evaluation

Get full access to this article

View all access options for this article.

References

Abrami

P. C.

Rosenfeld

Dedic

(2007). The dimensionality of student ratings of instruction: An update on what we know, do not know, and need to do. In Perry

R. P.

Smart

J. C.

(Eds.), The scholarship of teaching and learning in higher education: An evidence-based approach (pp. 385–456). Springer.

Alderman

(2019). University of Southern Queensland academic quality framework. Unpublished internal report.

Alderman

(2022). Introducing the Continuous Learning Framework : A strategic decision-making tool for internal evaluators. Evaluation Journal of Australasia, 22(4), 206–220. https://doi.org/10.1177/1035719X221120294

Alderman

Towers

Bannah

(2012). Student feedback systems in higher education: A focused literature review and environmental scan. Quality in Higher Education, 18(3), 261–280. https://doi.org/10.1080/13538322.2012.730714

Arnold

M. E.

(2006). Developing evaluation capacity in extension 4-H field faculty: A framework for success. American Journal of Epidemiology, 27(2), 257–269. https://doi.org/10.1177/1098214006287989

Australian Qualifications Framework . (2013). AQF glossary of terminology. https://www.aqf.edu.au/framework/australian-qualifications-framework#the-aqf-second-edition-january-2013

Bruffee

(1999). Collaborative learning: Higher education, interdependence, and the authority of knowledge. (2nd ed.). Johns Hopkins University Press.

Deming

W. E.

(1986). Out of the crisis. MIT Press.

Fetterman

D. M.

Rodríguez-Campos

Zukoski

A. P.

(2018). Collaborative, participatory, and empowerment evaluation: Stakeholder involvement approaches. The Guilford Press.

10.

Forbes

(2016). Playing the changes: An expanded view of higher music education through the use of collaborative learning and teaching [Doctoral dissertation, Griffith University Queensland Conservatorium]. https://eprints.usq.edu.au/30694/

11.

Forbes

(2020). The value of collaborative learning for music practice in higher education. British Journal of Music Education, 37(3), 207–220. https://doi.org/10.1017/S0265051720000200

12.

Forbes

(2022). Learning to make a difference: Value creation in social learning spaces [Book review]. Evaluation Journal of Australasia, 22(1), 55–57. https://doi.org/10.1177/1035719X211038767

13.

Heffernan

(2022). Sexism, racism, prejudice, and bias: A literature review and synthesis of research surrounding student evaluations of courses and teaching. Assessment & Evaluation in Higher Education, 47(1), 144–154. https://doi.org/10.1080/02602938.2021.1888075

14.

Higher Education Standards Framework (Threshold Standards) 2015 (Cth) . (2015). https://www.legislation.gov.au/Details/F2015L01639

15.

Johnson

Greenseid

L. O.

Toal

S. A.

King

J. A.

Lawrenz

Volkov

(2009). Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Epidemiology, 30(3), 377–410. https://doi.org/10.1177/1098214009341660

16.

Kitto

Williams

Alderman

(2019). Beyond average: Contemporary statistical techniques for analysing student evaluations of teaching. Assessment & Evaluation in Higher Education, 44(3), 338–360. https://doi.org/10.1080/02602938.2018.1506909

17.

Lakeman

Coutts

Hutchinson

Lee

Massey

Nasrawi

Fielden

(2021). Appearance, insults, allegations, blame and threats: An analysis of anonymous non-constructive student evaluation of teaching in Australia. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2021.2012643

18.

Lynch

M.,

Murphy

(2020). Course enhancement conversations: Supporting protocol. Unpublished internal report.

19.

Maloney

(2019). Book review: Collaborative, participatory, and empowerment evaluation: Stakeholder involvement approaches. Evaluation Journal of Australasia, 19(2), 107–109. https://doi.org/10.1177/1035719X19848528

20.

Marsh

(2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In Perry

R. P.

Smart

J. C.

(Eds.), The scholarship of teaching and learning in higher education: An evidence-based approach (pp. 319–383). Springer.

21.

Murphy

(2019). My Opinion review report. Unpublished internal report.

22.

Patton

(2008). Utilization-focused evaluation. SAGE.

23.

Patton

(2012). Essentials of utilization-focused evaluation. SAGE.

24.

Rodríguez-Campos

(2012). Advances in collaborative evaluation. Evaluation and Program Planning, 35(4), 523–528. https://doi.org/10.1016/j.evalprogplan.2011.12.006

25.

Rog

D. J.

(2012). When background becomes foreground: Toward context-sensitive evaluation practice. New Directions for Evaluation, 2012(135), 25–40. https://doi.org/10.1002/ev.20025

26.

Rogers

A.,

Gullickson

(2018). Evaluation champions: A literature review. Journal of MultiDisciplinary Evaluation. 14(30), 46–63. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/495/443

27.

Rogers

Kelly

L. M.

McCoy

(2019). Evaluation literacy: Perspectives of internal evaluators in non-government organizations. Canadian Journal of Program Evaluation, 34(1), 1–20. https://doi.org/10.3138/cjpe.42190

28.

TEQSA . (2018). University of Southern Queensland: Re-registration 26 April 2018 [Web page]. Accessed at https://www.teqsa.gov.au/national-register/condition-decision/re-registration-26-april-2018

29.

University of Southern Queensland . (2019a). Course level reports. Unpublished internal report.

30.

University of Southern Queensland . (2019b). Survey level reports. Unpublished internal report.

31.

University of Southern Queensland . (2021). Academic quality framework final implementation report. Unpublished internal report.