This is a mixed methods investigation of consistency in PhD examination. At its core is the quantification of the content and conceptual analysis of examiner reports for 804 Australian theses. First, the level of consistency between what examiners say in their reports and the recommendation they provide for a thesis is explored, followed by an examination of the degree of discrepancy between examiner recommendations and university committee decisions on the theses. Two groups of discrepant recommendations are identified and analysed in depth. Finally the main sources of inconsistency are identified. It was found that the comments of a small minority of examiners were inconsistent with each other or with the committee decision in a significant way. Much more commonly the texts of examiner reports were highly consistent and were closely reflected in the final committee decision.
BourkeS.HattieJ.AndersonL. (2004) Predicting examiner recommendations on PhD theses. International Journal of Educational Research, 41(2), 178–194.
2.
BourkeS.HolbrookA.LovatT. (2005, June). Using examiner reports to identify quality in PhD theses. Paper presented at the AARE Focus Conference: Quality in educational research, Cairns. Retrieved 17 January 2008 from http://www.aare.edu.au/05papc/bo05011y.pdf
3.
BourkeS.HolbrookA.LovatT. (2007a). Relationships of PhD candidate, candidature and examination characteristics with thesis outcomes. Paper presented at the Annual Conference of the AARE, Adelaide, November 2006. Retrieved 17 January 2008 from http://www.aare.edu.au/06pap/bou06655.pdf
4.
BourkeS.HolbrookA.LovatT. (2007b). Examiners and examination outcomes. In DenholmC.EvansT. (Eds), Supervising doctorates downunder. Melbourne: ACER Press
5.
DenicoloP. (2003). Assessing the PhD: A constructive view of criteria. Quality Assurance in Education, 11(2), 84–91.
6.
GilbertR. (2004). A framework for evaluating the doctoral curriculum. Assessment and Evaluation in Higher Education, 29(3), 299–309.
7.
GoldeC.WalkerG., & Associates (2006). Envisioning the future of doctoral education: Preparing stewards of the discipline. Carnegie Essays on the DoctorateSan Francisco: Jossey-Bass.
8.
GrabbeL. (2003). The trials of being a PhD external examiner. Quality Assurance in Education, 11(2), 128–133.
9.
HoddellS.StreetD.WildbloodH. (2002). Doctorates: converging or diverging pattterns of provision. Quality assurance in education10(2), 16–70.
10.
HolbrookA.BourkeS.FairbairnH.LovatT. (2007). Examiner comment on the literature review in PhD theses. Studies in Higher Education, 32(3) ((in press: accepted June 14, 2006).
11.
HolbrookA.BourkeS. (2004). An investigation of PhD examination outcome in Australia using a mixed method approach. Australian Journal of Educational and Developmental Psychology, 4, 153–169.
12.
HolbrookA.BourkeS.LovatT.DallyK. (2004a) Investigating PhD thesis e xamination reports. International Journal of Educational Research, 41(2), 178–194.
HolbrookA.BourkeS.LovatT.DallyK. (2004c). PhD theses at the margin: Examiner comment of re-examined theses. Melbourne Studies in Education, 45(1), 89–115.
15.
JacksonC.TinklerP. (2001). Back to basics: a consideration of the purposes of the Ph.D. Assessment and Evaluation in Higher Education, 26(4), 355–366.
16.
LawsonA.MarshH.TansleyT. (2003). Examining the examiners. Australian Universities Review, 46(1), 32–36.
17.
KwiramA. (2006). Time for Reform? In GoldeC.WalkerG. and Associates (Eds), Envisioning the future of doctoral education: Preparing stewards of the discipline. Carnegie Essays on the DoctorateSan Franscisco: Jossey-Bass.
18.
LovatT.MonfriesM.MorrisonK. (2004). Ways of knowing and power discourse in doctoral examination. International Journal of Educational Research, 41(2), 163–177.
LovittsB. (2007). Making the implicit explicit: Creating performance expectations for the dissertation. Sterling, VA: Stylus Publishing.
21.
MorleyL.LeonardD.DavidM. (2002). Variations in Vivas: quality and equality in British PhD assessments. Studies in Higher Education, 27(3), 263–272.
22.
MullinsG.KileyM. (2002). ‘It's a Ph.D., not a Nobel Prize’: how experienced examiners assess research theses. Studies in Higher Education, 27(4), 369–386.
23.
NettlesM.MillettC. (2006). Three magic letters: Getting to Ph.D. Baltimore, MD: Johns Hopkins University Press.
24.
PowellS.McCauleyC. (2002). Research degree examining—common principles and divergent practices. Quality Assurance in Education, 10(2), 104–115.
25.
PowellS.McCauleyC. (2003). The process of examining research degrees: some issues of quality. Quality Assurance in Education, 11(2), 73–83.
26.
ShawM.GreenD. (2002). Benchmarking the Ph.D.—a tentative beginning. Quality Assurance in Education, 10(2), 116–124.
27.
SlobodaJ.A.NewsteadS.E. (1997, September). Guidelines for examiners: An evaluation of impact. Psychologist, 407–410.
28.
TeddlieC.TashakkoriA. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In TashakkoriA.TeddlieC. (Eds), Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications.
29.
TinklerP.JacksonC. (2000). Examining the doctorate: Institutional policy and the Ph.D. examination process in Britain. Studies in Higher Education, 25(2), 67–180.
30.
TinklerP.JacksonC. (2004). The doctoral examination process: A handbook for students, examiners and supervisors. Berkshire, England: Society for Research into Higher Education and Open University Press.
