In educational contexts, questioning performs a number of functions. These include facilitating learning in the classroom and the recognition of achievement through examinations and other assessments. Good quality questions are important to ensuring that these functions are achieved. This research focused on educational exams and used views from question writers to explore conceptualisations of question quality and features thought to affect question quality. Seven examination question writers from four subjects were shown some example exam questions. Participants were asked to comment on the quality of the question, reflect on performance data, rate question quality and comment more generally on how they define quality in question writing. Three conceptualisations of question quality emerged, two relating to aspects of validity and one relating to pedagogical concerns. Participants varied in which definition dominated their views. Discussions also identified question features thought to affect quality. These are similar to features previously identified as affecting difficulty and fairness by studies analysing student performance.
AhmedAPollittA (2007) Improving the quality of contextualized questions: An experimental investigation of focus. Assessment in Education: Principles, Policy and Practice14(2): 201–232.
2.
AhmedAPollittA (2011) Improving marking quality through a taxonomy of mark schemes. Assessment in Education: Principles, Policy and Practice18(3): 259–278.
3.
Bejar II (2011) A validity-based approach to quality control and assurance of automated scoring. Assessment in Education: Principles, Policy and Practice18(3): 319–341.
4.
BraunVClarkeV (2006) Using thematic analysis in psychology. Qualitative Research in Psychology3(2): 77–101.
5.
ButtnyR (1993) Social Accountability in Communication, London: SAGE.
6.
CooperBDunneM (1998) Anyone for tennis?: Social class differences in children’s difficulties with ‘realistic’ mathematics testing. Sociological Review46(1): 115–148.
7.
CooperBDunneM (2000) Assessing Children’s Mathematical Knowledge: Social Class, Sex and Problem Solving, Philadelphia, PA: Open University Press.
8.
CrispV (2008) Improving students’ capacity to show their knowledge, understanding and skills in exams by using combined question and answer papers. Research Papers in Education23(1): 69–84.
9.
CrispV (2011) Exploring features that affect the difficulty and functioning of science exam questions for all candidates and specifically for those with reading difficulties. Irish Educational Studies30(3): 323–343.
10.
CrispVJohnsonMNovakovicN (2012) The effects of features of examination questions on the performance of students with dyslexia. British Educational Research Journal38(5): 813–839.
11.
CrispVSweiryE (2006) Can a picture ruin a thousand words? The effects of visual resources in exam questions. Educational Research48(2): 139–154.
12.
CrispVSweiryEAhmedA, et al.(2008) Tales of the expected: The influence of students’ expectations on question validity and implications for writing exam questions. Educational Research50(1): 95–115.
13.
CrooksTJKaneMTCohenAS (1996) Threats to the valid use of assessments. Assessment in Education: Principles, Policy and Practice3(3): 265–286.
14.
Ernst-SlavitGPrattKL (2017) Teacher questions: Learning the discourse of science in a linguistically diverse elementary classroom. Linguistics and Education40: 1–10.
Fisher-Hoch H and Hughes S (1996) What makes mathematics exam questions difficult? In: British Educational Research Association annual conference, Lancaster, UK, 12–15 September 1996.
17.
Fisher-Hoch H, Hughes S and Bramley T (1997) What makes GCSE exam questions difficult? Outcomes of manipulating difficulty of GCSE questions. In: British Educational Research Association annual conference, York, UK, 11–14 September 1997.
18.
HeritageMHeritageJ (2013) Teacher questioning: The epicenter of instruction and assessment. Applied Measurement in Education26(3): 176–190.
19.
Hughes S, Pollitt A and Ahmed A (1998) The development of a tool for gauging the demands of GCSE and a level exam questions. In: British Educational Research Association annual conference, Belfast, UK, 26–30 August 1998.
20.
KelleyTL (1927) Interpretation of Educational Measurements, New York: New World Book Co.
21.
MayerRELarkinJHKadaneJ (1984) A cognitive analysis of mathematical problem solving ability. In: SternbergR (ed.) Advances in the Psychology of Human Intelligence, Hillsdale, NJ: Erlbaum Associates, pp. 231–273.
22.
MehanH (1979) Learning Lessons: Social Organization in the Classroom, Cambridge, MA: Harvard University Press.
23.
MessickS (1989) Validity. In: LinnRL (ed.) Educational Measurement, 3rd ed. New York: American Council on Education/Macmillan, pp. 13–103.
24.
Nickson M and Green S (1996) A study of the effects of context in the assessment of the mathematical learning of 10/11 year olds. In: British Educational Research Association annual conference, Lancaster, UK, 12–15 September 1996.
25.
OlinghouseNGWilsonJ (2013) The relationship between vocabulary and writing quality in three genres. Reading and Writing26: 45–65.
26.
Pollitt A and Ahmed A (1999) A new model of the question answering process. In: International Association for Educational assessment conference, Bled, Slovenia, 23–28 May 1999.
27.
PollittAEntwhistleNHutchinsonC, et al.(1985) What Makes Exam Questions Difficult?, Edinburgh: Scottish Academic Press.
28.
RamseierE (1999) Task difficulty and curricular priorities in science: Analysis of typical features of the Swiss performance in TIMMS. Educational Research and Evaluation5(2): 105–126.
29.
SchegloffEA (2007) Sequence Organization in Interaction: A Primer in Conversation AnalysisVol. 1, Cambridge: Cambridge University Press.
30.
SchegloffEASacksH (1973) Opening up closings. Semiotica8(4): 289–327.
31.
ShaurdHRotheryA (1984) Children Reading Mathematics, London: John Murray.
32.
SinclairJCoulthardM (1975) Towards an Analysis of Discourse: The English Used by Teachers and Pupils, Oxford: Oxford University Press.
33.
Spalding V (2011) Structuring and formatting examination papers: Examiners’ views of good practice. Report, Assessment and Qualifications Alliance, UK.
34.
StiversTRossanoF (2010) Mobilizing response. Research on Language and Social Interaction43(1): 3–31.
35.
SullivanPLilburnP (2002) Good Questions for Maths Teaching: Why Ask Them and What to Ask (K-6), Sausalito, CA: Maths Solutions.
36.
TofadeTElsnerJHainesST (2013) Best practice strategies for effective use of questions as a teaching tool. American Journal of Pharmaceutical Education77(7): Article 155.
37.
WilenW (1991) Questioning Skills, for Teachers, Washington, DC: National Education Association.
