Skip to main content

Making IOAs work at scale: An evidence-based framework from six semesters of implementation

PDF

Abstract

The widespread availability of generative artificial intelligence (GenAI) has increased demand for assessment approaches that provide stronger assurance of student learning while maintaining authentic educational value. Interactive Oral Assessments (IOAs) have emerged as a promising solution, but practical guidance for implementing them sustainably in large undergraduate subjects remains limited. This paper proposes a framework grounded in six consecutive semesters of continuous IOA implementation in a large undergraduate computing subject enrolling between 300 and 560 students per offering. Evidence draws on cohort mark distributions, IOA–examination correlations, academic integrity outcomes, and examiners’ consistency, as reflected in inter-rater reliability data. Repeated iterations of IOA design, delivery, evaluation, and refinement, informed by the literature and feedback from students and staff, produced four design principles critical to successful implementation of IOAs at scale. These principles address examiner calibration, rubric design, question depth and individual mark weighting. They are synthesised into an operational blueprint that covers assessment architecture, rubric design, question development, examiner preparation, and student preparation. Together, these provide transferable guidance for institutions seeking to implement IOAs sustainably at scale in the GenAI era.

Keywords

Interactive Oral Assessment, Academic Integrity, Generative AI, higher education, Constructive Alignment, longitudinal case study.