Skip to main content

Mapping assessor judgement-in-action: A transition network analysis of dialogic moves in interactive oral assessment

PDF

Abstract

Interactive Orals (IOs) are increasingly being adopted in higher education as an authentic and academically rigorous form of assessment. Despite the assessor’s central role in shaping the dialogic interaction, research has focused predominantly on student experience and assessment design, with limited empirical investigation of how assessors enact real-time dialogic practices. This study addresses this gap by examining assessor dialogue during real-time IOs, with attention to the functions and sequencing of assessors’ contributions. Drawing on audio-recorded and transcribed IO sessions from three courses across three institutions, involving seven assessors and 235 IO sessions in the larger study, the paper reports preliminary findings from 25 IOs involving two assessors with different levels of IO assessment experience. Assessor dialogue was coded into 13 substantive dialogic moves. Transition Network Analysis (TNA) was used to model transitions between dialogic moves and compared between experienced and novice assessors. Across 25 sessions, 723 assessor moves were coded. ACKNOWLEDGE was the most common dialogic move (30.4%), functioning as a central interactional hub. Six transitions significantly differed between the experienced and novice assessors. The largest difference was ELICIT to ACKNOWLEDGE, which was more probable for the experienced assessor, while ACKNOWLEDGE to ELICIT and REFRAME to REFRAME moves were more probable for the novice assessor. The initial findings suggest that assessor expertise may be expressed through how they take up student responses and orchestrate subsequent moves rather than the type of the questions. The paper proposes assessor dialogic orchestration as a useful conceptualisation of judgement-in-action and discusses implications for assessor training, moderation and assessment quality. However, the study does not establish that particular dialogic moves produce better student responses or assessment outcomes; these relationships require investigation in subsequent research.

Keywords

Interactive oral assessment, dialogic orchestration, dialogic moves, transition network analysis, assessor training.

Author Biography

  • Popi Sotiriadou, Griffith University

    Associate Professor Popi Sotiriadou (PFHEA) serves as Deputy Dean of the Centre for Collaborative Educational Excellence at Griffith University.