Rethinking assessment strategy in the age of Generative AI and considering programmatic design

Results from an action-research study

Abstract

Generative Artificial Intelligence (Gen-AI) presents a profound challenge for designing and implementing authentic tertiary assessment. The rapid adoption of platforms such as OpenAI’s ChatGPT has intensified concerns about assessment validity and integrity (Namoun et al., 2024). These concerns are particularly salient in health science (HS) programs, where excessive reliance on Gen-AI may undermine the development of clinical reasoning and decision-making competencies, affecting professional readiness and patient safety.

This presentation reports on a project employing a three-cycle action research methodology to enhance responsiveness to Gen-AI–related assessment challenges. Cycle 1 involved developing an innovative Assessment Appraisal Tool aligned with national and institutional integrity guidelines (Flinders University, 2023; Lodge et al., 2023a, 2023b; Monash University, 2023; TEQSA, 2022; Torrens University, n.d.). The tool assigns an AI risk score (out of 30) indicating the likelihood that an assessment could be generated by Gen-AI and was piloted across three foundational HS subjects.

Cycle 2 refined the tool based on feedback and extended its application to additional HS subjects, identifying targeted strategies for redesign. Cycle 3, currently underway, implements and evaluates these redesigns to improve authenticity, learning, and academic integrity. We will also discuss the emerging theoretical links between this project and programmatic assessment design. We will conclude by considering the strategic and practical meeting points where this project may align with future institution-wide programmatic assessment design strategies, addressing the ongoing challenges of Gen-AI while ensuring assessment of learning.