Skip to main content

From expert feedback to AI-assisted course review: Scaling educational judgement in PAAIR

PDF

Abstract

Monash University has embarked on an ambitious Assessment Advancement project seeing the review of over 500 courses and areas of study redesigned programmatically with intentional integration of Generative Artificial Intelligence (GenAI) to prepare students for an ever-changing world. The Programmatic Assessment and AI Review (PAAIR) project at Monash University is generating rich, expert feedback on course design, assessment regimes, learning outcome alignment, and AI integration across coursework programs (Baartman et al., 2022; Fawns et al., 2026; Heeneman et al., 2021; Sotiriadou et al., 2020). GenAI is intensifying the need for scalable but trustworthy curriculum review practices that preserve human agency, accountability and professional judgement. AI governance guidance emphasises that AI should augment, not displace, human agency, with human oversight, accountability and risk management built into system design (Miao & Holmes, 2023). This poster presents emerging work to use feedback from the PAAIR Panel, a cross-faculty group of Monash education experts, to train an AI agent that can support education teams in reviewing Course Design Spreadsheets as part of course reaccreditation and continuous review. 

The proposed AI agent is deliberately positioned as an assistant, not a decision-maker. Its role is to highlight areas within a course that may warrant closer human attention, such as unclear alignment between course learning outcomes, unit learning outcomes and assessment tasks; limited evidence of progression across a course; unclear rationale for AI use or restriction; over-reliance on particular assessment types; or places where assessment security may depend too heavily on final products rather than visible evidence of learning. The AI does not replace educational judgement, approve course design, or provide final feedback to course teams. Instead, it prompts reviewers to look more closely, ask better questions, and provide contextualised human feedback.

The work responds to a key institutional challenge and a sector-wide response to enacting assessment reform in a time of AI: how to scale educative feedback across large numbers of courses while maintaining the professional judgement, disciplinary sensitivity and care that meaningful course review requires (Lodge et al., 2025). By training the agent on feedback authored by Monash education experts, the project seeks to preserve and amplify local educational expertise rather than outsource it. The AI is being designed to support consistency, reduce reviewer workload, and make recurring patterns visible across submissions, while keeping humans responsible for interpretation, dialogue and decision-making. Achieving these aims without compromising feedback quality is a high priority in our testing.

This approach contributes to broader conversations about responsible AI in higher education by exploring a human-in-the-loop model where AI assists educational teams to review complex curriculum data. The AI supports the review while decisions stay with educators. The model could transfer to institutions following a similar review process, though their policy, curriculum templates and staffing will shape feasibility. The poster will share the emerging design principles, potential workflow, and early opportunities for using expert-informed AI to support programmatic assessment and AI integration.

Keywords

Artificial intelligence, course review, assessment design, curriculum quality, peer feedback.