Abstract
Reviewing university course profiles against institutional policies remains a largely manual process that is time-consuming and repetitive. This paper presents the design of an AI-assisted framework for course profile university policy compliance checking that combines Large Language Model (LLM)-based semantic analysis with deterministic rule-based validation, together with the initial implementation and evaluation of its LLM-based validation component. A prototype using Claude Sonnet 4.5 was developed and evaluated though an exploratory case study on a published course profile, using a baseline scenario and three modified versions with deliberately introduced policy violations. Although the LLM was able to recognise a stated policy exception occasionally, it produced false positives on the compliant baseline course profile, failed to detect a numerical error, and returned inconsistent validation outcomes for the same input. These findings show that LLM-only validation is not sufficiently reliable, motivating a combined approach that integrates deterministic rule-based validation with an LLM-based agentic workflow. This work represents the first stage of a broader study investigating hybrid AI-assisted approaches to course profile policy validation in higher education.
Keywords
Compliance checking, course profiles, Large language models (LLMs), educational technology, higher education, agentic workflow, case study.
