2024: ASCILITE 2024 Conference Proceedings

ASCILITE Conference - Full Papers

• Integrating ICAP theory in learning analytics: A model for evaluating student engagement in online courses

  Eamon Vale, Damir Mitric, Noor Raheem

  This paper introduces a conceptual evaluation learning analytics (LA) model developed by Keypath Education and Melbourne Business School to analyse student engagement in an online learning environment. The paper describes how a targeted LA intervention was grounded within the ICAP theory for active learning to identify, visualise and analyse student engagement metrics in a 100% online setting. The architecture of the model, including its design principles and underlying assumptions, are broadly examined against the latest research in educational, LA, and data analytics research. This includes a discussion of our initial phase testing results of captured student data alongside a discussion of key questions and desired study outcomes for the upcoming phase 2 of the data intervention. As a result, this paper provides both specific insights into how we created an evidence based and pedagogically sound evaluation model of online student engagement within a specific suite of asynchronous educational tools, as well as more general and practical insights into how other universities could develop similar approaches to enhance understanding and support of a growing online student population.

  Read article: Integrating ICAP theory in learning analytics: A model for evaluating student engagement in online courses→

  • PDF

• Challenges for being self-directed in content and language integrated learning with instructional videos

  Shihua Yu, Michael Henderson, Thi Kim Anh Dang

  Self-directed skills are crucial for video-based learning as learners often need to manage their pace, set goals, and engage with content independently. While much research focuses on the effectiveness of design and design principles in facilitating self-directed learning (SDL) in multimedia environments, boundary conditions arise due to the highly individualized and unique nature of each learning trajectory. This paper reports the findings of a case study involving five university students who speak English as an additional language (EAL). It explores the difficulties they faced while learning mathematics with instructional videos and the approaches they used to address these challenges in the context of Content and Language Integrated Learning (CLIL). Findings from video-stimulated recall (VSR) and semi-structured interviews reveal that language issues influenced and drove many decisions in their self-directed learning paths. Notably, the challenges of grasping academic language and terminology led participants to spend considerable time and effort on navigating video designs to aid language comprehension and on seeking extra resources to better understand the lectures. However, the lack of timely feedback and expert guidance posed further challenges for these EAL learners in identifying their knowledge gaps, thereby complicating the search for resources appropriate to their language and academic levels.

  Read article: Challenges for being self-directed in content and language integrated learning with instructional videos →

  • PDF

• Inequity in exam experience: Large-scale survey of proctoring and online exam experience by gender, internationality and language

  Jennifer Chung, Alice Shihua Yu, Michael Henderson

  Online digital exams, both with and without online invigilation, have become a common form of assessment in Higher Education, particularly since the COVID-19 pandemic. However, the impact of online exams and online invigilation (proctoring) on student experience, stress levels, and academic performance remains underexplored, especially for disadvantaged or underrepresented student groups. Despite growing concerns over inequitable outcomes in Higher Education, there is limited large-scale research investigating how these online assessment methods may exacerbate existing disparities. This paper addresses this gap by analysing data from a large-scale survey (N = 16,010) conducted across five semesters at an Australian University (2021-2023). Specifically, it examines the exam preparedness and experiences of on-campus students who sat online digital exams with and without proctoring. By exploring the intersectionality of gender, international status, and language, the study aims to assess whether and how online proctoring exacerbates inequities among disadvantaged student populations. Our findings suggest that social and cultural factors significantly influence exam preparedness, stress levels, and perceived academic performance, with proctoring amplifying these disparities.

  Read article: Inequity in exam experience: Large-scale survey of proctoring and online exam experience by gender, internationality and language→

  • PDF

• Navigating integrity and innovation: Case studies of generative AI integration from an Arts Faculty

  Josephine Hook, Andrew Junor, Cathy Sell, Carmen Sapsed

  As generative artificial intelligence (GenAI) continues to disrupt the way higher education is delivered, universities are responding at pace with strategies and resources to ensure responsible and ethical use of the technology by our students as they prepare for future professions. The integration of GenAI into teaching and learning activities is being diffused into educators’ practice on a spectrum from caution through to positive educational transformation. In this paper, a team of educational designers in an Arts Faculty shares case studies and reflections from our educator colleagues to explore the uneven uptake of the technology. We argue that the focus on assessment security is blurring the potential of GenAI to create collaborative, exploratory and expansive learning experiences for our students. We suggest a programmatic approach to ‘becoming’ a graduate will allow space for educators to interrogate their teaching, learning and assessment activities to ensure our graduates can imagine themselves as professional selves in an uncertain future.

  Read article: Navigating integrity and innovation: Case studies of generative AI integration from an Arts Faculty→

  • PDF

• From Campus to Career: Leveraging Technology to Improve Work Readiness and Industry Engagement

  Amanda Samson, Eduardo Araujo Oliveira, Suhrid Gupta

  Students undertaking professional degrees are often expected to step into the workplace fully formed, with unrealistic expectations around their experience and capabilities. Universities, facing an increasingly competitive market in a neoliberal context, are focused the student experience, attraction and retention metrics, as students demand return on their investment of money and time. Alongside this, criticism of work readiness and employability comes from graduates and the industries that need them. It is in this space that technology and innovation can offer simple, albeit disruptive ways of increasing the possibility of connecting tertiary students with industry partners to facilitate authentic learning. Early workplace experience, undertaken while still studying, provides both students and employers an opportunity to understand their respective needs within a temporary low risk and low stakes environment. A platform for connecting students with industry for paid employment experiences that sit outside the ubiquitous and often unpaid internships and practicums, offers authentic ways of preparing graduates for the workforce. It also has the potential to improve the way industry connects with the university enabling a richer experience for all; students, universities and industry partners.

  Read article: From Campus to Career: Leveraging Technology to Improve Work Readiness and Industry Engagement→

  • PDF

• Mediating connection at scale through space design: The University of Sydney Business School CONNECTspace

  Peter Bryant

  This study explores the design of CONNECTspace, a novel teaching environment at the University of Sydney Business School, aimed at fostering and enabling connected learning at scale. Using participatory action research and cogenerative dialogue, the project addresses the challenge of creating spaces that build connections between students, staff, and disciplines. The design focuses on five key capacities: engaging with the crowd, the group, the individual and the knowledge, generating ideas, sharing work, building social connections, and creating safe spaces for success. The study highlights the importance of changing dynamics, functions, and relationships within teaching spaces to support connectivity, creativity, and effective learning. The findings offer insights for designers and educators seeking to transform traditional teaching spaces into collaborative, connected learning environments that transform both the structural and pedagogical experience of teaching at scale.

  Read article: Mediating connection at scale through space design: The University of Sydney Business School CONNECTspace→

  • PDF

• Applying community of inquiry in a collaborative experiential learning course

  Andi Sudjana Putra

  A collaborative experiential learning course involves two or more learning institutions. Operational constraints in designing such a course, especially when the institutions are in different locations, motivated the application of Community of Inquiry (COI) with online components, which typically finds application in blended learning. This article describes a new application of COI in a collaborative experiential learning course with a case study that involved two universities at different countries. The course included students from a university learning about Sustainable Development Goals (SDG) in communities at the locality of another university. The application of COI was measured using factor pattern matrix (FPM). Students’ perceived learning was measured using reflection and course feedback. COI was rated well by students from both universities with indication of strong social and cognitive presences. The cognitive presence of visiting students appeared dominant through activities, while of host students through reflection. The study describes a new application of COI in areas other than blended learning.

  Read article: Applying community of inquiry in a collaborative experiential learning course→

  • PDF

• Designing Immersive Reality Environments in Healthcare Education: A Proposed Research Framework

  Md Haseen Akhtar, Thomas Cochrane, Mark Anderson

  This paper presents a proposed research framework aimed at addressing the critical need for innovative approaches in healthcare education by exploring the implementation of Augmented Reality (AR) and Virtual Reality (VR) technologies across Australasia. Utilizing an Educational Design Research (EDR) methodology, the proposed study seeks to develop a comprehensive, context-sensitive framework for the effective integration of AR and VR in healthcare curricula. The study navigates the unique challenges posed by Australasia's diverse cultural landscape, varying resource availability, and distinct institutional policies. Through iterative cycles of analysis, design, development, and evaluation, the proposed research intends to create and assess contextualized AR/VR experiences, their impact on student learning outcomes, and their potential to enhance healthcare education quality. The resulting framework and guidelines are intended to serve as a robust foundation for healthcare institutions across Australasia, potentially revolutionising healthcare professional training and ultimately contributing to improved patient care and public health outcomes. This proposed research not only addresses a significant gap in the current literature but also provides practical, evidence-based solutions for the successful adoption of cutting-edge technologies in healthcare education. Finally, the proposed framework generalisability to a global context focusing on the United Nations Sustainable Development Goals of quality education and good health for all.

  Read article: Designing Immersive Reality Environments in Healthcare Education: A Proposed Research Framework →

  • PDF

• Combining self-reported and clickstream data to understand university students’ learning experiences and outcomes in flipped classrooms

  Feifei Han

  This study examined the relationships between students’ self-reported perceptions of learning environment, students’ course marks, learning strategy and online study duration measured by clickstream data in an undergraduate flipped classroom course amongst a cohort of Australian first-year university students. The clickstream data identified two groups of students using two distinct learning strategies focusing either on completing the assessments in the course or on the learning activities preparing for the face-to-face learning. Students’ perceptions of learning environment were measured by a self-reported questionnaire. Using cluster analysis, students were grouped as either having positive or negative perceptions. Of the students who adopted a learning strategy focusing on assessments, a significantly higher proportion of them hold negative perceptions of learning environment. In contrast, amongst students who used a learning strategy focusing on completing the preparation activities, a significantly lower proportion of them had negative perceptions. Furthermore, students who hold negative perceptions were also found to study less hours than their classmates who had positive perceptions. The multiple regression analyses showed that self-reported perceptions could explain only 4.8% of variance in students’ course marks, whereas learning strategy and online study duration measured by clickstream data predicted around 17.2% of variance in students’ course marks.

  Read article: Combining self-reported and clickstream data to understand university students’ learning experiences and outcomes in flipped classrooms →

  • PDF

• Do Beliefs About Intelligence Drive Engagement in Online Learning? Exploring the Impact of Growth and Fixed Mindsets

  Zijin Chen, Yige Song, Andrew Valentine, Paula de Barba, Shannon Rios, Eduardo Oliveira

  This study investigates the relationship between students’ beliefs about intelligence (fixed vs. growth mindset) and their engagement and behaviour in an online learning platform. The data was collected from 28 students enrolled in COMP90082 – Software Project subject at The University of Melbourne. Students’ beliefs about intelligence were measured through a survey administered during the first week, while their online learning behaviours were captured through clickstream data from the learning management system. The study adopted a two-fold approach: first, students’ learning strategies were identified through cluster analysis of their online session duration. By examining over 41,000 learning actions —such as accessing course materials, participating in discussions, and completing assignments— three distinct learning strategies emerged: “light”, “light-intensive”, and “intensive.” Subsequently, the relationship between these learning strategies and students’ beliefs about intelligence was examined using data visualisation techniques, chi-square tests, and logistic regression models. The findings revealed that students with a growth mindset were more likely to adopt intensive and engaged learning approaches within the online platform. Conversely, those with a fixed mindset tended to exhibit lower engagement levels and less intensive learning strategies. The study may contribute to the growing literature on the psychology of online learning and highlight the potential for tailoring instructional interventions and platform design to foster a growth mindset, thereby enhancing student engagement and academic success in digital learning environments.

  Read article: Do Beliefs About Intelligence Drive Engagement in Online Learning? Exploring the Impact of Growth and Fixed Mindsets→

  • PDF

• “All things are ready, if our mind be so”: Attitudes to STEM assessment redesign in the age of genAI

  Christopher Bridge, Ryan Naylor, Birgit Loch

  In the two years since the launch of ChatGPT in November 2022, the higher education sector in Australia has experienced both the great potential of generative artificial intelligence (genAI) to transform learning and teaching, but also the serious threat to assessment security and assurance of learning that it poses. In light of recent regulatory calls for accountability in academic integrity, this article focuses on the issue of assessment security and redesign, given the affordances of readily available large language models. A nation-wide survey of STEM academics revealed that 37% of respondents had not even tested their assessments in a genAI app; that there was little consensus – and possible misunderstandings – concerning approaches for improving assessment security; and that sector recommendations were either not well understood, or not favoured, particularly those advocating a move towards program-level assurance of learning and only securing assessments at key points of a degree program. These results highlight the need for significant support and guidance for teaching academics—in terms of both creating a clear understanding of how best to respond to assessment redesign as future consensus emerges, and institutional workload in implementing those responses —in order for challenges posed by genAI to assurance of learning to be met successfully.

  Read article: “All things are ready, if our mind be so”: Attitudes to STEM assessment redesign in the age of genAI→

  • PDF

• HCI Teaching and Learning Model Using Studio-Based Learning

  Runchen Zhang, Arzoo Atiq, Antony Chacon Salas

  This research explores Studio-based Learning (SBL) within Human-Computer Interaction (HCI) subjects, focusing on the crucial elements required in the design of maker spaces to enhance user experience (UX) design education. As HCI becomes increasingly complex and interdisciplinary, traditional classroom settings often fail to provide the required experiential learning environment for certain subjects. Our study examines various subjects in the discipline, highlighting diverse interactivity requirements and the potential of Studio-based Learning to effectively address students' and staff’s needs. We analyse the impact of studio settings on the learning outcomes and expectations of students, especially in fostering creativity, collaboration, and hands-on engagement. Our research findings indicate that studio-based classrooms significantly improve student learning experiences in interactive and practical subjects, although effective planning around space and time constraints can impede its full implementation.

  Read article: HCI Teaching and Learning Model Using Studio-Based Learning→

  • PDF

• From How Much to Whodunnit: A framework for authorising and evaluating student AI use

  Kate Tregloan, Huiseung (Sarah) Song

  The arrival of ChatGPT and other generative AI (genAI) tools has ushered in a new era in education and presented significant challenges to academic institutions. It has also delivered new concerns for educators who seek to support, and to certify, students’ learning. In addition, the potential and in some cases the necessity for students to learn to engage these new tools in preparation for future work in a professional or research context is emerging apace. This raises important questions for the form and focus of student learning in higher education. It also calls for guidance for educators, especially those who may not be familiar with the operation or implications of these new technologies for their teaching. This paper presents an innovative typology for designing assessment in this context, and that offers language to discuss academic integrity issues and to authorise AI use. The typology draws on and extends scholarship related to groupwork, considering the role of the genAI as a ‘group member’. It provides examples of related approaches to assessment design, and of level descriptors that educators may use as a basis for rubrics to recognise and define the qualities of good student use of genAI tools in this context.

  Read article: From How Much to Whodunnit: A framework for authorising and evaluating student AI use→

  • PDF