Modifying the ICAP framework to capture cognitive engagement in discussion forums in asynchronous and fully online learning environments

Abstract

Understanding and enhancing student engagement is crucial for online learning as it reflects student satisfaction and academic achievement. Cognitive engagement (CE) refers to students’ investment in learning through strategic, reflective and self-regulatory behaviours that foster problem-solving (Chi and Wylie, 2014). It involves critical thinking, conceptual understanding and cognitive skill development. Online discussion forums serve as a key tool that facilitates CE by enabling critical thinking and collaborative knowledge-building (Ahmad et al., 2022). Analysing students’ CE in discussion forums can enhance learning systems and inform course and assessment design (Prince et al., 2020). Although CE is difficult to detect, tools like the ICAP framework offer a structured approach by categorising engagement into four observable dimensions: Passive, Active, Constructive, and Interactive (Wiggins et al., 2017). ICAP hypothesis suggests that CE increases progressively from passive to interactive (Chi & Wylie, 2014). While widely applied in synchronous settings (Barlow & Brown, 2020), ICAP has seen limited use in asynchronous online environments. While prior studies have applied the ICAP framework as a complementary tool to analyse CE in online learning, they often combine it with other models such as Community of Inquiry (CoI) or Bloom’s taxonomy and focus primarily on synchronous or manually coded environments, highlighting the need to solely apply the ICAP and automate prediction of cognitive behaviours (Farrow et al., 2022, Gorgun et al., 2022; Ahmad et al.,2022).

Building on ICAP hypothesis, this research modifies ICAP framework, tailored to capture CE in discussion forums in asynchronous and fully online learning environments. The study employed a mixed-method approach, combining manual content analysis and descriptive analysis using visualisation to examine student interactions in discussion forums within STEM courses in an Australian University. Courses from IT programs across four consecutive study periods (2023-2024), each with over 80 students were selected to manually annotate and map against the modified ICAP framework (with passive excluded, and dimensions categorised into three levels low, medium, high) using keywords and phrases of the discussion forums. Inter-rater reliability was 100%, as project team members collaboratively resolved conflicts. The mapped data was then evaluated descriptively through visual and text analysis to uncover deeper patterns of CE. Students demonstrated higher active and constructive engagement, but gains from constructive to interactive were limited. To better support CE, forums should be designed to encourage more student interactions. To better support cognitive engagement, online discussion forums should be designed to encourage more interactions among students. These preliminary findings pave the way for empirical validation of the modified ICAP framework in future controlled studies, though generalisation is yet to be established due to domain specificity.

The research highlights the importance of educators understanding students CE in fully online and asynchronous learning and adopting the modified ICAP framework. This can be further utilised to better design the online courses to encourage deeper student participation, help students move from passive-active-constructive-interactive engagement. As universities continue to expand online education, understanding how to structure discussion activities to improve student CE is essential for better learning outcomes and shaping online teaching practices.