Choice Diversity in Educational Recommender Systems: User Perspectives
DOI:
https://doi.org/10.70770/5mkm9y92Keywords:
educational technology, recommender systems, recommendation diversity, decision making, cognitive closureAbstract
Educational recommender systems (ERSs) traditionally prioritize prediction accuracy, often overlooking the impact of recommendation diversity on user satisfaction. This study aims to understand how recommendation diversity and user psychological traits, such as need for cognitive closure, affect user satisfaction and preferences in ERSs. In two experiments involving university students, we analyzed subjective perceptions of recommendation qualities—including accuracy, novelty, and usefulness—and evaluated the effects of psychological priming on user evaluations. The results reveal that user satisfaction depends not only on perceived accuracy but also on the interplay of diversity, perceived usefulness, and novelty. Furthermore, priming users to consider accuracy or diversity prior to using the system appeared to mitigate the influence of psychological traits, resulting in more consistent evaluations. These findings highlight the potential of task-based strategies, psychological trait personalization, and priming for designing ERSs that foster more effective and satisfying learning experiences.
Downloads
References
Alhijawi, B., Awajan, A., & Fraihat, S. (2022). Survey on the Objectives of Recommender Systems: Measures, Solutions, Evaluation Methodology, and New Perspectives. ACM Computing Surveys, 55(5). https://doi.org/10.1145/3527449
Amer-Yahia, S., Lakshmanan, L. V. S., Vassilvitskii, S., & Yu, C. (2009). Battling Predictability and Overconcentration in Recommender Systems. IEEE Data Engineering Bulletin, 32(4), 33–40.
Berenbaum, H., Bredemeier, K., & Thompson, R. J. (2008). Intolerance of uncertainty: Exploring its dimensionality and associations with need for cognitive closure, psychopathology, and personality. Journal of anxiety disorders, 22(1), 117–125. https://doi.org/10.1016/j.janxdis.2007.01.004
Castells, P., Hurley, N. J., & Vargas, S. (2015). Novelty and Diversity in Recommender Systems. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (2nd ed., pp. 881-918). Springer. https://doi.org/10.1007/978-1-4899-7637-6_26
Chen, L., de Gemmis, M., Felfernig, A., Lops, P., Ricci, F., & Semeraro, G. (2013a). Human Decision Making and Recommender Systems. ACM Transactions on Interactive Intelligent Systems, 3(3). https://doi.org/10.1145/2533670.2533675
Chen, L., Wu, W., & He, L. (2013b). How personality influences users' needs for recommendation diversity? CHI '13 Extended Abstracts on Human Factors in Computing Systems, 829–834. https://doi.org/10.1145/2468356.2468505
Chen, L., Yang, Y., Wang, N., Yang, K., & Yuan, Q. (2019). How Serendipity Improves User Satisfaction with Recommendations? A Large-Scale User Evaluation. The World Wide Web Conference, 240–250. https://doi.org/10.1145/3308558.3313469
Choi, J. A., Koo, M., Choi, I., & Auh, S. (2008). Need for cognitive closure and information search strategy. Psychology & Marketing, 25(11), 1027–1042. https://doi.org/10.1002/mar.20253
Erdt nee Anjorin, M., Fernandez, A., & Rensing, C. (2015). Evaluating Recommender Systems for Technology Enhanced Learning: A Quantitative Survey. IEEE Transactions on Learning Technologies, 8(4), 326–344. https://doi.org/10.1109/TLT.2015.2438867
da Silva, F. L., Slodkowski, B. K., da Silva, K. K., & Cazella, S. C. (2022). A systematic literature review on educational recommender systems for teaching and learning: research trends, limitations and opportunities. Education and Information Technologies, 1–40. https://doi.org/10.1007/s10639-022-11341-9
Deschênes, M. (2020). Recommender systems to support learners’ Agency in a Learning Context: a systematic review. International Journal of Educational Technology in Higher Education, 17, 1–23. http://doi.org/10.1186/s41239-020-00219-w
Drachsler, H., Sanatos, O., & Manouselis, N. (2015). Panorama of Recommender Systems to Support Learning. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (2nd ed., pp. 421–451). Springer. https://doi.org/10.1007/978-1-4899-7637-6_12
Fazeli, S., Drachsler, H., Bitter-Rijpkema, M., Brouns, F., Vegt, W. v. d., & Sloep, P. B. (2018). User-Centric Evaluation of Recommender Systems in Social Learning Platforms: Accuracy is Just the Tip of the Iceberg. IEEE Transactions on Learning Technologies, 11(3), 294–306. https://doi.org/10.1109/TLT.2017.2732349
Gibbons, J. D. (1993). Nonparametric statistics. SAGE Publications, Inc.
https://doi.org/10.4135/9781412985314
Gronseth, S. L., Michela, E., & Ugwu, L. O. (2020). Designing for diverse learners. In J. K. McDonald & R. E. West, Design for learning: Principles, processes, and praxis. EdTech Books. Retrieved October 20, 2024, from https://edtechbooks.org/id/designing_for_diverse_learners
Gunawardana, A., & Shani, G. (2015). Evaluating Recommender Systems. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (2nd ed., pp. 265–308). Springer.
https://doi.org/10.1007/978-1-4899-7637-6_8
Haeubl, G., & Murray, K. (2003). Preference construction and persistence in digital marketplace: The role of electronic recommendation agents. Journal of Consumer Psychology, 13, 75–91.
https://doi.org/10.1207/S15327663JCP13-1&2_07
Hitsuwari, J., & Nomura, M. (2021). Developing and Validating a Japanese Version of the Multidimensional Attitude toward Ambiguity Scale (MAAS). Psychology, 12(4). https://doi.org/10.4236/psych.2021.124030
Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G., & Chen, L. (2015). Human Decision Making and Recommender Systems. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 611–648). Springer. https://doi.org/10.1007/978-1-4899-7637-6_18
Kaminskas, M., & Bridge, D. (2016). Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems. ACM Transactions on Interactive Intelligent Systems, 7(1). https://doi.org/10.1145/2926720
Kapoor, K., Kumar, V., Terveen, L., Konstan, J. A., & Schrater, P. (2015). "I like to explore sometimes": Adapting to Dynamic User Novelty Preferences. Proceedings of the 9th ACM Conference on Recommender Systems, 19–26. https://doi.org/10.1145/2792838.2800172
Kardes, F. R., Cronley, M. L., Kellaris, J. J., & Posavac, S. S. (2004) The Role of Selective Information Processing in Price-Quality Inference. Journal of Consumer Research, 31(2), 368–374. https://doi.org/10.1086/422115
Kim, E. J., Tanford, S., & Book, L. A. (2021). The Effect of Priming and Customer Reviews on Sustainable Travel Behaviors. Journal of Travel Research, 60(1), 86–101. https://doi.org/10.1177/0047287519894069
Kotkov, D., Konstan, J. A., Zhao, Q., & Veijalainen, J. (2018). Investigating serendipity in recommender systems based on real user feedback. Proceedings of the 33rd Annual ACM Symposium on Applied Computing, 1341–1350. https://doi.org/10.1145/3167132.3167276
Kruglanski, A. W., & Webster, D. M. (1996). Motivated closing of the mind: "Seizing" and "freezing". Psychological Review, 103(2), 263–283. https://doi.org/10.1037/0033-295X.103.2.263
Kruglanski, A. W., Webster, D. M., & Klem, A. (1993). Motivated resistance and openness to persuasion in the presence or absence of prior information. Journal of Personality and Social Psychology, 65(5), 861–876. https://doi.org/10.1037/0022-3514.65.5.861
Litman, J. A. (2010). Relationships between measures of I- and D-type curiosity, ambiguity tolerance, and need for closure: An initial test of the wanting-liking model of information-seeking. Personality and Individual Differences, 48(4), 397–402. https://doi.org/10.1016/j.paid.2009.11.005
Lu, W., Ioannidis, S., Bhagat, S., & Lakshmanan, L. V. S. (2014). Optimal recommendations under attraction, aversion, and social influence. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, 811–820. https://doi.org/10.1145/2623330.2623744
Lutz, C., Hoffmann, C., & Meckel, M. (2017). Online Serendipity: A Contextual Differentiation of Antecedents and Outcomes. Journal of the Association for Information Science and Technology, 68(7). 1698–1710. https://doi.org/10.1002/asi.23771
Maccatrozzo, V., Terstall, M., Aroyo, L., & Schreiber, G. (2017). SIRUP: Serendipity In Recommendations via User Perceptions. Proceedings of the 22nd International Conference on Intelligent User Interfaces, 35–44. https://doi.org/10.1145/3025171.3025185
Manouselis, N., Drachsler, H., Verbert, K., & Duval, E. (2012). Recommender Systems for Learning. Springer.
Marante, Y., Silva, V., Gomes Jr., J., Vitor, M., Martins, A., & De Souza, J. (2020). Evaluating Educational Recommendation Systems: a systematic mapping. Anais do XXXI Simpósio Brasileiro de Informática na Educação, 912–921. Porto Alegre: SBC. https://doi.org/10.5753/cbie.sbie.2020.912
Matt, C., Benlian, A., Hess, T., & Weiß, C. (2014). Escaping from the Filter Bubble? The Effects of Novelty and Serendipity on Users’ Evaluations of Online Recommendations, ICIS 2014 Proceedings, 67.
Mayer, R. E. (2022). Cognitive theory of multimedia learning. In R. E. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (3rd ed., pp. 57–72). Cambridge University Press. https://doi.org/10.1017/9781108894333.008
McGraw, K. O., & Wong, S. P. A common language effect size statistic. Psychological Bulletin, 111(2), 361–365. https://doi.org/10.1037/0033-2909.111.2.361
McNee, S. M., Riedl, J., & Konstan, J. A. (2006). Being accurate is not enough: how accuracy metrics have hurt recommender systems. CHI '06 Extended Abstracts on Human Factors in Computing Systems, 1097-1101. https://doi.org/10.1145/1125451.1125659
Mehrotra, R., Shah, C., & Carterette, B. (2020). Investigating Listeners’ Responses to Divergent Recommendations. Proceedings of the 14th ACM Conference on Recommender Systems, 692–696. https://doi.org/10.1145/3383313.3418482
Neuberg, S. L., Judice, T. N., & West, S. G. (1997). What the Need for Closure Scale measures and what it does not: Toward differentiating among related epistemic motives. Journal of Personality and Social Psychology, 72(6), 1396–1412. https://doi.org/10.1037/0022-3514.72.6.1396
Pariser, E. (2011). The filter bubble: what the Internet is hiding from you. Penguin Press.
Pu, P., Chen, L., & Hu, R. (2011). A user-centric evaluation framework for recommender systems. Proceedings of the 5th ACM Conference on Recommender Systems, 157–164. https://doi.org/10.1145/2043932.2043962
Pu, P., Chen, L., & Hu, R. (2012). Evaluating recommender systems from the user’s perspective: survey of the state of the art. User Modeling and User-Adapted Interaction, 22, 317–355. https://doi.org/10.1007/s11257-011-9115-7
Rajaraman, A., & Ullman, J. (2011). Data Mining. In Mining of Massive Datasets (pp. 1-17). Cambridge University Press. https://doi.org/10.1017/CBO9781139058452.002
Roets, A., & Van Hiel, A. (2007). Separating ability from need: Clarifying the dimensional structure of the need for closure scale. Personality and Social Psychology Bulletin, 33(2), 266-280. https://doi.org/10.1177/0146167206294744
Said, A., Fields, B., Jain, B. J., & Albayrak, S. (2013). User-centric Evaluation of a K-furthest Neighbor Collaborative Filtering Recommender Algorithm. Proceedings of the 2013 Conference on Computer Supported Cooperative Work, 1399–1408. https://doi.org/10.1145/2441776.2441933
Suzuki, K., & Sakurai, S. (2001). Ninchiteki Kanketsu Yokkyuu: Jyouhoushori Ryou Oyobi Taijinkankei kara no Kentou [The cognitive need for closure: A study in terms of information-processing quantity and social relationships]. Tsukuba Psychological Research, 23, 153-160.
Suzuki, K., & Sakurai, S. (2003). Ninchiteki Kanketsu Yokkyuu Shakudo no Sakusei to Shinraisei Datousei no Kentou [The construction, reliability and validity of a Japanese Need for Closure Scale]. The Japanese Journal of Psychology, 74(3), 270-275. https://doi.org/10.4992/jjpsy.74.270
Urdaneta-Ponte, M. C., Mendez-Zorrilla, A., & Oleagordia-Ruiz, I. (2021). Recommendation Systems for Education: Systematic Review. Electronics, 10(14). https://doi.org/10.3390/electronics10141611
Webster, D. M., & Kruglanski, A. W. (1994). Individual differences in need for cognitive closure. Journal of Personality and Social Psychology, 67(6), 1049–1062. https://doi.org/10.1037//0022-3514.67.6.1049
Wieland, M., Nordheim, G.V., & Kleinen-von Königslöw, K. (2021). One Recommender Fits All? An Exploration of User Satisfaction With Text-Based News Recommender Systems. Media and Communication, 9(4). https://doi.org/10.17645/mac.v9i4.4241
Wu, W., Chen, L., & He, L.. (2013). Using personality to adjust diversity in recommender systems. Proceedings of the 24th ACM Conference on Hypertext and Social Media, 225–229. https://doi.org/10.1145/2481492.2481521
Zhao, P. & Lee, D. L. (2016). How Much Novelty is Relevant? It Depends on Your Curiosity. Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, 315–324. https://doi.org/10.1145/2911451.2911488
Zhou, T., Kuscsik, Z., Liu, J., Medo, M., Wakeling, J.R., & Zhang, Y. (2010). Solving the apparent diversity-accuracy dilemma of recommender systems. Proceedings of the National Academy of Sciences, 107(10), 4511–4515. https://doi.org/10.1073/pnas.1000488107
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Robert Songer, Tomohito Yamamoto (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
