Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review

Tobias Alexander Bang Tretow-Fish; Md Saifuddin Khalid

doi:10.1007/978-3-031-06675-7_3

Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review

Tobias Alexander Bang Tretow-Fish^*
, Md Saifuddin Khalid

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Abstract

There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.

Original language	English
Title of host publication	Proceedings of 6^th EAI International Conference : Design, Learning, and Innovation
Editors	Eva Brooks, Jeanette Sjöberg, Anders Kalsgaard Møller
Publisher	Springer
Publication date	2022
Pages	37-52
ISBN (Print)	9783031066740
DOIs	https://doi.org/10.1007/978-3-031-06675-7_3
Publication status	Published - 2022
Event	6^th EAI International Conference on Design, Learning, and Innovation - Virtual, Online Duration: 10 Dec 2021 → 11 Dec 2021

Conference

Conference	6^th EAI International Conference on Design, Learning, and Innovation
Location	Virtual, Online
Period	10/12/2021 → 11/12/2021

Series	Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST
Volume	435 LNICST
ISSN	1867-8211

Keywords

Adaptive learning platforms
Evaluation
Learning analytics

Access to Document

10.1007/978-3-031-06675-7_3

OpenUrl availability

Check availability in DTU Findit

Towards RhapsodeTM Adaptive Learning System-Mediated Quality Improvement of Nursing Education: Evaluation Applying Data Science Approach
Tretow-Fish, T. A. B. (PhD Student), Khalid, M. S. (Main Supervisor), Clemmensen, L. K. H. (Supervisor), Olsen, N. A. L. (Supervisor), Buus, L. (Examiner), Salha, S. H. M. (Examiner) & Bækgaard, P. (Examiner)
01/01/2021 → 15/07/2024
Project: PhD
NURSEED: Designing 21st century nursing education by integrating adaptive learning technologies
Khalid, M. S. (CoPI), Ersbøll, B. K. (CoPI) & Tretow-Fish, T. A. B. (PhD Student)
01/02/2020 → 31/01/2024
Project: Research

Cite this

Tretow-Fish, Tobias Alexander Bang ; Khalid, Md Saifuddin. / Evaluating Learning Analytics of Adaptive Learning Systems : A Work in Progress Systematic Review. Proceedings of 6^th EAI International Conference: Design, Learning, and Innovation. editor / Eva Brooks ; Jeanette Sjöberg ; Anders Kalsgaard Møller. Springer, 2022. pp. 37-52 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, Vol. 435 LNICST).

@inproceedings{df4f0de445a84f13a5cb173ec44c839c,

title = "Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review",

abstract = "There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.",

keywords = "Adaptive learning platforms, Evaluation, Learning analytics",

author = "Tretow-Fish, \{Tobias Alexander Bang\} and Khalid, \{Md Saifuddin\}",

note = "Publisher Copyright: {\textcopyright} 2022, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.; 6<sup>th</sup> EAI International Conference on Design, Learning, and Innovation, DLI 2021 ; Conference date: 10-12-2021 Through 11-12-2021",

year = "2022",

doi = "10.1007/978-3-031-06675-7\_3",

language = "English",

isbn = "9783031066740",

series = "Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST",

publisher = "Springer",

pages = "37--52",

editor = "Eva Brooks and Jeanette Sj{\"o}berg and M{\o}ller, \{Anders Kalsgaard\}",

booktitle = "Proceedings of 6th EAI International Conference",

}

Tretow-Fish, TAB & Khalid, MS 2022, Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review. in E Brooks, J Sjöberg & AK Møller (eds), Proceedings of 6^th EAI International Conference: Design, Learning, and Innovation. Springer, Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, vol. 435 LNICST, pp. 37-52, 6^th EAI International Conference on Design, Learning, and Innovation, 10/12/2021. https://doi.org/10.1007/978-3-031-06675-7_3

Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review. / Tretow-Fish, Tobias Alexander Bang; Khalid, Md Saifuddin.
Proceedings of 6^th EAI International Conference: Design, Learning, and Innovation. ed. / Eva Brooks; Jeanette Sjöberg; Anders Kalsgaard Møller. Springer, 2022. p. 37-52 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, Vol. 435 LNICST).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

TY - GEN

T1 - Evaluating Learning Analytics of Adaptive Learning Systems

T2 - 6<sup>th</sup> EAI International Conference on Design, Learning, and Innovation

AU - Tretow-Fish, Tobias Alexander Bang

AU - Khalid, Md Saifuddin

PY - 2022

Y1 - 2022

N2 - There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.

AB - There is currently no systematic overview of methods for evaluating Learning Analytics (LA) and Learning Analytics Dashboards (LAD) of Adaptive Learning Platforms (ALPs). 10 articles and 2 reviews are analyzed and synthesized. Focusing on the purposes of evaluation, methods used in the studies are grouped into five categories (C1-5): C1) evaluation of LA and LAD design and framework, C2) evaluation of performance with LA and LAD, C3) evaluation of adaptivity functions of the system, C4) evaluation of perceived value, and C5) Evaluation of pedagogical and didactic theory/context. While there is a relative high representation of evaluations in the C1-C4 categories of methods, which contribute to the design and development of the interaction and interface design features, the C5 category is not represented. The presence of pedagogical and didactical theory in the LA, LAD, and ALPs is lacking. Though traces of pedagogical theory is present none of the studies evaluates on its impact.

KW - Adaptive learning platforms

KW - Evaluation

KW - Learning analytics

U2 - 10.1007/978-3-031-06675-7_3

DO - 10.1007/978-3-031-06675-7_3

M3 - Article in proceedings

AN - SCOPUS:85131941380

SN - 9783031066740

T3 - Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

SP - 37

EP - 52

BT - Proceedings of 6th EAI International Conference

A2 - Brooks, Eva

A2 - Sjöberg, Jeanette

A2 - Møller, Anders Kalsgaard

PB - Springer

Y2 - 10 December 2021 through 11 December 2021

ER -

Tretow-Fish TAB, Khalid MS. Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review. In Brooks E, Sjöberg J, Møller AK, editors, Proceedings of 6^th EAI International Conference: Design, Learning, and Innovation. Springer. 2022. p. 37-52. (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, Vol. 435 LNICST). doi: 10.1007/978-3-031-06675-7_3

Evaluating Learning Analytics of Adaptive Learning Systems: A Work in Progress Systematic Review

Abstract

Conference

Keywords

Access to Document

OpenUrl availability

Fingerprint

Projects

Towards RhapsodeTM Adaptive Learning System-Mediated Quality Improvement of Nursing Education: Evaluation Applying Data Science Approach

NURSEED: Designing 21st century nursing education by integrating adaptive learning technologies

Cite this