Towards Rhapsode^TM Adaptive Learning System ­Mediated-Quality Improvement of Nursing Education. Evaluation Applying Data Science Approach.

This dissertation evaluates the implementation of the Adaptive Learning Platform (ALP) Rhapsode^TM (hereinafter referred to as Rhapsode) within the Danish nursing education at University College Absalon (UCA) in the NurseEd project while developing a methodology to do so. Current literature lacks quality evidence in ALP implementation research, leading to this dissertation bridging this gap while developing a methodology for evaluating the impact of ALP implementation. The development and application of this evaluation methodology are designed to assess the multifaceted impact of Rhapsode across three identified quality dimensions within the educational context of UCA. The methodology devised for this purpose revolves around two central research questions (RQ), steering the direction of the investigation. RQ1: ”How to develop a methodology for evaluating the quality and impact of an adaptive learning platform in the Higher Education context?” seeks to establish a robust framework for evaluating the quality and impact of ALPs in the higher education landscape. This is achieved through a sequential progression of phases, namely ’Analysis and Exploration’, ’Design and Testing’, and ’Evaluation and Reflection’. The initial phase, ’Analysis and Exploration’, draw on insights from the literature through a literature review and the NurseEd project’s outline to obtain the entities to be evaluated within the ALP system. These entities are students, teachers, and designers, aligning with the three quality dimensions from the project outline: (1) students’ learning progression with Rhapsode, (2) teachers’ pedagogical integration and interactions with Rhapsode, and (3) Designers’ perspective for assessing Rhapsode as an ALP. Subsequently, the ’Design and Testing’ phase unfolds across seven research papers, each addressing specific evaluation categories identified in the previous phase. These papers adopt a social constructivist research paradigm, supplemented by a mixed­ methods approach, to provide a nuanced understanding of Rhapsode’s impact on students, teachers, and a designer perspective on the ALP itself. This phase aims to refine the evaluation methodology through empirical research conducted in real­world educational settings by integrating insights from empirical data and existing literature. The final phase, ’Evaluation and Reflection’, introduces an additional evaluated element of the broader implementation context surrounding the ALP. By examining the issues of students’ participation in research studies, students’ study habits, students’ experiences with technical issues, and teachers’ challenges with Learning Analytics Dashboards (LADs), this phase offers valuable insights into the implementation issues of the ALP that broaden out into both organization, management, and the researcher which set the stage for future studies. Moving on to RQ2: ”What is the impact of implementing Rhapsode in the Danish Nursing education at University College Absalon on the quality dimensions?” this examination encompasses a nuanced analysis of the three quality dimensions.
For Students’ Learning Progression:
• In conducting a detailed analysis of test scores, crucial insights into students’ performance on their knowledge dimension as facilitated by Rhapsode did not show students significantly improving from using it in their pre­lecture preparation and the teachers adjusted lectures.
Examining Teachers’ Pedagogical Integration and Interactions:
• Uncovers challenges related to usability and the integration of actionable LAD prototypes into teaching practices.
• Rhapsode offers helpful guidance for students’ pre­lecture preparation.
• The actionable Learning Analytics Dashboard (LAD) prototypes receive criticisms from teachers on potential biases in analysis and application and stir worry about reduced teacher autonomy.
• These findings highlight the challenges associated with integrating actionable LADs into teaching practices, particularly when the LADs are difficult for teachers to comprehend. Similarly, designers’ perspective for assessing Rhapsode as an ALP:
• Issues of usability (marginal acceptability among students) are underscored.
• User experience issues (students’ lack of perceived importance of these Learning
Analytic Dashboards) are identified.
• The importance of design principles (consistency, affordance, familiarity, navigation, and visibility) is uncovered in reshaping the platform to accommodate the needs of UCA’s nursing students and teachers.
The presence of identified limitations in the studies presents challenges in conclusively determining the net effect of Rhapsode on students, given factors such as low student engagement, the inherent constraints of natural experimental design, and dependence on test scores. Nevertheless, this study contributes valuable insights into the complexities encountered during the implementation of ALPs. Future studies should aim to deliver high­quality evidence and to elaborate on and uncover the complexities of implementation by examining the roles of management, organization, and researchers in implementing educational technology.