Restriction lift date: 2031-05-30
Creation of a framework for educators and learners to enable best practice in assessment development for a digital learning environment
Loading...
Files
Date
2020
Authors
McCarthy, Nora
Journal Title
Journal ISSN
Volume Title
Publisher
University College Cork
Published Version
Abstract
Technology is fundamental to advancing education. This has never been more relevant than in current pandemic times where technology enhanced learning (TEL) has taken centre stage. Computer-based assessment (CBA) brings stimulating possibilities; however, any innovations, while exciting, need to be underpinned by sound educational practices and relevant theory. Only then can CBA become integrated into the curriculum as assessment of, as and for learning. Furthermore, advances require complementary enhancement of learning management systems (LMS), with suitable data storage and retrieval capabilities, to allow implementation of novel assessment methods. Research reveals a lack of regular use of theoretically grounded assessment models. A central tenet for learning and assessment is feedback, yet students often lament the absence and quality of feedback they receive. A scoping review, exploring research linking assessment, models and education with feedback and proficiency in computer-based assessment revealed a paucity of research in this complex area. Together with real-world experience of the researcher as a medical practitioner and educator, and discussions with domain-specific experts, the research objective and research questions (RQ) emerged. The objective of the thesis was to create a framework for educators and learners to enable best practice in assessment development for a digital learning environment.
The essential elements of an assessment model were addressed in question 1. Pellegrino’s assessment model predominates in the literature, composed of cognition, observation and interpretation forming an assessment triangle. This research proposes an assessment model pyramid that incorporates feedback as an integral component to emphasise its essential role in assessment, converting the existing assessment model triangle into a square base, with proficiency as the pyramidal apex. The parameters used for the observation and interpretation vertices were also driven by literature review findings. Where possible, observation consisted of increasingly complex scenarios, while interpretation was composed of a weighted marking scheme of steps, errors and critical error, with an expert-derived benchmark for progression to more complex scenarios completing the interpretation element. A ‘5E assessment cycle’ was proposed, grounded in theory and educational practice, as an assessment cognition model for the cognition vertex of the assessment model pyramid.
Four instantiations of the assessment model pyramid, addressing different real-world instances of deficits in assessment, were developed for the digital learning environment (DLE) using action design research (ADR), leveraging behavioural research methods with artefact development in an organisational setting. A new principle of transferability is proposed in the ADR ‘formalisation of learning’ stage, demonstrated by the influence of and interactions between the instantiations developed. The challenge of delivering an assessment model in a digital learning environment (RQ2) is addressed in three layers, two layers being common to all assessment environments, the last being specific to the DLE. Consideration should be given to the authoring tool, the use of cognitive theory of multi-media learning to promote good assessment design, and LMS capabilities in management of complex marking systems. A set of design principles and guidelines for assessment development in a digital learning environment was developed.
The effectiveness of the assessment model pyramid in supporting learning (RQ3) was demonstrated in the DLE through qualitative, quantitative and mixed methods using domain-specific experts and learners. Validity of the developed assessments was shown using Kane’s argument approach to validity. Recognition of the value and success of the assessment model pyramid in the DLE resulted in deployment by South-Southwest Hospital Group, Health Service Executive Ireland of a training and assessment programme, developed using the proposed assessment model pyramid, for donning and doffing personal protective equipment for COVID-19. Future exciting research possibilities are proposed, highlighting the importance of inter-professional and inter-organisational collaboration to ensure that best practice underpins newly developed technology-enabled assessments.
Description
Keywords
Assessment , Feedback , Proficiency , Digital learning environment , Action design research
Citation
McCarthy, N. 2020. Creation of a framework for educators and learners to enable best practice in assessment development for a digital learning environment. PhD Thesis, University College Cork.