Creation of a framework for educators and learners to enable best practice in assessment development for a digital learning environment

Show simple item record

dc.contributor.advisor Neville, Karen Mary en
dc.contributor.advisor Pope, Andrew en
dc.contributor.author McCarthy, Nora
dc.date.accessioned 2021-05-27T09:32:52Z
dc.date.available 2021-05-27T09:32:52Z
dc.date.issued 2020
dc.date.submitted 2020
dc.identifier.citation McCarthy, N. 2020. Creation of a framework for educators and learners to enable best practice in assessment development for a digital learning environment. PhD Thesis, University College Cork. en
dc.identifier.endpage 403 en
dc.identifier.uri http://hdl.handle.net/10468/11395
dc.description.abstract Technology is fundamental to advancing education. This has never been more relevant than in current pandemic times where technology enhanced learning (TEL) has taken centre stage. Computer-based assessment (CBA) brings stimulating possibilities; however, any innovations, while exciting, need to be underpinned by sound educational practices and relevant theory. Only then can CBA become integrated into the curriculum as assessment of, as and for learning. Furthermore, advances require complementary enhancement of learning management systems (LMS), with suitable data storage and retrieval capabilities, to allow implementation of novel assessment methods. Research reveals a lack of regular use of theoretically grounded assessment models. A central tenet for learning and assessment is feedback, yet students often lament the absence and quality of feedback they receive. A scoping review, exploring research linking assessment, models and education with feedback and proficiency in computer-based assessment revealed a paucity of research in this complex area. Together with real-world experience of the researcher as a medical practitioner and educator, and discussions with domain-specific experts, the research objective and research questions (RQ) emerged. The objective of the thesis was to create a framework for educators and learners to enable best practice in assessment development for a digital learning environment. The essential elements of an assessment model were addressed in question 1. Pellegrino’s assessment model predominates in the literature, composed of cognition, observation and interpretation forming an assessment triangle. This research proposes an assessment model pyramid that incorporates feedback as an integral component to emphasise its essential role in assessment, converting the existing assessment model triangle into a square base, with proficiency as the pyramidal apex. The parameters used for the observation and interpretation vertices were also driven by literature review findings. Where possible, observation consisted of increasingly complex scenarios, while interpretation was composed of a weighted marking scheme of steps, errors and critical error, with an expert-derived benchmark for progression to more complex scenarios completing the interpretation element. A ‘5E assessment cycle’ was proposed, grounded in theory and educational practice, as an assessment cognition model for the cognition vertex of the assessment model pyramid. Four instantiations of the assessment model pyramid, addressing different real-world instances of deficits in assessment, were developed for the digital learning environment (DLE) using action design research (ADR), leveraging behavioural research methods with artefact development in an organisational setting. A new principle of transferability is proposed in the ADR ‘formalisation of learning’ stage, demonstrated by the influence of and interactions between the instantiations developed. The challenge of delivering an assessment model in a digital learning environment (RQ2) is addressed in three layers, two layers being common to all assessment environments, the last being specific to the DLE. Consideration should be given to the authoring tool, the use of cognitive theory of multi-media learning to promote good assessment design, and LMS capabilities in management of complex marking systems. A set of design principles and guidelines for assessment development in a digital learning environment was developed. The effectiveness of the assessment model pyramid in supporting learning (RQ3) was demonstrated in the DLE through qualitative, quantitative and mixed methods using domain-specific experts and learners. Validity of the developed assessments was shown using Kane’s argument approach to validity. Recognition of the value and success of the assessment model pyramid in the DLE resulted in deployment by South-Southwest Hospital Group, Health Service Executive Ireland of a training and assessment programme, developed using the proposed assessment model pyramid, for donning and doffing personal protective equipment for COVID-19. Future exciting research possibilities are proposed, highlighting the importance of inter-professional and inter-organisational collaboration to ensure that best practice underpins newly developed technology-enabled assessments.   en
dc.format.mimetype application/pdf en
dc.language.iso en en
dc.publisher University College Cork en
dc.rights © 2020, Nora McCarthy. en
dc.rights.uri https://creativecommons.org/licenses/by-nc-nd/4.0/ en
dc.subject Assessment en
dc.subject Feedback en
dc.subject Proficiency en
dc.subject Digital learning environment en
dc.subject Action design research en
dc.title Creation of a framework for educators and learners to enable best practice in assessment development for a digital learning environment en
dc.type Doctoral thesis en
dc.type.qualificationlevel Doctoral en
dc.type.qualificationname PhD - Doctor of Philosophy en
dc.internal.availability Full text not available en
dc.description.version Accepted Version en
dc.description.status Not peer reviewed en
dc.internal.school Business Information Systems en
dc.internal.conferring Summer 2021 en
dc.availability.bitstream embargoed
dc.check.date 2031-05-30


Files in this item

This item appears in the following Collection(s)

Show simple item record

© 2020, Nora McCarthy. Except where otherwise noted, this item's license is described as © 2020, Nora McCarthy.
This website uses cookies. By using this website, you consent to the use of cookies in accordance with the UCC Privacy and Cookies Statement. For more information about cookies and how you can disable them, visit our Privacy and Cookies statement