Creation of a framework for educators and learners to enable best practice in assessment development for a digital learning environment

Simple item page
dc.availability.bitstreamembargoed
dc.check.date2031-05-30
dc.contributor.advisorNeville, Karen Maryen
dc.contributor.advisorPope, Andrewen
dc.contributor.authorMcCarthy, Nora
dc.date.accessioned2021-05-27T09:32:52Z
dc.date.available2021-05-27T09:32:52Z
dc.date.issued2020
dc.date.submitted2020
dc.description.abstractTechnology is fundamental to advancing education. This has never been more relevant than in current pandemic times where technology enhanced learning (TEL) has taken centre stage. Computer-based assessment (CBA) brings stimulating possibilities; however, any innovations, while exciting, need to be underpinned by sound educational practices and relevant theory. Only then can CBA become integrated into the curriculum as assessment of, as and for learning. Furthermore, advances require complementary enhancement of learning management systems (LMS), with suitable data storage and retrieval capabilities, to allow implementation of novel assessment methods. Research reveals a lack of regular use of theoretically grounded assessment models. A central tenet for learning and assessment is feedback, yet students often lament the absence and quality of feedback they receive. A scoping review, exploring research linking assessment, models and education with feedback and proficiency in computer-based assessment revealed a paucity of research in this complex area. Together with real-world experience of the researcher as a medical practitioner and educator, and discussions with domain-specific experts, the research objective and research questions (RQ) emerged. The objective of the thesis was to create a framework for educators and learners to enable best practice in assessment development for a digital learning environment. The essential elements of an assessment model were addressed in question 1. Pellegrino’s assessment model predominates in the literature, composed of cognition, observation and interpretation forming an assessment triangle. This research proposes an assessment model pyramid that incorporates feedback as an integral component to emphasise its essential role in assessment, converting the existing assessment model triangle into a square base, with proficiency as the pyramidal apex. The parameters used for the observation and interpretation vertices were also driven by literature review findings. Where possible, observation consisted of increasingly complex scenarios, while interpretation was composed of a weighted marking scheme of steps, errors and critical error, with an expert-derived benchmark for progression to more complex scenarios completing the interpretation element. A ‘5E assessment cycle’ was proposed, grounded in theory and educational practice, as an assessment cognition model for the cognition vertex of the assessment model pyramid. Four instantiations of the assessment model pyramid, addressing different real-world instances of deficits in assessment, were developed for the digital learning environment (DLE) using action design research (ADR), leveraging behavioural research methods with artefact development in an organisational setting. A new principle of transferability is proposed in the ADR ‘formalisation of learning’ stage, demonstrated by the influence of and interactions between the instantiations developed. The challenge of delivering an assessment model in a digital learning environment (RQ2) is addressed in three layers, two layers being common to all assessment environments, the last being specific to the DLE. Consideration should be given to the authoring tool, the use of cognitive theory of multi-media learning to promote good assessment design, and LMS capabilities in management of complex marking systems. A set of design principles and guidelines for assessment development in a digital learning environment was developed. The effectiveness of the assessment model pyramid in supporting learning (RQ3) was demonstrated in the DLE through qualitative, quantitative and mixed methods using domain-specific experts and learners. Validity of the developed assessments was shown using Kane’s argument approach to validity. Recognition of the value and success of the assessment model pyramid in the DLE resulted in deployment by South-Southwest Hospital Group, Health Service Executive Ireland of a training and assessment programme, developed using the proposed assessment model pyramid, for donning and doffing personal protective equipment for COVID-19. Future exciting research possibilities are proposed, highlighting the importance of inter-professional and inter-organisational collaboration to ensure that best practice underpins newly developed technology-enabled assessments.  en
dc.description.statusNot peer revieweden
dc.description.versionAccepted Versionen
dc.format.mimetypeapplication/pdfen
dc.identifier.citationMcCarthy, N. 2020. Creation of a framework for educators and learners to enable best practice in assessment development for a digital learning environment. PhD Thesis, University College Cork.en
dc.identifier.endpage403en
dc.identifier.urihttps://hdl.handle.net/10468/11395
dc.language.isoenen
dc.publisherUniversity College Corken
dc.rights© 2020, Nora McCarthy.en
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/en
dc.subjectAssessmenten
dc.subjectFeedbacken
dc.subjectProficiencyen
dc.subjectDigital learning environmenten
dc.subjectAction design researchen
dc.titleCreation of a framework for educators and learners to enable best practice in assessment development for a digital learning environmenten
dc.typeDoctoral thesisen
dc.type.qualificationlevelDoctoralen
dc.type.qualificationnamePhD - Doctor of Philosophyen
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
McCarthyNMT_PhD2020.docx
Size:
6.4 MB
Format:
Microsoft Word XML
Description:
Full Text E-thesis (Word)
Thumbnail Image
Name:
McCarthyNMT_PhD2020.pdf
Size:
5.46 MB
Format:
Adobe Portable Document Format
Description:
Full Text E-thesis
Thumbnail Image
Name:
3. SubmissionforExamination Nora McCarthyAllSignatures - Nora McCarthy.pdf
Size:
399.76 KB
Format:
Adobe Portable Document Format
Description:
Submission for Examination Form
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
5.2 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Research Theses Submission
Business Information Systems - Doctoral Theses
College of Business and Law - Doctoral Theses