Irish-based Large Language Model with extreme low-resource settings in machine translation

Simple item page
dc.contributor.authorTran, Khanh-Tungen
dc.contributor.authorO'Sullivan, Barryen
dc.contributor.authorNguyen, Hoang D.en
dc.contributor.funderScience Foundation Irelanden
dc.date.accessioned2024-07-09T09:02:29Z
dc.date.available2024-07-09T09:02:29Z
dc.date.issued2024-08-11en
dc.description.abstractLarge Language Models (LLMs) have demonstrated exceptional performances in a wide range of natural language processing tasks. However, their success does not always extend to machine translation, particularly in challenging scenarios such as translating low-resource languages. This study investigates the multilingual capability of LLMs, with a case study on Irish, an extremely low-resource language, focusing on translation tasks between English and Irish. We propose a dynamic, efficient language adaptation framework for English-centric LLMs, which involves layer-specific adjustments and subsequent fine-tuning for machine translation. Our findings highlight several key insights: (1) different layers in the LLM serve distinct functions such as language understanding and task reasoning, (2) effective translation requires extensive pre-training on both source and target languages, and (3) targeted fine-tuning for machine translation leads to significant improvements of 36.7% for English to Irish and 133.4% for Irish to English compared to the previous state-of-the-art.en
dc.description.statusPeer revieweden
dc.description.versionAccepted Versionen
dc.format.mimetypeapplication/pdfen
dc.identifier.citationTran, K.-T., O'Sullivan, B. and Nguyen, H. D. (2024) 'Irish-based Large Language Model with Extreme Low-Resource Settings in Machine Translation', LoResMT 2024: The Seventh Workshop on Technologies for Machine Translation of Low-Resource Languages, @ACL2024, Bangkok, Thailand, August 11–16.en
dc.identifier.endpage10en
dc.identifier.startpage1en
dc.identifier.urihttps://hdl.handle.net/10468/16110
dc.language.isoenen
dc.publisherACLen
dc.relation.projectinfo:eu-repo/grantAgreement/SFI/SFI Research Centres/12/RC/2289/IE/INSIGHT - Irelands Big Data and Analytics Research Centre/en
dc.relation.projectinfo:eu-repo/grantAgreement/SFI/SFI Centres for Research Training Programme::Data and ICT Skills for the Future/18/CRT/6223/IE/SFI Centre for Research Training in Artificial Intelligence/en
dc.relation.urihttps://www.loresmt.orgen
dc.rights© 2023 Association for Computational Linguisticsen
dc.subjectLarge Language Models (LLMs)en
dc.subjectNatural Language Processing (NLP)en
dc.subjectTranslationen
dc.subjectMachine translationen
dc.subjectLanguage technologiesen
dc.subjectAccessibilityen
dc.subjectIrish languageen
dc.titleIrish-based Large Language Model with extreme low-resource settings in machine translationen
dc.typeConference itemen
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
2024_UCCIX_MT_loresmt.pdf
Size:
532.83 KB
Format:
Adobe Portable Document Format
Description:
Accepted version
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
2.71 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Computer Science - Conference Items
Insight Centre for Data Analytics - Conference Items