Efficient adaptation of Large Language Models for digital media and government applications
Loading...
Files
Full Text E-thesis
Date
2024
Authors
Trust, Paul
Journal Title
Journal ISSN
Volume Title
Publisher
University College Cork
Published Version
Abstract
The digital transformation has greatly increased the amount of data, particularly text generated across various fields, including the public sector and digital media. Information such as political campaigns, media reports, citizen feedback, and press releases is now commonly shared on digital platforms. Politicians, government agencies, businesses, and citizens use these platforms to express their views, strategies, goals, and policies.
Analyzing this data can provide valuable insights into public opinion, ongoing policy discussions, business strategies, and socio-political dynamics.
However, the sheer volume of data makes traditional manual analysis or early computational methods impractical, highlighting the need for more efficient automated approaches from Natural Language Processing (NLP), particularly Large Language Models (LLMs), to manage, analyze, and summarize this information. In the course of this work, document-based learning has seen a landmark advance with the progress of generative Artificial Intelligence and the availability of engines and models that are revolutionizing NLP.
Despite significant adoption and investment in the private sector, academia, and high-resource fields, LLMs are less utilized in low-resource fields and the public sector due to constraints such as lack of labeled data, and insufficient budget allocation for machine learning infrastructure and training.
In this Thesis, which was developed during a transition time of fast development of LLMs, we study ways of adapting the most up to date models to novel scenarios in order to both achieve efficiency and understanding of how to adapt LLMs to applications.
To achieve this, this Thesis adapts LLMs to applications in the public sector and digital media. The main approaches developed include: applying weak supervision by leveraging synthetic labels generated by other LLMs to fine-tune models for classifying news articles related to Economic Policy Uncertainty; proposing LLM-based methods for classifying citizen feedback into different categories, as well as for summarization and question answering of citizen feedback; adapting LLMs for automating the handling and navigation of public documents by incorporating strategies such as Retrieval Augmented Generation (RAG), LLM agents; and developing techniques for hallucination detection in these domains.
Description
Keywords
Large Language Models , Generative AI , Citzen feedback , RAG
Citation
Trust, P. 2024. Efficient adaptation of Large Language Models for digital media and government applications. PhD Thesis, University College Cork.