Access to this article is restricted until after the conference has taken place. Restriction lift date: 2024-09-14
Autofusion: Fusing multi-modalities with interactions
Loading...
Files
Date
2024
Authors
Nguyen, Thuy-Trinh
Deligianni, Fani
Nguyen, Hoang D.
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier
Published Version
Abstract
In the context of escalating data flows through diverse channels, multimodal machine learning holds the potential to simultaneously process varied data formats from multiple sources, offering robust solutions for uncertainty in various applications. The study highlights the often under-explored correlation information among data modalities and emphasizes the importance of disentangling enriched interactions for more informed decision-making. This paper navigates the burgeoning field of multimodal artificial intelligence (AI) by proposing Autofusion, a pioneering framework addressing representation learning and fusion challenges. Our proposed approach integrates autoencoder structures to address overfitting issues in unimodal machine learning, simultaneously tackling information balancing challenges. The framework’s application in Alzheimer’s disease detection, using DementiaBank’s Pitt corpus, demonstrates promising results, outperforming unimodal methods and showcasing a substantial advantage over traditional fusion techniques. This research significantly contributes by introducing Autofusion as a comprehensive multimodal machine learning solution, demonstrating its efficacy through DementiaBank’s Pitt corpus to detect Alzheimer’s disease, and shedding light on the influential role of cross-modality interaction for enhanced performance in complex applications.
Description
Keywords
Multimodal interactions , Multimodal fusion , Multimodal machine learning , Autofusion
Citation
Nguyen, T.-T., Deligianni, F. and Nguyen, H. D. (2024) ‘Autofusion: Fusing multi-modalities with interactions’, 28th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (KES 2024), Seville, Spain, 11-13 September. Procedia Computer Science.