Can VR help understand auditory-visual associations and synaesthesia through immersive battery tests?

Thumbnail Image
Files
Date
2023-04-05
Authors
O'Toole, Patrick
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published Version
10.1109/ACIIW57231.2022.10086003
Research Projects
Organizational Units
Journal Issue
Abstract
This paper gives an overview of my PhD research in the area of auditory-visual associations in Human-Computer Interaction (HCI). My project explores the condition called synaesthesia, and auditory-visual cross-modal associations, in order to gain a better understanding of how our auditory and visual senses combine to inform our perceptive and decision-makings skills. I seek to understand how technology can be used to help strengthen our auditory-visual associations. Previous work suggests that creating effective and immersive environments using Virtual Reality (VR) can help in gaining new insights into auditory-visual perception. Firstly, I will create a VR version of standard auditory-visual and synaesthesia tests, and conduct user studies using both traditional and the VR based tests, with a comparison of the results being investigated to see what impact using a more immersive environment, like VR, has on auditory-visual associations. Secondly, armed with new knowledge concerning auditory-visual associations using VR, I will create a framework that uses auditory-visual associations to help designers create interfaces that are personalised, adaptive and accessible to everyone.
Description
Keywords
Auditory-visual associations , Synaesthesia , Virtual reality , Music
Citation
O’Toole, P. (2022) 'Can VR help understand auditory-visual associations and synaesthesia through immersive battery tests?', 10th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). Nara, Japan: IEEE, pp. 1–5. https://doi.org/10.1109/ACIIW57231.2022.10086003
Link to publisher’s version
https://ieeexplore.ieee.org/xpl/conhome/10085961/proceeding
URI
https://hdl.handle.net/10468/18265
Collections
Computer Science - Conference Items
ADVANCE CRT - Conference Items
Copyright
© 2022, IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.