Express yourself simply: Mobile AI for bridging communication gaps
Loading...
Files
Date
2024
Authors
Kramar, Anna
Salim, Kinza
Nguyen, Hoang D.
Journal Title
Journal ISSN
Volume Title
Publisher
Association for Computing Machinery (ACM)
Published Version
Abstract
Doodle Me is an AI mobile application designed to bridge communication gaps and barriers. This tool is beneficial for individuals with articulation and phonological speech disorders. By using advanced machine learning algorithms, particularly a Convolutional Neural Network (CNN), the application interprets user-created drawings in real-time. This app provides a universal mode of expression that transcends traditional language barriers. The CNN model is trained on the QuickDraw dataset which achieves an accuracy of 92.6%. The application is accessible and practical, functioning seamlessly across all smartphones. Doodle Me enhances communication by recognizing and categorizing doodles, and provides visual and auditory feedback to users. This tool provides a holistic approach by aiding at rehabilitation of numerous medical conditions. It deals with multilinguistics of different age groups and is also eco-friendly. The application serves as an assistive tool in the healthcare system. It can support a wide range of medical conditions, including post-ischemic stroke aphasia, facial paralysis, stuttering/stammering, Autism Spectrum Disorder (ASD), post-operative laryngectomy or vocal cord paralysis, Alzheimer’s disease, and rehabilitation for post-cochlear implant patients (where children are taught about speech recognition by giving visual cues) and elderly with presbycusis (hearing loss at high frequencies). Doodle Me addresses communication barriers and promotes biosustainability by reducing the need for unnecessary stationary items. It is a practical application aimed at making a positive societal impact by facilitating seamless integration.
Description
Keywords
Convolutional Neural Network , Mobile AI , Doodle recognition , Human-Computer Interaction , Mobile application , Machine learning
Citation
Kramar, A., Salim, K. and Nguyen, H. D. (2024) 'Express yourself simply: Mobile AI for bridging communication gaps', MOBILEHCI Adjunct ’24, 30 September - 3 October, Melbourne, VIC, Australia. https://doi.org/10.1145/3640471.3680456
Link to publisher’s version
Collections
Copyright
© 2024, the Authors. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).