Empowering Communication for Paralyzed Individuals and Spinal Cord Injury Patients: An Intelligent System With Eye Gaze Tracking, Voice Assistance, and Chat-Bot Integration

Abstract— Advancements in artificial intelligence (AI) and machine learning (ML) have paved the way for transformative solutions in healthcare, particularly for individuals with severe physical disabilities. This research presents an innovative assistive communication platform tailored to Sinhala-speaking patients in Sri Lanka, addressing the unique challenges faced by individuals with Amyotrophic Lateral Sclerosis (ALS) and spinal cord injuries (SCI). These conditions severely restrict mobility and communication, leading to social isolation and diminished quality of life. The proposed system integrates eye-gaze tracking, speech recognition, and natural language processing (NLP) technologies to provide inclusive and effective communication solutions. For ALS patients, real-time eye-gaze tracking powered by MediaPipe interprets gaze movements as inputs, processed by a dense neural network (DNN)-based chatbot that generates responses in both English and Sinhala text and voice outputs. For SCI patients, who often retain speech abilities but face physical limitations, the system leverages speech recognition and NLP to interpret spoken commands, translating them into Sinhala text and synthesized speech. The dual-system approach ensures inclusivity, catering to the larger SCI-affected population while remaining adaptable to the needs of ALS patients. Despite challenges such as precision in gaze tracking, complexities in Sinhala speech recognition, and cultural nuances in NLP, the system demonstrates significant potential to enhance independence, social inclusion, and overall quality of life for patients. Empirical validation highlights the system's effectiveness, with high accuracy in intent identification and response generation. This research underscores the importance of culturally adapted, scalable assistive technologies, offering a foundation for future innovations in underserved regions. By bridging communication barriers, this platform represents a transformative step toward empowering individuals with disabilities.

Assistive Communication, Eye-Gaze Tracking, Speech Recognition, Natural Language Processing, Sinhala Localization

1. H, H, N, Kalyani., S, Dassanayake., Upul, Senarath. "Effects of paraplegia on quality of life and family economy among patients with spinal cord injuries in selected hospitals of Sri Lanka." Spinal Cord, 53 (2015).:446-450. doi: 10.1038/SC.2014.183 [Google Scholar] [Crossref]

2. M.Ezzat, M.Maged, Y. Gamal, M. Adel, M. Alrahmawy and S. El-Metwally, "Blink-ToLive eye-based communication system for users with speech impairments," scientific reports, 17 May 2023.. [Google Scholar] [Crossref]

3. Z. A. Wassan, "IoT Based Smart Home for Paralyzed Patients through Eye Blink," International Journal of Advanced Trends in Computer Science and Engineering, vol. 10, no. 2, 2021. [Google Scholar] [Crossref]

4. S.Heravian, N.Nouri, M.B.Taghadosi and S.M.H.Seyedkash, "Implementation of Eye Tracking in an IoT-Based Smart Home for Spinal Cord Injury Patients," Modern Care Journal •, October 2019. [Google Scholar] [Crossref]

5. O. Zahour, E. H. Benlahmar, A. Eddaoui, H. Ouchra, and O. Hourrane, “A system for educational and vocational guidance in Morocco: Chatbot EOrientation,” Procedia Computer Science, vol. 175, pp. 554–559, 2020. [Google Scholar] [Crossref]

6. F. Colace, M. D. Santo, M. Lombardi, F. Pascale, A. Pietrosanto, and S. Lemma, “Chatbot for E-Learning: A Case of Study,” IJMERR, pp. 528– 533, 2018. [Google Scholar] [Crossref]

7. F. Clarizia, F. Colace, M. Lombardi, F. Pascale, and D. Santaniello, “Chatbot: An Education Support System for Student,” Cyberspace Safety and Security, vol. 11161, A. Castiglione, F. Pop, M. Ficco, and F. Palmieri, Eds. Cham: Springer International Publishing, 2018, pp. 291– 302. [Google Scholar] [Crossref]

8. S. Ayanouz, B. A. Abdelhakim, and M. Benhmed, “A Smart Chatbot Architecture based NLP and Machine Learning for Health Care Assistance,” Proceedings of the 3rd International Conference on Networking, Information Systems & Security, Marrakech Morocco, pp. 1– 6, March 2020. [Google Scholar] [Crossref]

9. I. Poola, “Making Artificial Intelligence (AI) and Disrupted Business Intelligence (BI) truly Conversational with Humanity Touch, Automated Descriptions and Talking Bots,” International Journal of Advance Research, Ideas and Innovations in Technology, 2017. [Google Scholar] [Crossref]

10. B. Sharma, H. Puri, and D. Rawat, “Digital Psychiatry - Curbing Depression using Therapy Chatbot and Depression Analysis,” 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), Coimbatore, pp. 627–631, April 2018. [Google Scholar] [Crossref]

11. A. N. Vaidyam, H. Wisniewski, J. D. Halamka, M. S. Kashavan, and J. B. Torous, “Chatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape,” Can J Psychiatry, March 2019. [Google Scholar] [Crossref]

12. K. J. Oh, D. Lee, B. Ko, and H. J. Choi, “A Chatbot for Psychiatric Counseling in Mental Healthcare Service Based on Emotional Dialogue Analysis and Sentence Generation,” 2017 18th IEEE International Conference on Mobile Data Management (MDM), Daejeon, South Korea, pp. 371–375, May 2017. [Google Scholar] [Crossref]

13. T. Kamita, T. Ito, A. Matsumoto, T. Munakata, and T. Inoue, “A Chatbot System for Mental Healthcare Based on SAT Counseling Method,” Mobile Information Systems, vol. 2019, pp. 1–11, March 2019. [Google Scholar] [Crossref]

14. N. Harilal, R. Shah, S. Sharma, and V. Bhutani, “CARO: An Empathetic Health Conversational Chatbot for People with Major Depression,” Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pp. 349–350, January 2020. [Google Scholar] [Crossref]

• Patriarchy and Gender Roles in Shobhaa De's Second Thoughts and Easterine Kire’s Life on Hold
• Comparative Effects of Zeolite and Fly Ash on Soil Properties and Agricultural Productivity
• Assessing Interrelationships among Banking Satisfaction, Cooperative Perspectives and Rural Service Beliefs in Cooperative Banking in Northern Nigeria
• High-Performance Alloys and Composites’ Applications in Production Engineering
• Integrating Tidal Energy into Sub-Saharan Africa’s Power Mix: A Strategic Framework for Renewable Energy Expansion