Vocal Gaze Mouse Controller

Abstract- This paper presents the design and implementation of a multimodal, real-time Human-Computer Interaction (HCI) system called the Vocal Gaze Mouse Controller. The system integrates eye-gaze tracking, blink detection, and voice recognition to control mouse functions, providing a fully hands-free interaction mechanism. Designed especially for individuals with physical disabilities, the system enables cursor navigation and command execution using webcam-based eye tracking and microphone-enabled speech processing. We use Dlib for facial landmark detection, OpenCV for image processing, PyAutoGUI for mouse control, and Python’s Speech Recognition library for interpreting voice inputs. This solution is non-invasive, affordable, and highly accurate, making it suitable for real-world accessibility applications. Results from experimental evaluations show high accuracy in blink and voice recognition, as well as low latency and user satisfaction.

Eye Gaze Tracking, Blink Detection, Voice Commands, Human-Computer Interaction (HCI), Computer Vision, Accessibility, Python, Py Auto GUI, Speech Recognition, Real-Time Processing

1. Priya Nandihal, Vijay Shetty, Tapas, Piyush Pareek “ Giloma Detection Using Improved Artificial Neural Network in MRI Images”, 2022 IEEE 2nd Mysore Sub Section International Conference (MysuruCon), 2022, pp 1-9. [Google Scholar] [Crossref]

2. N. Kumar, P. Nandihal, M. R. B, P. K. Pareek, N. T and S. S. R, "A Novel Machine Learning-Based Artificial Voice Box," 2022 Second International Conference on Advanced Technologies in Intelligent Control, Environment, Computing & Communication Engineering (ICATIECE), Bangalore, India, 2022, pp. 1-7, doi: 10.1109/ICATIECE56365.2022.10046967. [2] [Google Scholar] [Crossref]

3. Kumar, P. R., Meenakshi, S., Shalini, S., Devi, S. R., & Boopathi, S. (2023). Soil Quality Prediction in Context Learning Approaches Using Deep Learning and Blockchain for Smart Agriculture. In R. Kumar, A. Abdul Hamid, & D. Binti Ya’akub (Eds.), Effective AI, Blockchain, and E-Governance Applications for Knowledge Discovery and Management (pp. 1-26). IGI Global Scientific Publishing. https://doi.org/10.4018/978-1-6684-9151-5.ch001 [Google Scholar] [Crossref]

4. P. Nandihal, P. K. Pareek, V. H. C. De Albuquerque, M. R. B, A. Khanna and V. S. Kumar, "Ant Colony Optimization based Medical Image Preservation and Segmentation," 2022 Second International Conference on Advanced Technologies in Intelligent Control, Environment, Computing & Communication Engineering (ICATIECE), Bangalore,India,2022,pp. 1-7,doi:10.1109/ICATIECE56365.2022.10047584. [Google Scholar] [Crossref]

5. S A, K. ., Nandihal , P. ., K, S., D R , M. ., &Liyakathunisa. (2022). PRIOR DETECTION OF ALZHEIMER’S DISEASE WITH THE AID OF MRI IMAGES AND DEEP NEURAL NETWORKS. Malaysian Journal of Computer Science, 16–28. https://doi.org/10.22452/mjcs.sp2022no2.2 [Google Scholar] [Crossref]

6. Padthe, M. Mathapati, P. M S and P. Nandihal, "APOA based Multi-scale Parallel Convolution Blocks with Hybrid Deep Learning for Gastric Cancer Prediction from Endoscopic Images," 2023 International Conference on Ambient Intelligence, Knowledge Informatics and Industrial Electronics (AIKIIE), Ballari, India, 2023, pp. 1-7, doi: 10.1109/AIKIIE60097.2023.10390430. [Google Scholar] [Crossref]

7. N. Scott, C. Green, and S. Fairley, “Investigation of the use of eye tracking to examine tourism advertising effectiveness,” Current Issues in Tourism, vol. 19, no. 7, pp. 634–642, 2016. [Google Scholar] [Crossref]

8. K. Takemura, K. Takahashi, J. Takamatsu, and T. Ogasawara, “Estimating 3-D point-of-regard in a real environment using a head-mounted eye-tracking system,” IEEE Transactions on Human-Machine Systems, vol. 44, no. 4, pp. 531–536, 2014. [Google Scholar] [Crossref]

9. O. Ferhat and F. Vilarino, “Low cost eye tracking: the current panorama,” Computational Intelligence and Neuroscience, vol. 2016, Article ID 8680541, pp. 1–14, 2016. [Google Scholar] [Crossref]

10. M. A. Eid, N. Giakoumidis, and A. El Saddik, “A novel eye- gaze-controlled wheelchair system for navigating unknown environments: case study with a person with ALS,” IEEE Access, vol. 4, pp. 558–573, 2016. [Google Scholar] [Crossref]

11. L. Sun, Z. Liu, and M.-T. Sun, “Real time gaze estimation with a consumer depth camera,” Information Sciences, vol. 320, pp. 346–360, 2015. [Google Scholar] [Crossref]

12. Nagaraj, G., & Channegowda, N. “Video Forgery Detection using an Improved BAT with Stacked Auto Encoder Model”. Journal of Advanced Research in Applied Sciences and Engineering Technology, 42(2), 175–187.(2024). https://doi.org/10.37934/araset.42.2.175187 Scopus indexed, SJR Q2. [Google Scholar] [Crossref]

13. Nandihal, Priya; Pareek, Piyush Kumar; Suhas, G K; Deepak, N R; Bhagappa.” The Clinical and Public Health Impact of The Combined Work of hysicians and AI Systems” NeuroQuantology; Bornova Izmir Vol. 20, Iss. 16, (2022):2154-2163. DOI:10.48047/NQ.2022.20.16. NQ88214 [Google Scholar] [Crossref]

14. Volume.10, Issue 4, pp.e73-e78, April 2022 Reddy, Shiva & Channegowda, Nandini. (2021). Edge Boost Curve Transform and Modified ReliefF Algorithm for Communicable and Non-Communicable isease Detection Using Pathology Images. International Journal of Intelligent Engineering and Systems. 14. 463-473.10.22266/ijies2021.0430.42. [Google Scholar] [Crossref]

15. Manjunath, D. R., Selva Kumar S., Sai Shiva Sumanth Reddy, and Lohith J. J. "Enhancing Personalized Learning Based on AI-Driven Lesson Plan Generator Using Mistral- 7B for Efficient Content Extraction and Summarization." Gradiva Journal, June 2024, https://gradiva.it/june-2024/. [Google Scholar] [Crossref]

• Most Advanced Construction Material for Typical Boilers
• Nutrigenetic Impact of PEMT Gene Polymorphism Rs7946 On Choline Metabolism and Its Role in Personalised Nutrition
• Evaluating the Influence Factors of Media on Investor Sentiment Towards Stock Market Investments.
• To Compare the Effectiveness of Manual Traction and Mechanical Traction in Patient with Cervicogenic Headache
• Is Green Human Resource Management the Newest Trend or A Strategic Necessity?