Emotion-Aware Multilingual Multimodal Emergency Detection System Using Edge AI and Context-Adaptive Learning

In critical emergency situations, victims often express distress through voice, language, and physical movements rather than explicit manual actions. Existing safety systems fail to capture such multimodal and multilingual cues effectively. This paper proposes a novel Emotion-Aware Multilingual Multimodal Emergency Detection System (EMMEDS) that integrates speech emotion recognition, multilingual text understanding, motion sensing, and contextual awareness using lightweight edge AI models.

The proposed framework combines convolutional neural networks (CNNs) for audio feature extraction, transformer-based multilingual text processing, and long short-term memory (LSTM) networks for motion sequence analysis. A context-adaptive attention mechanism dynamically adjusts the importance of each modality based on environmental conditions. Unlike existing cloud-dependent solutions, the system performs real-time inference on-device, ensuring low latency and privacy preservation.

Experimental results demonstrate a significant improvement of 19% in detection accuracy and a 25% reduction in false alarms compared to traditional unimodal and cloud-based approaches. The system is highly scalable and suitable for real-world deployment in personal safety, smart cities, and healthcare monitoring.

Multimodal Learning, Emotion Detection, Multilingual NLP, Edge AI, Emergency Detection, Deep Learning, Context-Aware Systems

1. T. T. Sasidhar, B. Premjith, and K. P. Soman, "Emotion detection in Hinglish (Hindi+English) code-mixed social media text," Procedia Computer Science, vol. 171, pp. 1346–1352, 2020. [Google Scholar] [Crossref]

2. D. Vijay, A. Bohra, V. Singh, S. S. Akhtar, and M. Shrivastava, "Corpus creation and emotion prediction for Hindi-English code-mixed social media text," in Proc. 2018 Conf. North Amer. Chapter Assoc. Comput. Linguistics: Student Res. Workshop, 2018. [Google Scholar] [Crossref]

3. S. Kumar, S. Kumar, S. R. Singh, and S. Nandi, "indiDataMiner at SemEval-2025 Task 11: From text to emotion: Transformer-based models for emotions detection in Indian languages," in Proc. 19th Int. Workshop Semantic Evaluation (SemEval-2025), 2025. [Google Scholar] [Crossref]

4. S. I. Khan, F. B. Aziz, and M. M. Uddin, "Emotion detection from multilingual text and multi-emotional sentence using difference NLP feature extraction technique and ML classifier," Int. J. Adv. Networking Appl., vol. 14, no. 3, pp. 5429–5435, 2022. [Google Scholar] [Crossref]

• Block-Based Programming for Education: A Comprehensive Analysis of Visual Programming Environments in K-12 Learning
• Management of Academic Libraries and Client Satisfaction Towards Digital Utilization: Basis for Monitoring Library Operations in SOCCSKSARGEN Region.
• Revenue Leakages in TPA Insurance Claims and Corporate Claims: An Institutional Overview of Aster Prime Hospital, Hyderabad
• Technology and Innovation in Hospitality and Tourism: A Management Perspective
• Geospatial Distribution of Tarok Sacred Grove of Langtang North and Langtang South Local Government Areas