MUMBAI, India, Aug. 17 -- Intellectual Property India has published a patent application (202641096102 A) filed by Manivannan K; R. Murugesan; B. Bharathi; C. Vishnupriya; S. Muthulakshmi; and V. S. B. Engineering College, Karur on August 08, 2026, for Context-Aware Adaptive Multimodal Intelligence For Real-Time Sign Language Recognition And Translation.
Inventors include Manivannan K; R. Murugesan; B. Bharathi; C. Vishnupriya; S. Muthulakshmi; and V. S. B. Engineering College, Karur.
The application for the patent was published on August 14, 2026, under issue no. 33/2026.
Abstract: Communication barriers experienced by deaf, hard-of-hearing, and speech-impaired individuals continue to limit their participation in education, healthcare, public services, and social interactions. Although recent sign language recognition systems have achieved promising results using deep learning, many existing approaches primarily focus on isolated hand gestures and exhibit reduced robustness when dealing with continuous signing, facial cues, body movements, environmental variations, and differences among signers. To address these limitations, this study proposes an **advanced multimodal deep learning framework for real-time continuous sign language recognition and translation**. The proposed framework extracts complementary spatial representations from hand gestures, facial expressions, and upper- body movements using lightweight convolutional feature extraction, followed by temporal modelling to capture the contextual relationships between successive signs. An adaptive attention-based multimodal fusion mechanism is introduced to dynamically assign importance to discriminative features from different modalities, thereby improving recognition under variations in illumination, background, signing speed, and signer appearance. The resulting representation is processed through a sequence classification and translation module to generate textual output and corresponding speech in real time. In addition, the framework is designed with computational efficiency in mind to support deployment on resource-constrained edge and mobile platforms. Comprehensive experiments using sign language datasets and real-world test scenarios are conducted to evaluate recognition accuracy, robustness, computational efficiency, and generalization across unseen signers. The proposed approach demonstrates improved recognition consistency and multimodal feature utilization compared with conventional single-modal and sequential sign language recognition methods. The developed system provides a scalable and accessible solution for intelligent sign language interpretation and has potential applications in assistive communication, smart healthcare, inclusive education, and public-service environments. An Adaptive Multimodal Deep Learning Framework for Real-Time Continuous Sign Language Recognition and Translation
Disclaimer: Curated by HT Syndication.
