MUMBAI, India, Aug. 12 -- Intellectual Property India has published a patent application (202641094149 A) filed by Anurag University on August 04, 2026, for A Multilingual Multimodal Lip-Reading And Speech Recognition System Using A Video Transformer- Based Cross-Modal Fusion Architecture.
Inventors include Banavatu Jyothi; T. Venkata Subbamma; Dr. M. Sridevi; Keerthana Bomma; Syed Mohammed Hafeeza; and P. Charitha.
The application for the patent was published on August 07, 2026, under issue no. 32/2026.
Abstract: ABSTRACT The present invention discloses a multilingual multimodal lip-reading and speech recognition system (100) using a video transformer-based cross-modal fusion architecture. The system comprises a video input module (102) and an audio input module (106) configured to capture synchronized visual lip movements and speech signals. A lip detection and visual feature extraction module (104) extracts temporal lip movement features, while a speech processing and audio feature extraction module (108) extracts acoustic features from the received speech signals. An audio-visual fusion model (110) combines the visual and audio features through an adaptive cross-modal fusion mechanism to generate enhanced speech representations. A multilingual recognition module (112) processes the fused representations for real-time speech recognition across multiple languages. A text output module (114) generates live subtitles, translations, and command outputs. The system improves recognition accuracy in noisy or silent environments and enables efficient assistive communication, human-computer interaction, and real-time multilingual transcription applications. The figure associated with the abstract is Fig. 1.
Disclaimer: Curated by HT Syndication.
