MUMBAI, India, Sept. 28 -- Intellectual Property India has published a patent application (202621098048 A) filed by Dr. D. Y. Patil Institute Of Technology, Pimpri, Pune -; and Dnyaan Prasad Global University, School Of Technology And Research, Pimpri, Pune - on August 12, 2026, for A Hybrid Perception Based Multilingual Accessibility Assistant For Mobile Screen Understanding.
Inventors include Chetana Shravage; Anurag Gaikwad; Niral Shah; Rushikesh Waghmare; and Yashvardhan Jadhav.
The application for the patent was published on September 25, 2026, under issue no. 39/2026.
Abstract: The present invention discloses a hybrid perception based multilingual accessibility assistant for mobile screen understanding that enables users to comprehend complex smartphone interfaces through automated interpretation and audio guidance. The system captures the active mobile screen and processes the visual content using a combination of optical character recognition and vision-language understanding techniques. Textual information, graphical symbols, interface controls, icons, menus, and layout relationships are analysed together to generate a comprehensive understanding of the screen content. A hybrid processing module integrates extracted text and visual context to produce a meaningful representation of the displayed interface. The generated information is further simplified to highlight essential functions and reduce unnecessary details, thereby improving user comprehension. A multilingual translation component converts the interpreted content into the user’s preferred language, allowing broader accessibility across diverse linguistic backgrounds. The translated description is then transformed into natural speech through a speech synthesis engine, enabling hands-free interaction and audio-based navigation. The proposed system performs processing primarily on the device, supporting privacy, reducing dependency on external services, and maintaining efficient operation in limited connectivity environments. By combining visual reasoning, language translation, and speech generation within a unified framework, the invention significantly improves accessibility, usability, and understanding of modern mobile applications for users worldwide today.
Disclaimer: Curated by HT Syndication.
