MUMBAI, India, June 30 -- Intellectual Property India has published a patent application (202641073803 A) filed by Vellore Institute Of Technology on June 14, 2026, for Workflow-Aware Evaluation Framework For Public-Service Task Completion By Language Models In Low-Resource Indic Languages.
Inventors include R Sujatha; Ayush Priyaranjan; and Varun Verma.
The application for the patent was published on June 26, 2026, under issue no. 26/2026.
Abstract: “WORKFLOW-AWARE EVALUATION FRAMEWORK FOR PUBLIC- SERVICE TASK COMPLETION BY LANGUAGE MODELS IN LOW- RESOURCE INDIC LANGUAGES” A workflow-aware evaluation system and method benchmark language models executing multi-step public-service procedures in low-resource Indic languages. The system comprises a workflow modeling module that encodes procedures as finite-state machines (FSMs) with defined states and transitions. A scenario generator samples synthetic user profiles to produce simulated user utterances with controlled code-mixing, script-mixing, and dialectal variations. A dialogue environment orchestrates multi-turn interactions between a candidate language model and the scenario generator, parsing model outputs into workflow actions to advance the FSM states. An evaluation engine analyzes logged interaction traces to compute performance metrics, including a workflow success score, a linguistic robustness score, and a safety compliance score. This provides a standardized, region-specific benchmark evaluating stateful task completion under realistic linguistic perturbations. [FIG. 6]
Disclaimer: Curated by HT Syndication.
