Intellectual Property India Publishes Patent Application for A System And Method For Dynamic, Hardware-Adaptive, Semantically- Routed Execution Of Large Language Model Inference On Local Computing Devices

MUMBAI, India, July 7 -- Intellectual Property India has published a patent application (202621056189 A) filed by Vinay Kiran Diddi on May 03, 2026, for A System And Method For Dynamic, Hardware-Adaptive, Semantically- Routed Execution Of Large Language Model Inference On Local Computing Devices.

Inventor includes Vinay Kiran Diddi.

The application for the patent was published on July 03, 2026, under issue no. 27/2026.

Abstract: The present invention discloses a system and method for locally executing Large Language Model (LLM) inference with real-time, self-optimizing hardware adaptation. The system, named DEPIE (Dynamic Execution Pipeline for Intelligent Engines), includes an Adaptive Layer Allocator that dynamically computes GPU layer offloading proportional to real-time available VRAM; a dual- precision control architecture comprising a stateless prompt-length-based PrecisionController and a stateful PrecisionElasticController that autonomously adjusts quantization precision (INT4/FP16/FP32) based on rolling latency observations against service-level thresholds; a three-tier Model Registry mapping "fast" (Phi-3.5 Mini, INT4), "balanced" (Mistral 7B, FP16), and "powerful" (Meta- Llama 3.1 8B, FP32) execution routes; a Semantic Router with Feedback Engine that classifies prompts by intent, domain, and complexity and routes them to appropriately-sized configurations; a multi-tier caching system comprising exact-match, semantic similarity, and response segment reuse layers; a Prompt Decomposition module that splits compound prompts into independent sub- tasks each processed through the full pipeline; an extensible Plugin Architecture enabling third-party model runtime adapters; and an out-of-memory recovery mechanism with geometric parameter reduction. Together, these components form a closed-loop, self- optimizing local AI inference pipeline capable of operating entirely without cloud connectivity, providing intelligent resource utilization, improved response times, and graceful fault tolerance on consumer hardware.

Disclaimer: Curated by HT Syndication.