MUMBAI, India, Sept. 22 -- Intellectual Property India has published a patent application (202641108525 A) filed by Vellore Institute Of Technology on September 10, 2026, for System And Method For Hardware-Aware Context Transformation And Multi-Tier Memory Orchestration For Long-Context Inference.
Inventors include Ramesh Kumar D; and M S Shabreena.
The application for the patent was published on September 18, 2026, under issue no. 38/2026.
Abstract: Embodiments herein provide a runtime inference system (106) for performing long-context inference using a transformer-based neural network including a large language model (LLM). The system (106) includes a processor (108), accelerator memory (112), system memory (114), and secondary storage (116). The processor (108) generates Key-Value (KV) representations from an input context (102), predicts future contextual relevance across successive layers, and classifies context segments as active or transformable. The processor (108) retains active KV representations in the accelerator memory (112), transforms transformable KV representations into compact differential representations, and allocates the representations across the accelerator memory (112), system memory (114), and secondary storage (116). The processor (108) further schedules asynchronous transfers based on an execution timeline, restores required KV representations before a corresponding future layer, and updates runtime policies based on observed execution behavior. FIG. 1
Disclaimer: Curated by HT Syndication.
