MUMBAI, India, Sept. 7 -- Intellectual Property India has published a patent application (202611084221 A) filed by Noida Institute Of Engineering And Technology Niet on July 09, 2026, for Hardware-Aware Kv Cache Compression System With Fused Gpu Kernels For Llm Inference.
Inventors include Dr. Sangeeta Arora; and Tushar.
The application for the patent was published on September 04, 2026, under issue no. 36/2026.
Abstract: A hardware-aware sliding-window key-value cache compression system (100) for efficient inference serving of large language models on GPU accelerator hardware is disclosed. The system (100) comprises a sliding-window compression engine (110), a fused GPU kernel module (120) executing on GPU streaming multiprocessors that consolidates bidirectional attention scoring, TopK candidate selection via warp-level bitonic sort, and chunk boundary annotation into a single kernel launch using shared memory tiling, a Token-to-Chunk expansion module (130) preserving flexible-sized contiguous KV pair groups, a hardware-aware compression policy controller (140) dynamically adjusting parameters based on GPU telemetry, a periodic compression scheduler (150), and a PagedAttention integration layer (160) performing block-level memory compaction using coalesced GPU memory copy operations. The system reduces GPU high-bandwidth memory footprint while preserving inference accuracy, enabling higher throughput under memory-constrained concurrent serving. [Figure 1]
Disclaimer: Curated by HT Syndication.
