MUMBAI, India, Aug. 12 -- Intellectual Property India has published a patent application (202641094003 A) filed by Neelambika Basavaraj Hiremath; Dr. Sachchida Nand Mishra; Dayananda Sagar Academy Of Technology And Management; Pranav Patil; Sonu H T; Prajwal; and Sachin B on August 03, 2026, for Data Lakehouse Pipeline System: A Three-Layer Medallion Architecture With Etl/Elt Framework For Automated Data Quality Management And Real-Time Business Intelligence.
Inventors include Dr. Sachchida Nand Mishra; Pranav Patil; Sonu H T; Prajwal; and Sachin B.
The application for the patent was published on August 07, 2026, under issue no. 32/2026.
Abstract: Data Lakehouse Pipeline System: A Three-Layer Medallion Architecture with ETL/ELT Framework for Automated Data Quality Management and Real-Time Business Intelligence” The present invention relates to a Data Lakehouse Pipeline System implementing a three-layer Medallion Architecture comprising Bronze, Silver, and Gold layers integrated with an ETL/ELT framework for automated data quality management and real-time business intelligence generation. The system receives e- commerce transaction data through live web form ingestion, CSV batch import with independent column mapping, synthetic transaction stream generation, and Apache Kafka event simulation. The Bronze Layer stores raw records in Delta Lake append-only storage with ACID-compliant transactions. The Silver Layer performs ETL transformations including null-value removal, data type casting, text standardization, and MD5 content-hash-based duplicate detection using customer_name, customer_email, purchase_amount, product_category, and transaction_date to prevent duplicate data ingestion. The Gold Layer generates business intelligence fact tables including Monthly Sales by Category and Daily KPI Metrics for reporting and visualization. The invention further supports simultaneous ingestion of multiple CSV datasets with different schema structures through independent column mapping. The system was validated on datasets exceeding 2.2 million records and achieved approximately 50.1% improvement in data quality while completing ingestion in under 2 seconds on standard CPU-based hardware without requiring distributed computing infrastructure. The system is implemented using Python, Delta Lake, and Streamlit and supports both local and cloud deployment environments including Azure Data Lake and Databricks platforms.
Disclaimer: Curated by HT Syndication.
