Sumit Kumar
Architecting enterprise data platforms, robust ingestion engines, and autonomous Agentic AI pipelines across fintech and retail domains using AWS, Redshift, Kafka, and Generative AI ecosystems.
About Me
My Journey
Data engineering lead focused on building highly-scalable data platforms, real-time pipelines, and autonomous AI systems. Experienced in driving data engineering excellence across fintech and retail domains—from Redshift bulk ingestion engines to LLM-driven Agentic AI architectures.
I believe in the power of data to transform businesses and lives. My focus is on creating robust, scalable solutions that not only solve immediate problems but enable future growth.
0+
Years Experience
0+
Projects Completed
Impact & Leadership
Professional Influence
Beyond building pipelines, I focus on driving technical maturity, mentoring teams, and architecting solutions that scale with the business.
Team Mentorship
Mentoring junior engineers and interns, fostering a culture of technical excellence and best practices in data engineering.
Platform Design
Contributing to data platform architecture decisions, ensuring scalability and reliability for organization-wide analytics.
Operational Impact
Driving initiatives that improved data processing efficiency by 30% and significantly reduced pipeline maintenance overhead.
Data Governance
Implementing robust data quality checks and security protocols across multiple high-impact data products.
Architecture Deep Dive
Scalable Data Systems
Visualizing the complex data foundations I've built. These architectures are designed for high throughput, reliability, and business impact.
CDC Pipeline
Real-time Change Data Capture (CDC) pipeline enabling low-latency data replication from transactional databases to analytics hubs.
Core Components
Business Impact
Experience
Senior Manager – DSIT
Data Pipelines & Platform Architecture Architected a React/FastAPI Master Data Management platform, consolidating 65+ pi...
Data Engineer
Engineered custom Java-based Kafka Connect SMT library for Debezium CDC processing from AWS DocumentDB, achieving 100% d...
Data Engineer
Developed a metadata-driven ETL framework (Python, PySpark, Airflow) for PayPal’s merchant data migration to BigQuery, o...
Data Science Research Intern
Developed a real-time forest fire detection system utilizing Python-based machine learning algorithms and fuzzy logic....
Technical Skills
Technologies and tools I work with professionally
My Work / Projects
Showcase of my professional work, internships, and side projects.
Featured Work
Smaxiso Writes
A collection of poetry exploring themes of love, life, and introspection. Hosted poetry portfolio.
Smaxiso Writes
A collection of poetry exploring themes of love, life, and introspection. Hosted poetry portfolio.
AI Hub
Centralized hub for AI models and tools, streamlining access and management of various artificial intelligence resources.
Local RAG
Local Retrieval-Augmented Generation system for private document chat, enabling secure and offline AI interactions with personal data.
Real-time Transaction Normalization & Scalable Data Lake Design
Built a real-time transaction normalization pipeline using Kafka (AWS MSK), ECS, and Java (Spring Boot) for fraud detection. Optimised the normalization service for vendor compatibility. Designed a scalable Data Lake using S3 Hudi and optimized ETL pipelines for Athena querying. Built a Java-based test automation service.
School Chale Ham
An academic blogging platform for K-12 education. Features include efficient blog creation and management. Backend powered by Express.js with MongoDB; Frontend built with Next.js.
Enterprise MDM Platform
Architected a React/FastAPI Master Data Management platform, consolidating 65+ pipelines from ad-hoc Excel/Glue workflows into a centrally governed system with RBAC-scoped self-service access for business teams.
Autonomous Self-Healing AI Agent
Architected a Self-Healing AI Agent that connects into any underlying automation service, intercepts crashes, generates LLM-driven code fixes, validates via AWS CodeBuild, and submits Pull Requests for human review.
Amazon SP-API SDK & Pricing Engine
Engineered an agentic pricing pipeline on SP-API that scores product similarity and branches decisioning. Built the underlying SP-API SDK (SigV4 signing, token caching, circuit breakers), now adopted across the team's automation suite.
Hybrid Supply Chain Optimizer
Engineered a hybrid ML/OR system (ADMM, EOQ) for supply chain inventory optimization, demonstrating architectural rigor across complex mathematical domains.
CLI Chat Bot
Command-line interface chatbot focused on developer productivity, offering quick access to tools and information via terminal.
VS Code Productivity Extension
Custom Visual Studio Code extension built to enhance developer workflows and automate repetitive coding tasks. Features include enhanced markdown previewing and snippet automation.
Contextual News System
AI-driven news aggregation system using NLP for content personalization and relevance.
Data Migration Framework
Developed a scalable Data Migration Framework for a global payments company using Python, AWS, GCS, and BigQuery. Reduced data migration time by 20% and improved scalability by 30%.
Lynx Framework Optimization
Optimized the Lynx entity linkage framework, improving accuracy and efficiency. Enhanced the Locality-Sensitive Hashing (LSH) algorithm, reducing nearest neighbor search time by 40%. Streamlined feature aggregation pipelines, reducing processing latency by 25%.
Google Search Pro
Advanced search scraper and utility tool designed for enhanced information retrieval and data extraction efficiency.
Android Bloatware Removal Guide
A basic blog webpage created for the Android bloatware removal guide. Simple and informative design for easy navigation.
Reporting Framework
Developed on-demand merchant reporting solutions, improving data accuracy by 15% and reducing report generation time by 25%. Contributed to the Argo Framework for report generation at scale.
Fuzzy Control System for Forest Fire Detection
Developed a real-time forest fire detection system utilizing Python-based machine learning algorithms and fuzzy logic. Achieved 90% accuracy in predicting the likelihood and severity of forest fires.
Bihar COVID Help
Built a resource-sharing platform for COVID-19 relief, connecting volunteer doctors with patients and aggregating information on critical supplies like oxygen and hospital beds.
Joint Image Compression & Encryption
Created an algorithm for joint image compression and encryption using lossless JPEG2000 and RC4 encryption. Achieved a compression ratio of 5.2, 99.69% NPCR, and 47.63% UACI in processed images.
Digital Image Compression
Designed an algorithm for digital image compression using K-means clustering and PCA. Achieved a compression ratio of 2.8 with 55-70% compression and a PSNR of 30 and above. Collaborated with a team of 4 members.
Image Classification using CNNs
Led a team of 4 in developing a machine learning-based CNN model for image classification, utilizing the CIFAR10 dataset. Achieved a high accuracy rate of 87.44% on the training dataset and 82.5% on the testing dataset.
YouTube Spam Comment Filter
Developed and implemented a machine learning algorithm for binary classification of spam and non-spam comments on YouTube videos. Achieved an accuracy rate of 96.21% through training on multiple datasets.