Machine Learning Engineer
Building AI agents and ML systems at Chaos Genius. Passionate about making intelligent systems that optimize cloud costs and transform how enterprises use data.
Professional Journey
Currently focused on building AI agents to enable automatic cost savings across Databricks and Snowflake based on users' SLA requirements. Using SOTA models (both open-weight and closed source) and building MCP servers and tools to allow agents to understand the history of clusters and warehouses and optimize them accordingly.
Created an LLM-powered parser for spark application logs, to find inefficiencies and methods on how to fix them. Built our entire pipeline for our Databricks Cost Observability product. Deployed a mixture of ARIMA, Prophet and Neural Prophet models to enable forecasting. Created ETL pipelines to provide statistical analysis of usage costs. Worked on creating the backend for Snowflake Observability product using FastAPI, and added SSO support through AWS Cognito.
Building an open source platform to perform automated Root Cause Analysis, Anomaly Detection and Data Quality at scale for enterprises and easily deployable for developers & small organisations.
Working on building a robust Chatbot and Q&A system to answer queries related to the funding (shares, options, securities) and management part of startups.
Started the organization to coordinate relief efforts from NGOs to make sure areas were not oversupplied. Built a web platform to achieve this. We also participated in Mental Health, getting people jobs and fundraising for NGOs.
This book was published by O'Reilly and is currently topping the charts internationally. I worked closely with Anuj Gupta and Harshit Surana on reading and editing drafts of the book and contributed to checking and quality testing the code for the book. The book was also endorsed by Zachary Lipton, Sebastian Ruder & Marc Najork.
Built models to predict marketing and usage based on requirements from clients using various ML algorithms. Worked on custom models such as GAMs (Generalized Additive Models) to boost the interpretability of the model.
Deployed and maintained an elaborate logging system for AWS servers with dashboards and alerts for real-time updates on server status and loads using Logstash and Kibana.
Selected Works
A minimal, borderless desktop timer application built with Rust and egui. Features decimal minute inputs, visual red flashing alerts, draggable interface, and always-on-top toggle. Cross-platform support for macOS, Linux, and Windows.
Built extensions for Chrome, Firefox and VSCode to allow users to clone and open GitHub projects into VSCode directly from your browser. No more copy pasting git URLs.
Worked on creating a program to remove backgrounds from frames in real-time for use in video streaming. A custom architecture heavily inspired from Mask R-CNN and tinyYOLO was used, which significantly reduced the model size and complexity and achieved 30fps on a CPU.
Matplotlib and Seaborn didn't have built-in support for Bump Plots which are used to represent rank changes over time. Writing a single function to do this proved to be a complex function, so I created a small library for it.
A camera that tracks your face around the room using servos to pan and tilt controlled by an Arduino which is fed instructions by a RasPi 2B+. Using openCV and ROS, 5-10 FPS is obtained.
Visualization of the Q-Learning algorithm and how it applies to an example path finding problem where we can see the Q-Table updating as the agent learns to move around.
Publications
AIMLSystems '23: Third International Conference on AI-ML Systems
Shows that popular transformer-based NLP models perform poorly on noisy text data across tasks like classification, NER, QA, and summarization, with performance degrading as noise increases.
ICAPIE 2022, Advances in Transdisciplinary Engineering
Novel architecture for near-real-time image segmentation using YOLOv4 for lightweight object detection, enabling deployment on consumer-grade hardware.
Academic Background
Delhi Technological University
Amity International School, Noida
Amity International School, Noida