Data Engineer

I develop event-driven data ingestion pipelines on the MAT platform and have contributed to approximately 15 production automations. These solutions replaced manual monitoring workflows and reduced recurring NOC reporting effort by an estimated 70–80%.

I create Python normalization layers for multi-vendor network telemetry in JSON and XML, integrating REST and GraphQL APIs to standardize payloads for reporting and troubleshooting. I also implement idempotency and state-control patterns to support reliable retries and prevent configuration drift.

I built a real-time log monitoring pipeline using OpenSearch (ELK), reducing SOC incident detection time from approximately 20–30 minutes to under five minutes. I introduced Git-based CI/CD practices with structured releases and mandatory peer reviews, and integrated operational platforms into a unified near-real-time network-health view.

I refactored legacy SQL Server queries using CTEs, window functions, and materialized views, improving runtime by around 30% for critical incident-tracking tables. I also developed automated ETL flows in Microsoft Fabric for 8,000–12,000 monthly Remedy ticket logs, creating a standardized semantic layer for MTTR and SLA monitoring.

I replaced a six-hour manual Excel KPI reporting process with an automated Python, SQL, and Power BI pipeline that delivered results in 15 minutes for more than 20 stakeholders. I automated Jira API ingestion using Python, Requests, and Pandas, reducing manual capacity-planning effort by approximately 95%.

I designed Power BI dashboards with custom DAX measures for MTTR and SLA tracking. I also mapped operational processes using ITIL, eTOM, and BPMN, contributing to the successful funding of the Automation department.

I architected an AWS Lambda-architecture data lake combining batch ingestion from Airbyte to PostgreSQL with real-time Kafka to Spark Structured Streaming workloads. The platform used Raw, Processed, and Gold layers on Amazon S3.

I built PySpark ETLT pipelines that moved approximately 2.6 million rows from 11 sources into a Delta Lake Gold layer, using partitioning, caching, Airflow orchestration, and integrated data-quality checks. I also modeled Gold OBTs in Amazon Athena for sales trends, RFM segmentation, regional performance, and weather impact analysis.

I automated deployment of Airflow DAGs and Spark jobs through GitHub Actions CI/CD to support streamlined maintenance and updates.

Focused on applied quantitative analysis, linear programming, and stochastic modeling for optimization of data-driven processes.

Completed a project-based program covering Airflow orchestration, Spark/PySpark, AWS services, and dimensional modeling. Built an end-to-end AWS Medallion data lake capstone using Airflow, PySpark, Delta Lake, and SCD Type 2 methodologies.