Senior Data & Machine Learning Engineer

Designed and implemented a real-time fraud detection pipeline using Spark Structured Streaming, Kafka, and Postgres enrichment, enabling sub-second anomaly alerts and reducing fraudulent transaction losses by 17% in the first year. Operationalized ML models in SageMaker using MLflow for tracking and version control, integrating FastAPI APIs to serve batch and streaming inference with a 35% improvement in model deployment time. Re-engineered legacy pipelines into modular dbt projects with Airflow orchestration, reducing developer onboarding time by 40% and significantly improving DAG clarity for cross-team collaboration. Integrated S3-based data lakes with Redshift Spectrum to allow ad-hoc analytics without additional ETL processing, cutting query turnaround times by 50% for business teams. Built monitoring dashboards with Grafana and Prometheus to track Kafka queue lag, ML latency, and Spark job health, reducing incident resolution time by 25% through proactive alerting. Developed an internal Feature Store using DynamoDB and S3, ensuring feature parity between training and inference stages, eliminating 90% of feature drift cases. Mentored junior engineers on Terraform, Docker, reproducible Jupyter workflows, and experiment tracking best practices, raising team productivity and code quality. Partnered with compliance and governance teams to create ML explainability standards, including SHAP visualizations for regulatory audit readiness. Automated model retraining workflows with CI/CD integration, ensuring production models remain above 95% accuracy thresholds without downtime. Collaborated with data scientists and product managers to align ML feature engineering with evolving fraud prevention strategies, improving detection coverage for new fraud patterns.

Built predictive maintenance models from high-volume telemetry data using Azure Event Hubs, Azure ML Pipelines, and Python, decreasing unplanned downtime by 20% across monitored assets. Migrated legacy ETL jobs from SSIS/SQL Server to Azure Data Factory + Synapse, reducing refresh cycles from multiple hours to under 30 minutes for key datasets. Implemented automated retraining pipelines with integrated performance thresholds and model monitoring, ensuring only high-quality models were promoted to production. Developed and deployed secured inference endpoints using Azure Functions and Application Gateway with RBAC, handling over 100K scoring requests daily without service degradation. Created a centralized Model Registry to manage versioned ML deployment and rollback, improving deployment safety and reducing incident recovery time by 40%. Authored Jupyter-based experimentation templates for consistent model evaluation, reducing experimentation setup time by 60% for new projects. Led the design of hybrid online/offline inference strategies for personalization engines, increasing user engagement by 12% in targeted pilots. Enhanced operational visibility by integrating Azure Monitor with model endpoints, tracking latency, accuracy drift, and feature health in real time. Collaborated with cross-functional teams to align ML deployment with product release schedules, ensuring zero downtime during production model swaps.

Modernized ETL workflows by migrating from Informatica to PySpark on AWS EMR, reducing batch execution time by 50% and improving scalability for future workloads. Built IoT telemetry streaming pipelines with Kafka and Spark for real-time enrichment and aggregation stored in S3, enabling near-instant analysis for operational dashboards. Developed dbt transformation models for Snowflake and Redshift powering Power BI dashboards with consistent KPIs across global teams. Designed a segmentation engine with partitioned Redshift loads and snapshot logic, reducing dashboard latency by 30% for marketing analytics. Migrated legacy reporting systems from Oracle DW to Snowflake using automated schema mapping and validation scripts, ensuring zero data loss in the transition. Authored Python CLI utilities for DAG orchestration, log parsing, and incremental data quality checks, improving debugging efficiency. Tuned materialized views and Snowflake clustering strategies in collaboration with BI teams, leading to a 25% improvement in dashboard query performance. Maintained Jenkins CI/CD pipelines for Spark batch jobs and dbt assets, ensuring consistent deployment across environments.

Maintained and enhanced daily ETL pipelines in PL/SQL and Informatica for HR and finance dashboards, ensuring high data freshness and reliability. Built SQL Developer data quality dashboards to monitor pipeline health, row counts, and SLA compliance, enabling early detection of failures. Refactored complex Oracle SQL queries through indexing and CTE restructuring, reducing runtimes by 60% for critical reports. Contributed to warehouse re-architecture efforts, normalizing dimensions and improving fact table grain definitions for better analytics performance. Automated QA scripts for nightly snapshots and reconciliation reports, reducing manual validation workload by 80%. Migrated reports from Oracle Discoverer to Oracle BI Publisher, adapting SQL sources and layouts for modern reporting.