Data Engineer

Designed and developed ETL workflows using Azure Data Factory, Apache NiFi, and Snowflake, optimizing ingestion of 500M+ records monthly and accelerating business intelligence reporting. Implemented PySpark transformations in Databricks, reducing data processing time and enabling near real-time analytics for operations and finance teams. Built and maintained Tableau dashboards and SQL-based pipelines, streamlining KPI monitoring and reducing ad performance reporting time. Collaborated with cross-functional stakeholders to define KPI logic, develop scalable data models, and enable self-service reporting across commercial and marketing analytics teams.

Led 11 structured lab sessions on PostgreSQL and DBeaver, teaching relational joins, subqueries, and indexing techniques using 3 practice databases. Demonstrated cloud-based ETL workflows using Azure Data Factory and BigQuery, guiding 32 students through 5 ingestion pipelines built for class projects. Evaluated 28 SQL and data modeling assignments, scoring 6 schema design tasks and 4 optimization challenges per submission for final grading. Integrated Apache Airflow and Azure Synapse Analytics into 3 hands-on labs, simulating orchestration tasks and warehouse query execution across 2 course modules.

Migrated 10 enterprise data workflows from Netezza to BigQuery using Ab Initio, applying source-to-target schema mapping to support readiness across 3 environments. Orchestrated 12 ETL pipelines in Control-M, automating nightly data transfers and reducing manual reruns by 8 per week. Provisioned BigQuery and AWS S3 buckets for secure archive storage and configured Cloud Shell access, streamlining data delivery for 6 downstream systems. Constructed CI/CD hooks using GitHub Actions to auto-deploy migration configs across Dev, UAT, and Prod, reducing manual deployment steps by 18 per cycle. Transformed 40 legacy SQL queries for BigQuery schema compatibility, reducing syntax-related parsing errors by 22 during QA cycles. Applied UNIX shell scripting and Ab Initio to standardize backend batch jobs across 15 structured datasets, reducing job failure frequency by 5 per week. Integrated Kafka with GCP Pub/Sub for ingesting operational logs during test runs, enabling detection of 2 critical alert triggers pre-production. Automated post-deployment notifications using AWS Lambda and documented rollback protocols in Confluence, accelerating onboarding for 2 offshore teams.

Automated ingestion of real-time sensor logs from 7 grid substations using GCP Dataflow, consolidating telemetry in BigQuery to accelerate fault detection and inventory alerts. Engineered a Python forecasting model with Pandas and NumPy to predict energy load spikes, enabling proactive transformer dispatch across 3 energy zones. Developed 5 interactive Power BI dashboards with DAX, visualizing outage trends and maintenance schedules for 8 control centers, improving decision-making for operations teams. Investigated failure patterns and lead times for 40+ critical grid components using SQL and Python, streamlining spare inventory planning and reducing manual tracking effort. Standardized lifecycle data for 120+ equipment records and created 8 metadata templates in Azure Data Lake and BigQuery, improving data consistency for preventive maintenance and audits.

Queried and cleaned 12 equipment datasets from PostgreSQL and MySQL, and performed staging in Azure Data Lake for reporting automation and schema alignment. Constructed 2 Tableau dashboards to visualize downtime, throughput, and energy usage across 5 logistics hubs, improving visibility for the asset maintenance team. Generated 7 weekly Excel summary reports using PivotTables and VLOOKUP, supporting lifecycle analysis and performance tracking for grid transformers.