Senior Data Engineer

Build and operate batch and incremental ingestion pipelines for structured and semi-structured insurance data using Python, SQL, PySpark, and Databricks. Deliver transformed and curated datasets for downstream consumers.

Create reusable ingestion and transformation patterns with schema checks, source-to-target reconciliation, audit controls, and exception handling. Investigate Spark bottlenecks and failed jobs, applying partitioning, join optimization, shuffle reduction, caching, and compute right-sizing.

Implement monitoring, SLA alerts, failure recovery, and operational support processes. Maintain Delta Lake assets with metadata, lineage, and access controls, and use Git and CI/CD for repeatable deployments across development and production environments.

Developed Python, SQL, and PySpark ETL/ELT workflows to ingest, cleanse, transform, and publish financial data for reporting and downstream applications. Implemented reusable logic for complex joins, deduplication, aggregation, and standardized business rules.

Designed reconciliation checks for record counts, control totals, null values, duplicate records, schemas, and business-rule exceptions. Investigated source-to-target mismatches and supported consistent, reliable financial datasets.

Resolved orchestration dependencies and production job failures through root-cause analysis, controlled reruns, and backfills. Tuned Spark execution plans and partitioning strategies while supporting REST-based data exchange, secure access, audit controls, and SLA monitoring.

Built cloud-based data pipelines combining healthcare claims, clinical, laboratory, and operational data from multiple upstream systems for analytics-ready use. Authored Python, SQL, PySpark, and Spark SQL transformations for normalization, validation, and curation of high-volume healthcare datasets.

Implemented incremental processing patterns, reusable data completeness checks, schema handling, and exception-management logic to improve downstream delivery reliability. Supported Spark and Kafka batch and streaming workflows, including reconciliation, monitoring, recovery, and operational troubleshooting.

Applied role-based access controls, masking, and auditable processing practices for sensitive PII and PHI data in HIPAA-aligned environments.

Master of Science in Computer Systems Technology. GPA: 3.5/4.0.