Data Engineer H2O Innovation
Develop data workflows in Python and SQL using AWS Glue, S3, Athena, Lambda, IAM, and Lake Formation.
Implement REST API ingestion with pagination, authentication and session handling, and failure recovery. Build and maintain ETL/ELT pipelines integrating SAP, APIs, and on-premises and cloud databases into bronze, silver, and gold lakehouse layers.
Provision and maintain AWS lakehouse infrastructure through Terraform, including state backend management and development and production environments. Orchestrate workflows with Airflow and investigate production failures using execution logs and Grafana.