Azure Data Engineer Unizen Technologies
Designed and implemented end-to-end batch data pipelines using Azure Data Factory and PySpark jobs on Azure Databricks. Ingested and processed large-scale transactional data from multiple source systems.
Organized data storage using medallion architecture, created linked services and datasets, and secured connections with Azure Key Vault. Implemented SCD Type 1 and Type 2 patterns, scheduled incremental loads, optimized Spark job performance, and resolved complex transformation issues.