Data Engineer

Developed data transformation jobs using Databricks SQL and Azure Datafactory (ADF) in Azure, enhancing data quality scores by 25% through rigorous validation and cleansing processes before transferring data. Built and tested data pipelines using Talend cloud; improved data accuracy by deploying automated quality checks that ensured compliance with business requirements before migrating to the AWS cloud platform. Performed unit and integration testing to ensure data quality and job reliability. Developed and optimized complex SQL queries using PostgreSQL. Tuned performance, reducing query time by 30% through indexing techniques. Created 5 PowerBI reports analyzing employee utilization for the HR department. Implemented precise mapping documents and refined outbound scripts, resulting in a 30% reduction in errors during data processing.

Designed 5 PowerBI reports and dashboards using DAX formulas, based on requirements from business users. Built 5 Nifi jobs to Extract, transform and load data from various data sources using Apache Nifi and Pentaho integration tool.

Managed the Data warehouse and kept data consistency and integrity in the database. Developed 15 PowerBI dashboards to visualize customer data trends, supporting the marketing teams in executing targeted campaigns, using DAX logic and tabular modeling techniques. Led the design and implementation of more than 20 ETL pipelines using SSIS and SSRS to ingest and transform 8GB of client data daily for reporting. Troubleshot and fixed data issues and monitored SSIS jobs and reports.

Solved customer’s application support incidents raised by the Users. Owned the weekly report generation and supporting User's request related to the module.

Involved in building 20 ETL jobs using Informatica Power Center to load data from Oracle, MS SQL, Flat files to staging database and from staging to the target Data Warehouse database. Prepared 5 DataStage jobs for importing metadata from repository, new job categories and creating new data elements. Designed 10 SSIS packages to extract data from Excel files, MS Access files using various transformations to generate data for the reports and to export cleaned data to the data warehouse. Completed all KT Application Understanding Documents & other support materials along with the other Team members.