Data Engineer

Roles & Responsibilities:
• Oversaw the entire project lifecycle, including development, deployment management, and analysing production behaviour.
• Design and development of nineteen distinct ETL workflows into a single scalable data pipeline.
• Automated data validation and implemented data quality checks for reporting data and saved almost 50% manual effort.
• Developed and enforced robust data governance policies and standards to maintain compliance with organizational and regulatory requirements.
• Incorporated AWS Cloud Services as part of “Phase-I migration to Cloud” for other modules.
• Designed a generic architecture for Cloud Security model of an application in AWS, leveraging AWS Inspector, AWS CloudTrail, and AWS Security Hub.
• Administered data access controls, including permissions and roles, to uphold data security and confidentiality
• Successfully onboarded and led a new team, from organizing knowledge transfer sessions to task assignment, to enhance system performance.

Roles & Responsibilities:
• Developed and tested numerous stored procedures and views, optimizing their design to enhance performance and reduced execution time.
• Implemented new data pipeline with Autosys and python scripts to onboard new feed, which enabled the monitoring and execution of multiple processes concurrently and improved efficiency by 50%.
• Designed and implemented automated workflows in Snowflake for efficient data migration, leveraging Snowflake capabilities.
• Independently addressed daily user queries related to system performance issues, functional changes in Snowflake dataset.
• Optimised the overall runtime of approximately 15 interconnected data loading jobs (SQL stored procedure) and reduced run time around 90% (from 2 hours to 14 minutes).
• Introduced a python script for automating data validation processes and data quality checks for both incoming and outgoing Snowflake datasets and eliminated human efforts.
• Monitored and ensured strict adherence to data retention policies and lifecycle management procedures.
• Solely managed a new team, from organizing knowledge transfer sessions to task assignment as well as contributed to improved system performance.
• Employed Agile methodologies extensively, including requirement analysis, sprint planning, and scenario-based test case closure.

Roles & Responsibilities

• Have worked on multiple applications parallelly for masking execution and testing through informatica TDM and IICS.
• Created mappings using different transformations like Filter, Lookup, Union, Joiner, Aggregator, Rank, Update Strategy, Expression, Router and Sequence Generator
• Generated automated test reports using reporting functionalities like Crosstab, Repeater Table, List and chart to reduce around 40% manual effort.

Roles & Responsibilities

• Created stored procedure, view and worked on performance tuning of stored procedure in DB2.
• Modified and created new jobs in Autosys with UNIX shell script to follow several QA frameworks and UAT validation frameworks.
• Developed and worked on OLTP, OLAP processes by using Dimensions, Facts, and standard Data modelling measures.
• Created shell script and new stored procedures for small analytic purpose like run time checks, feed delivery timing, alarm raising, issue logging.
• Scheduled the outbound feed files to deliver on specified folder and directly to business user’s dashboard.
• Worked on Software Development Life Cycle (SDLC) from requirement analysis to Scenario base test case closure with QA team.