I am a Principal Data Engineer with over 9+ years of experience designing, building, and optimizing data platforms, data warehouses, and ETL pipelines. My expertise spans working with technologies such as Apache Kafka, Apache NiFi, Spark, Airflow, ELK Stack, Docker, Kubernetes, and cloud platforms including AWS, GCP, and Azure. I specialize in leading large-scale data integration and warehousing projects for global organizations, focusing on creating scalable and reliable data solutions.
Throughout my career, I have developed Python-based automation frameworks and real-time streaming pipelines that have significantly improved data processing efficiency and supported informed decision-making. I have a proven track record of leading and mentoring cross-functional teams to deliver actionable insights and drive data-driven strategies.
In my current role as Principal Data Engineer at MicroDev Solutions, I lead a team of five data engineers in designing and maintaining enterprise-scale data pipelines and infrastructure across AWS and Azure. I have engineered containerized, Docker- and Kubernetes-based data pipelines using Kafka and Python microservices, reducing processing time by 40%. I also designed scalable data architectures supporting real-time analytics across multiple business units.
Previously, as a Senior Data Engineer at TechnoGenics SMC, I designed and optimized large-scale ETL/ELT pipelines using Spark and Airflow, migrated legacy systems to Snowflake and AWS S3, and created Power BI dashboards to provide actionable business insights. I have experience integrating processed datasets into Amazon Redshift and implementing data validation and governance frameworks.
Earlier in my career, I worked as a Software Engineer at Ebryx Pvt. Ltd., where I modernized legacy ETL pipelines and data warehouse workflows, achieving a 30% improvement in processing speed. I implemented solutions using Python, Kafka, Elasticsearch, FluentD, and GCP, supporting large-scale daily data ingestion. I also contributed to migrating on-prem Oracle systems to AWS Redshift, improving scalability and reporting performance.
I hold an MS in Data Science for Business from the University of Stirling and a BS in Computer Science from FAST, National University. I was awarded a Gold Medal for exceptional academic performance. I am passionate about building scalable data architectures, automating data workflows, and enabling data-driven decision-making to empower organizations.
