BigData DevOps Engineer

Remote from: Costa Rica
Annual salary: Undisclosed
Salary information is not provided for this position. Check our Salary Directory to estimate the average compensation for similar roles.
Department: DevOps & Infrastructure
Employment type: Full Time,
Job posted: 22 May 2026
Apply before: 22 Jun 2026
Views / Applies: 77 / 5

Experian unlocks the power of data to create opportunities for consumers, businesses and society.

Actively Hiring

AI Summary

Experian is seeking a BigData DevOps Engineer to support and administer Hadoop and AWS EMR environments. The role involves implementing CI/CD solutions, automating deployments, and ensuring security best practices. The ideal candidate has 3+ years of Hadoop administration experience and skills in Terraform, Ansible, and Linux. This position offers an opportunity to work with a global team on cutting-edge big data technologies.

Job Complexity

Easy Hard

AI Insight The role requires a blend of DevOps and big data skills, including Hadoop, AWS, and automation tools, which is challenging but typical for experienced engineers.

Salary Analysis

AI Insight The salary range is not provided, but based on market data for a BigData DevOps Engineer in the US, the median is around $145,000. This role typically commands a premium due to the specialized combination of skills.

Key Skills

Hadoop AWS EMR Terraform Ansible CI/CD Linux Spark Kafka Python DevOps

Cover Letter Sample

I am writing to express my interest in the BigData DevOps Engineer position at Experian. With over 3 years of experience in Hadoop administration and AWS EMR, I have developed a strong skill set in managing big data environments. My expertise in Terraform, Ansible, and CI/CD tools aligns well with the requirements of this role.

In my previous role, I successfully automated deployment processes and optimized cluster performance, resulting in a 20% reduction in operational costs. I am passionate about leveraging data to drive business value and am eager to contribute to Experian's innovative team.

Thank you for considering my application. I look forward to the opportunity to discuss how my skills can support your big data initiatives.

Possible Interview Questions

Describe your experience with administering Hadoop clusters in a production environment.

I have administered Hadoop clusters for 3+ years, including Cloudera and AWS EMR. I managed HDFS, YARN, and Hive, performed capacity planning, and tuned MapReduce and Spark jobs. I also handled Kerberos authentication and security policies.

How do you approach automating infrastructure provisioning using Terraform?

I write Terraform modules to define AWS resources like EMR clusters, VPCs, and IAM roles. I use state management and version control to ensure reproducibility. I also integrate Terraform with CI/CD pipelines for automated deployments.

Can you explain how you would troubleshoot a performance issue in a Spark job on EMR?

I would start by reviewing Spark UI for stage durations and shuffle metrics. Then check YARN container logs and cluster resource utilization. I might adjust executor memory, parallelism, or use data caching to optimize.

What security measures do you implement for big data platforms?

I implement Kerberos authentication, TLS encryption, and Apache Sentry for authorization. I also enforce least privilege IAM roles and regularly audit access logs. For data at rest, I enable encryption using AWS KMS.

Describe your experience with CI/CD tools like Jenkins and how you use them for big data deployments.

I have used Jenkins to automate build and deployment pipelines for big data applications. I set up jobs to run tests, build Docker images, and deploy to EMR using scripts. I also integrate with Git for version control.

Company Description

Experian is a global data and technology company, powering opportunities for people and businesses around the world. We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more. Experian invests in people and new advanced technologies to unlock the power of data. We have an amazing team of 25,200 people in 32 countries

Job Description

As an important aide to both the IT Infrastructure and Development teams, you will help support existing systems 24×7 and responsible for administering current Big Data environments. You will manage BigData Group environments and will work with teammates to maintain working solutions for our big data tech stack. To support product development following the product roadmap for product maintenance and enhancement such that the quality of software deliverables maintains excellent customer relationships and increases the customer base. You will report to the Engineering Manager, Data Platforms.

If you have the skills and we would love to talk to you!

What you’ll do:

Implement, support, and administer Hadoop and AWS EMR environments.
Maintain Terraform and IaC for provisioning and managing AWS resources.
Provide CI/CD solutions.
Automate deployment and operations of Big Data technologies using DevOps tools.
Onboard Hadoop users and manage Kerberos, HDFS, Hive, HBase, and Yarn access.
Implement security best practices across HBase, HDFS, Kafka, Hive, and related components.
Tune Hadoop clusters, MapReduce, Spark workloads, and optimize EMR for performance and cost.
Monitor group health, logs, storage, and perform proactive capacity management.
Collaborate with infrastructure, network, DB, application, and BI teams to ensure platform reliability.
Support upgrades and patching for EMR, HBase, Spark, and other data platforms.
Troubleshoot system issues and analyze CPU, memory, OS, storage, and network performance.
Deploy and manage Hadoop clusters, configure HA, manage nodes, schedule jobs, and perform backups.
Support integrations across cloud/on‑prem networks and data platforms.
Configure and maintain tools like Sentry, Spark, Kafka, Oozie, Solr, MongoDB, DocumentDB, and ELK.
Integrate AD/LDAP with Cloudera, manage Sentry policies.

Qualifications

Typically requires a bachelor’s degree (in Computer Science or related field) or equivalent.
3+ years of Hadoop infrastructure administration – AWS EMR or Cloudera environments
2+ years of Linux (Redhat) system administration (Optional)
Cloud Platforms IaaS/PaaS – Cloud solutions: AWS, Azure, VMWare.
Gen AI / Agentic AI knowledge is plus
Automation skills – Ansible and Terraform.
Kerberos administration skills
Experience coding languages SQL, Python
Good experience in CI/CD tools such as Jenkins, Puppet and Shell scripting
Must have knowledge on DevOps tools.
Working Knowledge of YARN, HBase, Hive, Spark, and Kafka.
Experience working with geographically distributed teams
Bachelors or master’s degree in Computer Science or equivalent experience
Knowledge of our strategy and use of back-office applications.
To multi-lingual and multicultural environment, additional language skills are a bonus.
Receptive to change.
With our users at all levels.

Additional Information

Our uniqueness is that we celebrate yours. Experian’s people first, inclusive and purpose driven culture is multi award-winning; World’s Best Workplaces™ 2025 (Fortune Global Top 25), Great Place To Work™ in 26 countries to name a few. Check out Experian Life on social or explore our Careers Site to understand why.

Experian is proud to be an Equal Opportunity Employer for all groups protected under applicable federal, state and local law, including protected veterans and individuals with disabilities. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.

Benefits/Perks:

Our benefits include: Medical, life and dental insurance, Asociacion Solidarista, International Share Save Plan, Flex Work/Work from home, Paid time off, Annual Performance Bonus, Education Reimbursement, Family Bonding, Bereavement Leave, Referral Program, and more.