All remote jobs

Lead Data Engineer

AI Summary
Oddball is seeking a Lead Data Engineer to establish the technical direction for migrating legacy data warehouse assets to an Azure and Databricks lakehouse for a VA-focused federal engagement. The role owns reusable migration-pipeline patterns, reconciliation methods, quality gates, and resolution of complex performance issues. It combines hands-on expertise in Azure Data Factory, Databricks/Spark, SQL, Python, Delta Lake, and medallion architecture with engineering leadership and mentoring. The position requires close collaboration with product, QA, platform, and customer stakeholders in a remote Agile environment. Employment is contingent upon contract award and requires authorization to work in the United States.
Remote from
USA
Salary
USD 110k–145k / yr
Employment
Full Time
Experience
Senior
Published
Apply before
7 Dec 2026
Listing views
62
Application actions
3
Application toolkit

Make your next move.

Prepare your resume, explore your fit, and draft a cover letter for this opportunity.

Opportunity details

About this role.

Oddball believes that the best products are built when companies understand and value the things they are working on. We value learning and growth and the ability to make a big impact at a small company. We believe that we can make big changes happen and improve the daily lives of millions of people by bringing quality software to the federal space.

This position is contingent upon contract award.

We’re looking for a Lead Data Engineer to join our VA team, setting the technical approach for moving legacy data warehouse data and pipelines onto an Azure and Databricks lakehouse.

What you’ll be doing:

  • Set the technical approach for migrating data and pipelines onto the lakehouse
  • Guide migration teams on patterns, reconciliation, and quality gates
  • Design reusable migration pipelines using Azure Data Factory and Azure Databricks
  • Define how source and target data are reconciled to confirm accuracy
  • Resolve the hardest pipeline and performance problems
  • Mentor data engineers and review their work
  • Partner with product, QA, and platform teams

What you’ll bring:

  • Data engineering leadership on large migration or modernization programs
  • Deep experience with Databricks or Spark and Azure Data Factory
  • Strong SQL and Python skills
  • Experience moving legacy ETL and data warehouse workloads to the cloud
  • Understanding of lakehouse and medallion architecture, including Delta
  • Experience in federal or regulated data environments is a plus
  • Thrives in a remote, collaborative Agile environment and genuinely enjoys leading and mentoring engineers
  • Communicates clearly and openly, whether setting technical direction or aligning with the customer
  • Perform other related duties as assigned

Requirements:

  • Applicants must be authorized to work in the United States. In alignment with federal contract requirements, certain roles may also require U.S. citizenship and the ability to obtain and maintain a federal background investigation and/or a security clearance.

Education:

  • Bachelor’s Degree

Benefits:

  • Fully remote
  • Tech & Education Stipend
  • Comprehensive Benefits Package
  • Company Match 401(k) plan
  • Flexible PTO, Paid Holidays

Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities:

Oddball is an Equal Opportunity Employer and does not discriminate against applicants based on race, religion, color, disability, medical condition, legally protected genetic information, national origin, gender, sexual orientation, marital status, gender identity or expression, sex (including pregnancy, childbirth or related medical conditions), age, veteran status or other legally protected characteristics. Any applicant with a mental or physical disability who requires an accommodation during the application process should contact an Oddball HR representative to request such an accommodation by emailing hello@Oddball.io

The contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor’s legal duty to furnish information. 41 CFR 60-1.35(c)

At Oddball, we may use AI-assisted tools as part of our recruitment process to help review applications, verify applicant information, and identify potentially fraudulent submissions. These tools support our recruiting team in maintaining a fair and secure hiring process. AI-generated findings are reviewed by our recruiting team and are not used as the sole basis for hiring decisions.

Compensation:

At Oddball, it’s important each employee is compensated competitively and fairly. In alignment with state legal requirements. A range for the included position is listed below. Be advised, actual offer details are determined by job category, job location, and candidate skill level.

United States Wage Range: $110,000 – $145,000

Apply now >

This job listing has been manually reviewed by the Jobicy Trust & Safety Team for compliance with our posting guidelines, including verification of the company's legitimacy, accuracy of job details, clarity of remote work policy, and absence of misleading or fraudulent content.

Role DNA

A quick view of the complexity, pace, ownership and collaboration implied by the job description.

Job Complexity

5/5
EasyHard

Pace & Pressure

4/5
RelaxedFast-paced

Autonomy Level

5/5
GuidedFull ownership

Communication Load

5/5
IndependentCollaborative
AI insightThis is a senior technical leadership role involving complex legacy-data modernization, data validation, and pipeline-performance challenges. The engineer must set architecture and standards while mentoring others and aligning technical decisions with multiple stakeholders in a regulated federal context.

Salary analysis

Estimated compensation compared with the broader US market for similar roles.

Estimated job median (per year)Below market
$127,500
US market range (per year)$130k–$175k
AI insightThe yearly pay period is inferred from the job context; it was not explicitly stated by the employer. The employer discloses a United States wage range of USD 110,000–145,000. Although no cadence is explicitly stated, the annual period is inferred from the compensation scale and senior full-time US data engineering context. Estimated US market compensation for a lead data engineer with Azure, Databricks, and federal modernization experience is USD 130,000–175,000 per year.

Core skills

Skills and capabilities most closely associated with this opportunity.

Sample interview questions
How would you plan a migration from a legacy data warehouse to an Azure Databricks lakehouse?

I would begin with data-domain and workload discovery, profiling source quality, dependencies, volumes, SLAs, and security requirements. I would define target medallion layers, ingestion and transformation patterns, reconciliation criteria, phased cutover plans, and measurable acceptance gates before migrating workloads incrementally.

What reconciliation controls would you implement to prove that migrated data is accurate?

I would use record counts, primary-key completeness checks, aggregates by business dimension, checksum or hash comparisons where appropriate, null and duplicate-rate checks, and exception reporting. Controls should be automated in the pipeline and tracked over time, with clear tolerances and documented sign-off criteria for each dataset.

How do Azure Data Factory and Databricks work together in a modern data platform?

Azure Data Factory can orchestrate ingestion, scheduling, dependency management, and integrations across systems, while Databricks handles scalable Spark-based transformation, data quality logic, and Delta Lake processing. I use clear interfaces between orchestration and compute, parameterized pipelines, reusable notebooks or workflows, and centralized observability.

Describe your approach to resolving a difficult Spark pipeline performance issue.

I first use Spark UI and platform metrics to isolate skew, shuffle pressure, inefficient joins, excessive small files, or poor partitioning. I then test targeted changes such as broadcast joins, repartitioning, optimized file sizing, predicate pushdown, caching only when justified, and Delta optimization, validating both runtime improvement and output correctness.

How do you lead and mentor data engineers while maintaining technical standards?

I set practical architecture principles, code-review expectations, reusable templates, testing standards, and definition-of-done criteria. I pair these with regular design reviews, coaching through real delivery work, and a supportive feedback process so engineers gain ownership while the team maintains consistent quality.

This analysis is generated from the job description. Salary estimates, role characteristics and sample answers are guidance, not employer-provided facts.
Next step

Apply now.

Follow the employer’s application method and review Jobicy’s safety guidance before sharing personal information.

Did you apply?Let us know, and we’ll help you track your application.

Continue on the employer website

Protect your personal information and never pay to secure an interview or job offer. .

Log in to save
One quick step before you apply

Sign in to continue.

Sign in or create a free account to continue to the employer's application.

Add alert
Jobs Talent AI Tools Salaries
Menu