Lead Data Engineer - Azure Databricks Lakehouse

Devconnectplatform; United States; Remote; USD 110,000 - 145,000; Full time

Benefits offered by this job

Fully remote

Job summary

Oddball is seeking a Lead Data Engineer to set the technical approach for migrating legacy data warehouse data and pipelines onto an Azure Databricks lakehouse. You will guide migration teams, design reusable pipelines, and mentor engineers in a remote, Agile environment.

Requirements include leadership on large migration programs, deep Databricks/Spark and Azure Data Factory experience, and strong SQL/Python skills.

Qualifications

• Bachelor's degree in a related field.
• Experience leading data migration/modernization programs.
• Deep experience with Databricks or Spark and Azure Data Factory.
• Strong SQL and Python skills.

Responsibilities

• Set the technical approach for migrating data and pipelines onto the lakehouse.
• Guide migration teams on patterns, reconciliation, and quality gates.
• Design reusable migration pipelines using Azure Data Factory and Azure Databricks.
• Define how source and target data are reconciled to confirm accuracy.
• Mentor data engineers and review their work.

Skills

SQL

Python

Databricks

Spark

Leadership

Education

Bachelor's Degree

Tools

Azure Data Factory

Azure Databricks

Delta Lake

Job description

Oddball is seeking a Lead Data Engineer to set the technical approach for migrating legacy data warehouse data and pipelines onto an Azure Databricks lakehouse. You will guide migration teams, design reusable pipelines, and mentor engineers in a remote, Agile environment.

Requirements include leadership on large migration programs, deep Databricks/Spark and Azure Data Factory experience, and strong SQL/Python skills.