Lead Data Engineer: Build Data Products on Databricks

Protective; Birmingham (AL); On-site; USD 140,000 - 170,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

401(k) with company match

Paid time off

Job summary

Protective is hiring a Lead Data Engineer to set the technical direction for a data-pod building products on Voyager, our Databricks lakehouse on Azure. You will own the end-to-end data product design, including ingestion, cleansing, modeling, and the Gold consumption layer, while writing and reviewing production code daily.

You will define data contracts, enforce compatibility, and set engineering standards across Python, SQL, and CI gates, partnering with ML and platform teams to ensure

Qualifications

• 6+ years building and operating production data pipelines and consumer-facing data models.
• Strong Python and SQL, ability to write and review production code.
• Hands-on Databricks or Spark-based lakehouse experience with Delta Lake.
• Deep dimensional modeling: grain, keys, slowly changing dimensions, facts.
• Demonstrated technical leadership and setting engineering standards.
• Experience owning data that other teams depend on and handling breaking changes.

Responsibilities

• Lead the design of data products end to end, including ingestion, cleansing, conformance, and Gold layer.
• Own dimensional design and conformed dimensions for cross-pod reuse.
• Define data contracts and enforce compatibility with consumers.
• Set engineering standards for Python, SQL, dbt, testing, and CI gates.
• Lead code review and ensure pipeline health with observability and SLAs.

Skills

Python

SQL

Technical leadership

Data modeling

Communicating design choices

Problem solving

Education

Bachelor’s degree in CS/IS/Engineering

Tools

Databricks

Delta Lake

MERGE

Dagster

Airflow

Azure DevOps

Job description

Protective is hiring a Lead Data Engineer to set the technical direction for a data-pod building products on Voyager, our Databricks lakehouse on Azure. You will own the end-to-end data product design, including ingestion, cleansing, modeling, and the Gold consumption layer, while writing and reviewing production code daily.

You will define data contracts, enforce compatibility, and set engineering standards across Python, SQL, and CI gates, partnering with ML and platform teams to ensure