Hands-On Lead Data Engineer — AI-Driven Lakehouse

SIDRAM TECHNOLOGIES; Cary (NC); Hybrid; USD 160,000 - 230,000; Full time

Job summary

Altimetrik is hiring a Lead Data Engineer for MetLife in Cary, NC. This hands-on leadership role requires deep expertise in building and scaling a lakehouse data platform on Azure Databricks, with production-grade PySpark and Scala code.

You will own end-to-end data architecture, AI-augmented ingestion, data quality, and governance. Expect mentoring, code reviews, and delivering reusable components for the client's data ecosystem.

Qualifications

• Expert-level proficiency in Python, Scala, and PySpark with production-grade code experience.
• Strong SQL, data modeling, and experience delivering lakehouse architectures.
• Databricks Delta Lake, Unity Catalog, and Databricks Jobs/Workflows expertise.
• Azure data stack including ADLS Gen2, Data Factory, and Event Hubs.
• Experience designing and reviewing scalable data pipelines and CI/CD for data platforms.

Responsibilities

• Own end-to-end data platform architecture for lakehouse (Bronze/Silver/Gold).
• Design and implement CI/CD pipelines for Databricks and ADF using Azure DevOps.
• Lead AI-augmented ingestion, canonical mapping, and data quality initiatives.
• Build a semantic layer and knowledge-graph interfaces for data discovery.
• Mentor engineers, conduct design reviews, and establish engineering patterns.

Skills

Python

Scala

PySpark

Databricks

Azure

Tools

Delta Lake

Unity Catalog

Azure Data Factory

ADLS Gen2

Job description

Altimetrik is hiring a Lead Data Engineer for MetLife in Cary, NC. This hands-on leadership role requires deep expertise in building and scaling a lakehouse data platform on Azure Databricks, with production-grade PySpark and Scala code.

You will own end-to-end data architecture, AI-augmented ingestion, data quality, and governance. Expect mentoring, code reviews, and delivering reusable components for the client's data ecosystem.