Lead Data Engineer: AI-First Data Platform Lead

Nazztec group; Cary (NC); In loco; USD 180.000 - 320.000; Tempo pieno

Descrizione del lavoro

MetLife Cary, NC seeks a Lead Data Engineer – Hands-On to own the end-to-end data and AI architecture. You will mentor engineers, deliver production Python/Scala/PySpark code, and drive the AI-driven data hub across 150+ feeds and 35+ downstream systems.

You will implement Bronze/Silver/Gold contracts, CI/CD with Azure DevOps, and IaC with Terraform while optimizing Spark workloads and ensuring data governance and security.

Competenze

• Total 12–18 years in Data Engineering / Data Platform delivery.
• Hands-on with production-grade Python, Scala and PySpark in modern lakehouse environments.
• Experience building AI-first data hubs and enterprise-scale ingestion pipelines.
• Proven ability to mentor engineers and write/review production code within the last year.
• Strong security, governance and metadata capabilities across Unity Catalog and RBAC.

Mansioni

• Design and implement enterprise Lakehouse architecture with Bronze/Silver/Gold.
• Own end-to-end data and AI layer design and production code delivery.
• Lead CI/CD for Databricks and IaC with Terraform and Azure DevOps.
• Mentor teams, conduct code reviews, optimize Spark workloads and pipelines.
• Deliver production-grade AI/LLM solutions and governance frameworks.

Conoscenze

Azure Databricks

Python

Scala

PySpark

Spark

Data modeling

Data governance

Mentoring

Architecture design

Strumenti

Unity Catalog

Databricks Job/Workflows

Terraform

Azure DevOps

ADLS Gen2

LangChain

LangGraph

Databricks Model Serving

Descrizione del lavoro

MetLife Cary, NC seeks a Lead Data Engineer – Hands-On to own the end-to-end data and AI architecture. You will mentor engineers, deliver production Python/Scala/PySpark code, and drive the AI-driven data hub across 150+ feeds and 35+ downstream systems.

You will implement Bronze/Silver/Gold contracts, CI/CD with Azure DevOps, and IaC with Terraform while optimizing Spark workloads and ensuring data governance and security.