Senior Lead Data Engineer - AI-First Lakehouse on Azure

Brickstech; Cary (NC); Hybrid; USD 150,000 - 185,000; Full time

Job summary

Brickstech seeks a true hands-on Lead Data Engineer to own the end-to-end data and AI architecture for an enterprise lakehouse on Azure Databricks. You will guide design, review production code, optimize Spark workloads, and establish engineering guardrails.

The role requires 12–18 years of data engineering leadership, strong Python/Scala skills, and expertise in Delta Lake, Unity Catalog, and modern data QA practices. Hybrid onsite in Cary, NC with collaboration across teams.

Qualifications

• 12–18 years in data engineering or enterprise data-platform delivery.
• Proven experience delivering enterprise-scale medallion/lakehouse architectures.
• Expert-level hands-on experience with Python, Scala, and PySpark.
• Strong SQL and data modeling skills.
• Experience with Azure Databricks and Delta Lake.
• Ability to lead engineers and establish standards.

Responsibilities

• Own enterprise lakehouse architecture and data contracts.
• Design scalable, production-grade data pipelines for large workloads.
• Lead ingestion, streaming and metadata-driven frameworks.
• Mentor engineers, review code, and enforce testing standards.
• Drive CI/CD and observability for Databricks and Azure Data Factory.

Skills

Azure Databricks

Python

Scala

PySpark

Data architecture

Production pipelines

Spark tuning

CI/CD pipelines

Leadership

Mentoring

Education

Bachelor's degree in CS or related field

Tools

Azure Data Factory

Databricks Unity Catalog

Delta Lake

Kafka

Event Hubs

Spark Structured Streaming

Terraform

Azure DevOps

Job description

Brickstech seeks a true hands-on Lead Data Engineer to own the end-to-end data and AI architecture for an enterprise lakehouse on Azure Databricks. You will guide design, review production code, optimize Spark workloads, and establish engineering guardrails.

The role requires 12–18 years of data engineering leadership, strong Python/Scala skills, and expertise in Delta Lake, Unity Catalog, and modern data QA practices. Hybrid onsite in Cary, NC with collaboration across teams.