Lead Data Engineer: Azure Databricks & Data Architecture

MetLife; Bridgewater (MA); On-site; USD 140,000 - 185,000; Full time

Job summary

MetLife is seeking a Lead Data Engineer to design and implement end-to-end data architectures and reusable data products. You will lead a team, build scalable ETL pipelines, and collaborate with IT, analytics, and business teams across our U.S. data platforms.

The role focuses on Azure Databricks, data governance, and cloud-based data solutions. Ideal candidates bring 8+ years in data solutions with 4+ years in Azure/Databricks, strong SQL, Spark, Python, and leadership abilities for

Qualifications

• 8+ years in data solutions and delivery, with 4+ years in Azure/Databricks.
• Proficient in SQL, Spark and Python/Scala; strong performance tuning.
• Hands-on with Big Data stack: Hive, HBase, Cosmos DB, Data Factory.
• Experience designing data warehouses for cloud analytics.
• Experience with data governance and data quality tools.
• Strong communication and leadership abilities.

Responsibilities

• Design end-to-end data architecture for data hubs and products.
• Own reusable data pipelines using Big Data and Azure Databricks.
• Lead a data engineering team and ensure data flow efficiency.
• Ingest large data volumes and develop high-performance ETL.
• Collaborate with architects, analysts, PMs and BI teams.
• Track KPIs for solution delivery and data quality.

Skills

SQL

Spark

Python

Azure Databricks

Big Data

ETL

Data warehouses

Data governance

Shell scripting

Leadership

Communication

Tools

Azure Functions

Cosmos DB

Data Factory

Databricks

Hive

HBase

Delta Lake

Job description

MetLife is seeking a Lead Data Engineer to design and implement end-to-end data architectures and reusable data products. You will lead a team, build scalable ETL pipelines, and collaborate with IT, analytics, and business teams across our U.S. data platforms.

The role focuses on Azure Databricks, data governance, and cloud-based data solutions. Ideal candidates bring 8+ years in data solutions with 4+ years in Azure/Databricks, strong SQL, Spark, Python, and leadership abilities for