Senior Data Engineering Lead - Azure & Databricks

State Street; Quincy (CA); Hybrid; USD 120,000 - 203,000; Full time

Benefits offered by this job

401K match

Health insurance

Paid time off

EAP

Incentive pay

Savings plans

Job summary

State Street is seeking a Data Engineer Lead to develop, maintain, and optimize data pipelines and systems in Azure Databricks and Synapse. You will work closely with analysts, software engineers, and operations teams to ensure data availability, integrity, and security across centralized data lakes and data warehouses.

The role emphasizes migration from on-prem to cloud, scalable data models, and robust data governance.

Qualifications

• Bachelor’s degree in a computer/IT field or equivalent.

Responsibilities

• Design, develop, and maintain data pipelines in Azure for ingestion, transformation, and loading into data lakes.
• Implement ELT/ETL processes to ensure data quality and consistency.
• Model and optimize database schemas for structured and unstructured data.
• Collaborate with data scientists, analysts, and engineers to meet data requirements.
• Monitor pipelines, troubleshoot issues, and ensure data security and governance.
• Identify bottlenecks and optimize queries and infrastructure for performance.
• Ensure compliance with data privacy regulations and implement security measures.

Skills

Data engineering

ETL/ELT

SQL

Programming

Education

Bachelor’s degree in Computer Science/IT

Tools

Azure Synapse Analytics

Azure Data Factory

Azure Databricks

Azure SQL Database

Snowflake

Kubernetes

Docker

Git

Job description

State Street is seeking a Data Engineer Lead to develop, maintain, and optimize data pipelines and systems in Azure Databricks and Synapse. You will work closely with analysts, software engineers, and operations teams to ensure data availability, integrity, and security across centralized data lakes and data warehouses.

The role emphasizes migration from on-prem to cloud, scalable data models, and robust data governance.