Senior Data Engineering Lead – Azure & Databricks

STATE STREET CORPORATION; Quincy (MA); On-site; USD 120,000 - 203,000; Full time

Benefits offered by this job

401K with company match

Paid time off

Volunteer days

Job summary

State Street Corporation is seeking a Data Engineer Lead to design, build, and optimize Azure-based data pipelines that ingest, transform, and load data into centralized data lakes and Databricks Delta Lake. You will drive data quality, security, and governance while collaborating with cross-functional teams to meet business needs.

The role emphasizes on-prem-to-cloud migration, Azure services, and scalable data architecture with a focus on reliability and performance.

Qualifications

• Bachelor’s degree in computer/IT field.
• 10+ years IT industry experience.
• 8+ years Big Data pipeline experience.
• 8+ years Data Engineer experience focused on Azure Databricks.

Responsibilities

• Design, develop, and maintain data pipelines in Azure.
• Ingest, transform, and load data into centralized data lakes and Databricks.
• Ensure data quality and integrity across processes.
• Implement ETL/ELT to ensure data reliability.
• Collaborate with analysts, software engineers, and operations teams.
• Monitor pipelines and ensure data security and governance.

Skills

Azure

Databricks

SQL

Scala/Java

Azure Synapse

Data Modeling

Data Governance

Problem Solving

Education

Bachelor’s Degree in Computer/IT

10+ years IT experience

8+ years Big Data pipeline experience

Azure cloud development

Tools

Azure Data Factory

Azure Databricks

Azure SQL Database

Azure Synapse Analytics

Snowflake

Kubernetes

Docker

Git

Azure DevOps

Databricks Delta Lake

Job description

State Street Corporation is seeking a Data Engineer Lead to design, build, and optimize Azure-based data pipelines that ingest, transform, and load data into centralized data lakes and Databricks Delta Lake. You will drive data quality, security, and governance while collaborating with cross-functional teams to meet business needs.

The role emphasizes on-prem-to-cloud migration, Azure services, and scalable data architecture with a focus on reliability and performance.