Azure Data Engineering Lead - Databricks & ETL Architect

Statestreet; Quincy (MA); Hybrid; USD 120,000 - 203,000; Full time

Job summary

State Street is looking for a Data Engineer Lead to design, build, and optimize data pipelines in Azure, enabling centralized lakes and Databricks Delta Lake. You will collaborate with analysts, software engineers, and operations to ensure data quality, reliability, and security.

The role requires 8+ years in data engineering, strong SQL and programming skills (Scala/Java), ETL/ELT expertise, and cloud experience with Azure.

Qualifications

• Bachelor’s degree in computer/IT field.
• 10+ years in IT industry.
• 8+ years in Bigdata data pipelines.
• 8+ years as a Data Engineer on Azure Databricks.
• 8+ years cloud dev (Azure, DevOps, Kubernetes, Docker).
• Snowflake experience is a plus.

Responsibilities

• Design, develop, and maintain Azure data pipelines for ingesting, transforming, and loading data.
• Implement ELT/ETL processes to ensure data quality and reliability.
• Design data models and database schemas for structured and unstructured data.
• Optimize storage, access, and performance of data.
• Collaborate with cross-functional teams to translate data requirements into solutions.
• Monitor pipelines, troubleshoot issues, and ensure data security and governance.
• Identify bottlenecks and optimize queries and infrastructure.
• Implement data security measures and privacy compliance.
• Document data flows and configurations.

Skills

Data engineering

ETL/ELT development

SQL proficiency

Programming in Scala/Java

Data modeling

Cross-functional collaboration

Education

Bachelor’s Degree in CS/IT

10+ years IT experience

8+ years Bigdata pipelines

8+ years Data Engineer with Azure Databricks

8+ years cloud development (Azure, DevOps, Kubernetes, Docker)

Snowflake experience (plus)

Tools

Azure Synapse Analytics

Azure Data Factory

Azure Databricks

Azure SQL Database

Snowflake

Job description

State Street is looking for a Data Engineer Lead to design, build, and optimize data pipelines in Azure, enabling centralized lakes and Databricks Delta Lake. You will collaborate with analysts, software engineers, and operations to ensure data quality, reliability, and security.

The role requires 8+ years in data engineering, strong SQL and programming skills (Scala/Java), ETL/ELT expertise, and cloud experience with Azure.