VP Data Engineering Lead: Azure Pipelines & Data Lakes

State Street; Quincy (MA); Hybrid; USD 120,000 - 203,000; Full time

Benefits offered by this job

401K with company match

Insurance coverage (medical, dental, &

Job summary

State Street is seeking an experienced Data Engineer to design, develop, and optimize data pipelines in Azure, leveraging Azure Synapse, Data Factory, and Databricks to ingest, transform, and store data across data lakes and warehouses. The role emphasizes data quality, governance, and secure data handling.

You will collaborate with cross-functional teams including data scientists and software engineers to deliver scalable data solutions, with onsite 4 days a week and one day remote, aligning

Qualifications

• Bachelor’s Degree in computer or IT related subject.
• 8+ years of Big Data/Data Pipeline experience.
• 8+ years designing data solutions on Azure Databricks.
• 8+ years cloud development including Azure services, Azure DevOps, Kubernetes, Docker.

Responsibilities

• Design, develop, and maintain data pipelines in Azure for ingesting, transforming, and loading data into centralized Azure data lakes and Databricks Delta Lake.
• Implement ELT/ETL processes to ensure data quality and reliability across sources.
• Design data models and schemas to support structured and unstructured data with scalable storage and access.
• Use modern data processing principles to optimize ingestion and transformation processes.
• Collaborate with data scientists, analysts, and engineers to translate data requirements into technical solutions.
• Monitor pipelines, troubleshoot issues, and enforce data quality and security controls.
• Define and enforce data governance standards and policies.
• Identify performance bottlenecks and optimize queries, data structures, and infrastructure.

Skills

Azure data pipelines

SQL

Scala/Java

Data modeling

ETL/ELT

Databricks

Microsoft Azure services

Education

Bachelor’s Degree in Computer/IT

Tools

Snowflake

SQL Server

Oracle

Git

Kubernetes

Docker

Job description

State Street is seeking an experienced Data Engineer to design, develop, and optimize data pipelines in Azure, leveraging Azure Synapse, Data Factory, and Databricks to ingest, transform, and store data across data lakes and warehouses. The role emphasizes data quality, governance, and secure data handling.

You will collaborate with cross-functional teams including data scientists and software engineers to deliver scalable data solutions, with onsite 4 days a week and one day remote, aligning