Azure Data Engineer: Lakehouse & Pipelines Lead

Northern Trust Corp; Chicago, Northern (IL, KY); Hybrid; USD 115,000 - 195,000; Full time

Benefits offered by this job

retirement benefits (401k and pension)

health and welfare benefits (medical,

paid time off

parental and caregiver leave

life & accident insurance

other voluntary and well-being

Job summary

Northern Trust is seeking an experienced Azure Data Engineer to design, build, and operate secure, scalable cloud data solutions. You will work within the Corporate Architecture team to translate business requirements into production-ready data products for analytics and reporting.

You will leverage Azure Data Factory, Databricks, Data Lake Gen2, Spark, Python, and SQL to implement lakehouse and data warehouse patterns, with strong emphasis on data governance and CI/CD practices.

Qualifications

• Proven experience designing enterprise-grade data pipelines and analytics platforms in Azure.
• Hands-on with Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, and Delta Lake.
• Advanced Python, PySpark, and SQL skills including performance tuning.
• Strong knowledge of ETL/ELT, distributed data processing, orchestration, and schema evolution.
• Experience modeling relational, dimensional, and lakehouse structures (star schemas, conformed dimensions).
• Experience with relational, analytical, columnar, and NoSQL platforms (Oracle, PostgreSQL, Azure SQL, Snowflake).
• Experience with Git-based development, automated testing, and CI/CD using Azure DevOps, GitHub Actions, Jenkins, or equivalent.

Responsibilities

• Partner with business, architecture, analytics, security, and engineering stakeholders to translate business needs into data requirements.
• Design scalable batch and near-real-time data pipelines for structured and unstructured data.
• Build cloud-native Azure data solutions using ADP, Databricks, Data Lake Gen2, Spark, Python, SQL.
• Develop lakehouse and data warehouse patterns with Bronze, Silver, and Gold layers and appropriate modeling.
• Champion modern data modeling and data quality practices.
• Create reusable, metadata-driven pipelines and tests in collaboration with QA teams.
• Implement CI/CD and IaC for repeatable deployments across environments.

Skills

Azure Data Factory

Azure Databricks

Azure Data Lake Storage Gen2

Delta Lake

Python

PySpark

SQL

ETL/ELT patterns

Data modeling (lakehouse)

CI/CD

Git-based development

Tools

Azure DevOps

GitHub Actions

Jenkins

Terraform

Bicep / ARM templates

Job description

Northern Trust is seeking an experienced Azure Data Engineer to design, build, and operate secure, scalable cloud data solutions. You will work within the Corporate Architecture team to translate business requirements into production-ready data products for analytics and reporting.

You will leverage Azure Data Factory, Databricks, Data Lake Gen2, Spark, Python, and SQL to implement lakehouse and data warehouse patterns, with strong emphasis on data governance and CI/CD practices.