Data Engineer II

Milwaukee Succeeds; Milwaukee (WI); Hybrid; USD 89,000 - 134,000; Full time

Benefits offered by this job

Flexible work schedules

Concierge service

Comprehensive benefits

Employee resource groups

Job summary

Northwestern Mutual is seeking a data engineer to design and maintain scalable ELT pipelines powering analytics and reporting. You will work with Python, Spark, Databricks, and orchestration tools to deliver reliable data products for Power BI and SSRS.

Join a team focused on production reliability, data quality, observability, and continuous improvement, partnering across business and technology to transform complex data needs into usable data products.

Qualifications

• 2-5 years of professional experience in data or software engineering with production systems.
• Strong Python programming skills and distributed processing with PySpark or Spark.
• Solid SQL skills and experience with cloud data platforms (Snowflake, Databricks, or equivalents).
• Experience with Power BI data modeling, datasets, and report troubleshooting.
• Experience with orchestration tools (Airflow, Control-M) and CI/CD pipelines.
• Familiarity with event streaming systems (Kafka, SQS) and batch/stream integration.
• Proficient with Git and DevOps practices; able to read logs.
• Ability to translate business needs into reliable data solutions.

Responsibilities

• Design, implement, and maintain ELT pipelines and transformations using Python/PySpark/SQL.
• Own production reliability: monitor SLAs, respond to incidents, notebooks, and reduce MTTR.
• Implement and maintain CI/CD pipelines to automate addition and modification of custom stages in ELT workflows.
• Orchestrate workflows with Airflow, Control M (or equivalent); handle retries, backfills, and schema changes.
• Build and maintain streaming integrations (Kafka/Kinesis or equivalent) for nearreal-time use cases.
• Implement and maintain data quality, observability and auditing.
• Optimize pipelines for cost and performance on cloud platforms (Databricks, Snowflake, or similar).
• Participate in code reviews, design discussions, and documentation; mentor junior engineers.
• Identify root causes of data issues and implement robust fixes and preventative measures.
• Collaborate with stakeholders to translate business requirements into reliable data solutions.

Skills

Consulting

Data Architecture

Data ETL

Data Security

Databases & Data Platforms

DevOps

Engineering Practices

Software Documentation

Programming Languages

Agile Methodology

Root Cause Analysis

Software Engineering

Application Platforms

Data Quality

Tools

Python

PySpark

Spark

Databricks

SQL

Airflow

Control-M

Git

CI/CD

Kafka

Power BI

Job description

Our dev/ops team builds and supports the data engineering solutions that power analytics, reporting, and operational decision-making across the business and Field. We develop and maintain scalable ELT pipelines using tools like Python, Spark, Databricks, and orchestration frameworks to deliver reliable, high-quality data for downstream consumers, including Power BI and SSRS reporting solutions. We focus on production reliability, data quality, observability, and continuous improvement, and we partner closely with business and technology teams to turn complex data needs into usable data products.

What You’ll do

Design, implement, and maintain ELT pipelines and transformations using Python/PySpark/SQL.

Own production reliability: monitor SLAs, respond to incidents, notebooks, and reduce MTTR.

Implement and maintain CI/CD pipelines to automate addition and modification of custom stages in ELT workflows.

Orchestrate workflows with Airflow, Control M (or equivalent); handle retries, backfills, and schema changes.

Build and maintain streaming integrations (Kafka/Kinesis or equivalent) for nearrealtime use cases.

Implement and maintain data quality, observability and auditing (e.g., DQX, data expectations, pydat, custom checks).

Optimize pipelines for cost and performance on cloud platforms (Databricks, Snowflake, or similar).

Participate in code reviews, design discussions, and documentation; mentor junior engineers.

Identify root causes of data issues and implement robust fixes and preventative measures.

Collaborate with stakeholders to translate business requirements into reliable data solutions.

What You’ll Bring to the Role

2-5 years professional experience in data engineering or software engineering with production systems.

Strong Python programming skills and experience with distributed processing (PySpark or Scala + Spark).

Solid SQL skills and experience with at least one cloud data platform (Snowflake, Databricks, or equivalent).

Working knowledge of Power BI, including building datasets and reports, data modeling, and troubleshooting dataset refreshes and performance issues.

Experience with orchestration tools (Airflow, Control-M, or equivalent) and CI/CD pipelines.

Familiarity with event streaming systems (Kafka, SQS, or similar) and batch/stream integration patterns.

Experience implementing data quality checks, testing data pipelines, and handling schema changes.

Proficient with Git and modern DevOps practices; comfortable reading and troubleshooting logs.

Has or develops understanding of 1-3 subject areas/domains of data.

Good communicator; able to explain technical solutions to peers and stakeholders; self-directed.

Preferred Qualifications

Familiarity with Databricks ELT architecture (Declarative Automation Bundles, Delta Lake, Jobs/Workflows, and Delta Live Tables).

Experience with Terraform, Kubernetes, Docker, or other IaC/container tooling.

Familiarity with observability/monitoring tools (Grafana, Dynatrace, etc.) and alerting best practices.

Experience with data quality frameworks (DQX, Deequ) and data governance/catalog tools.

Domain knowledge in one or more business areas outside of the Field domain (e.g., Client, Product, etc).

Skills You’ll Have

Consulting (NM): Partners with stakeholders to gather requirements, solve problems, and develop effective technical solutions.

Data Architecture (NM): Designs and maintains scalable data structures, integration frameworks, and data management solutions.

Data ETL (NM): Develops and supports data extraction, transformation, and loading processes across multiple systems and platforms.

Data Security (NM): Protects sensitive data and ensures compliance with governance, privacy, and regulatory requirements.

Databases & Data Platforms (NM): Utilizes databases and modern data platforms to store, manage, and access enterprise data.

DevOps (NM): Applies CI/CD practices and automation to improve delivery speed, reliability, and quality.

Engineering Expertise & Practices (NM): Applies engineering principles and technical best practices to deliver reliable and scalable data solutions.

Compensation Range

Pay Range - Start: $89,360.00

Pay Range - End: $134,040.00

Geographic Specific Pay Structure

Structure 110: $98,320.00 USD - $147,480.00 USD

Structure 115: $102,800.00 USD - $154,200.00 USD

We believe in fairness and transparency. It’s why we share the salary range for most of our roles. However, final salaries are based on a number of factors, including the skills and experience of the candidate; the current market; location of the candidate; and other factors uncovered in the hiring process. The standard pay structure is listed but if you’re living in California, New York City or other eligible location, geographic specific pay structures, compensation and benefits could be applicable.

About the Role

Our dev/ops team builds and supports the data engineering solutions that power analytics, reporting, and operational decision-making across the business and Field. We develop and maintain scalable ELT pipelines using tools like Python, Spark, Databricks, and orchestration frameworks to deliver reliable, high-quality data for downstream consumers, including Power BI and SSRS reporting solutions. We focus on production reliability, data quality, observability, and continuous improvement, and we partner closely with business and technology teams to turn complex data needs into usable data products.

Skills

• Strategic Thinking
• Data Auditing
• Engineering Practices
• Data Integrity
• Technical Communication
• DevOps
• Written Communication
• Software Documentation
• Programming Languages
• Agile Methodology
• Root Cause Analysis
• Software Engineering
• Application Platforms

Find Your Future

We’re excited about the potential people bring to Northwestern Mutual. You can grow your career here while enjoying first-class perks, benefits, and our commitment to a culture of belonging.

• Flexible work schedules
• Concierge service
• Comprehensive benefits
• Employee resource groups

EEO/AA Employer/Vets/Disability

Northwestern Mutual is an equal opportunity employer that welcomes talented individuals of all backgrounds. We are committed to creating and maintaining an environment in which each employee can contribute creative ideas, seek challenges, assume leadership and continue to focus on meeting and exceeding business and personal objectives.