Senior Manager Data Engineer

DataJobs; Chicago (IL); On-site; USD 209,000 - 239,000; Full time

Benefits offered by this job

Performance-based incentive

Comprehensive benefits

Job summary

Information Technology Senior Management Forum in Chicago, IL is seeking a Lead Cloud-First Data Engineer with 6+ years of development and 3+ years in people management. You will lead agile teams, design and review data pipelines, and mentor a diverse group of engineers to deliver scalable data solutions on cloud platforms.

The role emphasizes architecture, ownership, and collaboration with product and software teams to drive robust, observable data systems using Databricks, PySpark, SQL, and

Qualifications

• Bachelor’s Degree or higher in a quantitative field.
• 5+ years of application development experience.
• 3+ years of people management experience.
• 2+ years driving technical delivery of roadmap features.
• 4+ years programming with Python/Java/Scala.
• 4+ years designing and developing data pipelines.
• 2+ years data modeling and designing end-to-end solutions.

Responsibilities

• Lead Agile teams across full lifecycle of data engineering solutions on cloud platforms.
• Influence developers, data analysts, and data scientists in ML and lakehouse architectures.
• Review architecture designs, data models, and pipeline code in Python, Spark, Databricks, Snowflake.
• Collaborate with product managers to deliver robust cloud-first data solutions.
• Establish common design patterns to improve code quality and reusability across platforms.
• Oversee health of data platforms and pipelines, setting scalability and data quality standards.
• Mentor and grow a diverse data engineering team.
• Lead end-to-end delivery of large-scale data initiatives, including evaluations between Snowflake and Databricks.

Skills

People management

Agile leadership

Mentoring

Communication

Leadership

Data engineering

Cloud architectures

Education

Bachelor’s Degree

Master’s Degree

Tools

Databricks

PySpark

Snowflake

Python

Java

Scala

Spark

Airflow

Dagster

EMR

Glue

Kubernetes

Snowflake vs Databricks

NoSQL

Redshift

Job description

Lead cloud-first data engineering at Information Technology Senior Management Forum in Chicago, IL (onsite). This Senior Manager Data Engineer role brings people together to design, develop, test, and support scalable platforms using modern tools such as Databricks, PySpark, and Snowflake, while helping drive data solutions that support financial empowerment for millions of Americans.

You will also have room to grow through hands-on technical involvement, mentoring, and active participation in internal and external technology communities. The position combines leadership, architecture, and delivery accountability, with an emphasis on scalable, resilient pipelines and strong engineering practices.

What you’ll do

• Lead high-performing Agile teams across the full lifecycle of scalable data engineering solutions on full-stack and cloud platforms
• Influence developers, data analysts, and data scientists with expertise across machine learning, distributed microservices, and lakehouse architecture
• Stay hands-on by reviewing architecture designs, data models, and pipeline code in Python, Spark, Databricks, and Snowflake environments
• Collaborate with product managers and software engineers to deliver robust cloud-first data solutions
• Establish and enforce common design patterns to improve code quality, maintainability, and reusability across data platforms and pipelines
• Oversee the health of data platforms and pipelines, setting standards for scalability, resilience, data quality, and operational efficiency
• Serve as an ambassador for the data engineering team by communicating technical concepts and data outcomes clearly to internal and external stakeholders
• Lead, mentor, and grow a diverse team of data engineers while driving best practices and technical career development
• Lead end-to-end delivery of large-scale, transformative data initiatives, including independent architectural decisions such as evaluating Snowflake versus Databricks based on technical and business requirements

Required qualifications

• Bachelor’s Degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
• 6+ years of experience in application development (internship experience does not apply)
• 3+ years of people management experience
• 2+ years driving technical delivery of roadmap features
• 4+ years programming with at least one of: Python, Java, or Scala
• 4+ years designing and developing data pipelines
• 2+ years data modeling and designing end-to-end solutions using both relational and non-relational database systems

Technologies you’ll work with

• Databricks, PySpark, Snowflake, Spark
• Python, Java, Scala
• Machine learning, distributed microservices, lakehouse architecture
• AWS, Microsoft Azure, Google Cloud
• EMR, Glue, Airflow, Dagster
• Monte Carlo, Splunk
• MongoDB, Cassandra, DynamoDB, Redshift, NoSQL, SQL

Benefits

• Performance-based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
• Comprehensive, competitive, and inclusive health, financial, and other benefits supporting total well-being

Preferred qualifications

• Master’s Degree in Computer Science or a related field
• 9+ years of application development with demonstrated proficiency in Python, SQL, Scala, or Java
• 5+ years hands-on experience designing, deploying, and operating data workloads in at least one public cloud environment (AWS, Microsoft Azure, or Google Cloud)
• 5+ years experience building or supporting distributed data or compute workloads using tools such as EMR, Spark, Glue, or Databricks
• 5+ years experience designing, implementing, and operating real-time or streaming data pipelines
• 3+ years experience in data observability (e.g., Monte Carlo, Splunk) or data orchestration tools (e.g., Airflow, Dagster)
• 5+ years experience working with unstructured or semistructured data using NoSQL databases (e.g., MongoDB, Cassandra, DynamoDB)
• 5+ years experience designing and supporting data warehousing solutions (e.g., Snowflake, Redshift)
• 3+ years of experience working in an Agile development environment
• 3+ years of experience developing user-centric reusable data products

Salary: Chicago, IL: $209,000 - $238,500 per year.