Cloud Data Engineering Lead - Databricks & Snowflake

DataJobs; Chicago (IL); On-site; USD 209,000 - 239,000; Full time

Benefits offered by this job

Performance-based incentive

Comprehensive benefits

Job summary

Information Technology Senior Management Forum in Chicago, IL is seeking a Lead Cloud-First Data Engineer with 6+ years of development and 3+ years in people management. You will lead agile teams, design and review data pipelines, and mentor a diverse group of engineers to deliver scalable data solutions on cloud platforms.

The role emphasizes architecture, ownership, and collaboration with product and software teams to drive robust, observable data systems using Databricks, PySpark, SQL, and

Qualifications

• Bachelor’s Degree or higher in a quantitative field.
• 5+ years of application development experience.
• 3+ years of people management experience.
• 2+ years driving technical delivery of roadmap features.
• 4+ years programming with Python/Java/Scala.
• 4+ years designing and developing data pipelines.
• 2+ years data modeling and designing end-to-end solutions.

Responsibilities

• Lead Agile teams across full lifecycle of data engineering solutions on cloud platforms.
• Influence developers, data analysts, and data scientists in ML and lakehouse architectures.
• Review architecture designs, data models, and pipeline code in Python, Spark, Databricks, Snowflake.
• Collaborate with product managers to deliver robust cloud-first data solutions.
• Establish common design patterns to improve code quality and reusability across platforms.
• Oversee health of data platforms and pipelines, setting scalability and data quality standards.
• Mentor and grow a diverse data engineering team.
• Lead end-to-end delivery of large-scale data initiatives, including evaluations between Snowflake and Databricks.

Skills

People management

Agile leadership

Mentoring

Communication

Leadership

Data engineering

Cloud architectures

Education

Bachelor’s Degree

Master’s Degree

Tools

Databricks

PySpark

Snowflake

Python

Java

Scala

Spark

Airflow

Dagster

EMR

Glue

Kubernetes

Snowflake vs Databricks

NoSQL

Redshift

Job description

Information Technology Senior Management Forum in Chicago, IL is seeking a Lead Cloud-First Data Engineer with 6+ years of development and 3+ years in people management. You will lead agile teams, design and review data pipelines, and mentor a diverse group of engineers to deliver scalable data solutions on cloud platforms.

The role emphasizes architecture, ownership, and collaboration with product and software teams to drive robust, observable data systems using Databricks, PySpark, SQL, and