Senior Data Engineering Lead

Allstate; United States; Remote; USD 120,000 - 194,000; Full time

Job summary

Allstate seeks two Managing Engineers, Data Engineering to lead the design, development, and delivery of scalable data solutions that support digital products, analytics, and business decision-making. You will mentor data engineers, drive best practices, and partner with product and analytics teams to deliver reliable data pipelines and platforms.

The roles require hands-on coding with Python/Java/Scala, Spark, Kafka, dbt, and Docker/Kubernetes, plus strong SQL/NoSQL skills and data governance

Qualifications

• 5+ years designing, developing, and supporting data solutions or enterprise software.
• 1+ year of technical leadership, mentorship, or project leadership.
• Hands-on with Python/Java/Scala for data pipelines.
• Experience with batch or streaming data solutions.
• Experience with Spark, Kafka, Flink, dbt or equivalent.
• Strong SQL/NoSQL and data modeling knowledge.
• Experience integrating data from APIs and enterprise systems.
• CI/CD, automated testing, and DevOps practices.
• Docker and Kubernetes experience.
• Strong communication and stakeholder management.

Responsibilities

• Lead design, development, and delivery of scalable data solutions.
• Mentor engineers and promote collaboration and best practices.
• Guide ingestion, transformation, storage, and access patterns.
• Collaborate with product, architecture, and analytics teams.
• Ensure reliability, security, scalability, and cost-efficiency.
• Drive data governance, quality, and observability.
• Support production operations and incident response.

Skills

5+ years data engineering

Python/Java/Scala

Spark

Kafka

SQL/NoSQL

Docker

Kubernetes

dbt

APIs

Leadership

Tools

AWS

Azure

Microsoft Fabric

Datadog

Job description

Allstate seeks two Managing Engineers, Data Engineering to lead the design, development, and delivery of scalable data solutions that support digital products, analytics, and business decision-making. You will mentor data engineers, drive best practices, and partner with product and analytics teams to deliver reliable data pipelines and platforms.

The roles require hands-on coding with Python/Java/Scala, Spark, Kafka, dbt, and Docker/Kubernetes, plus strong SQL/NoSQL skills and data governance