Lead Data Engineer: Pipelines, Fabric & AI

allstate; Illinois; On-site; USD 110,000 - 181,025; Full time

Job summary

Allstate is seeking a Lead Data Engineering (DAE) Consultant to design scalable data infrastructure and analytics solutions for Data Science and business teams. You will lead technical efforts, partner with data scientists and analysts, and translate ambiguous problems into production-ready data solutions.

The ideal candidate brings strong Python, SQL, and Microsoft Fabric experience, with a track record in building ETL/ELT pipelines at scale and leveraging AI/ML tools to drive insights in a

Qualifications

• 5+ years of experience in Data Engineering or related field.
• Strong expertise in Python, SQL, and Microsoft Fabric.
• Experience building ETL/ELT pipelines at scale.
• Experience supporting Data Science and analytics use cases.
• Experience with AI/ML tools or platforms.
• Ability to work in a fast-paced environment.
• Ability to manage ambiguity and prioritize effectively.
• Strong attention to detail and data quality.
• Strong communication and stakeholder engagement skills.

Responsibilities

• Lead the design and implementation of scalable data pipelines and ETL workflows.
• Architect and optimize enterprise data infrastructure using Microsoft Fabric.
• Partner with stakeholders to translate ambiguous business needs into effective technical solutions.
• Translate complex technical solutions into clear, actionable insights for business partners.
• Develop prototypes and lead transition into production-grade systems.
• Establish and maintain data quality, monitoring, and validation frameworks.
• Leverage AI/ML tools to enhance automation and insights.
• Drive change management and adoption of new data solutions.
• Optimize performance for large-scale data environments.
• Serve as SME guiding data engineering best practices.
• Influence data strategy and support evolving business needs.

Skills

Actionable Insights

Analytics

Communication

Computer Programming

Data Analytics

Data Engineering

Data Science

Python

SQL

Microsoft Fabric

Tools

Microsoft Fabric

Job description

Allstate is seeking a Lead Data Engineering (DAE) Consultant to design scalable data infrastructure and analytics solutions for Data Science and business teams. You will lead technical efforts, partner with data scientists and analysts, and translate ambiguous problems into production-ready data solutions.

The ideal candidate brings strong Python, SQL, and Microsoft Fabric experience, with a track record in building ETL/ELT pipelines at scale and leveraging AI/ML tools to drive insights in a