Remote Data Engineer: Risk Data Pipelines & Analytics

Allstate Private Limited; United States; Remote; USD 100,000 - 171,000; Full time

Benefits offered by this job

Laptop provided

Connectivity reimbursement

Home office stipend

Job summary

Allstate seeks a Data Engineer to design and scale risk data pipelines and analytics-ready datasets. You will collaborate with analytics engineers, product teams, risk partners, and governance stakeholders to reduce manual reporting and enable dashboards and AI-ready capabilities.

The role emphasizes data quality, lineage, and automated testing within modern data lake/lakehouse environments, including Fabric/OneLake, with a strong focus on scalability and cost efficiency.

Qualifications

• 4+ years of experience building production-grade data pipelines and data products.
• Hands-on experience with Apache Spark for large-scale data processing.
• Strong proficiency in Python, SQL, and data engineering best practices.
• Experience developing ETL/ELT solutions within cloud-based data lakes or lakehouses.
• Experience designing analytical data models for reporting, dashboards, and advanced analytics.
• Familiarity with CI/CD pipelines, version control, automated testing, and IaC concepts.

Responsibilities

• Design, build, and maintain trusted risk data products and scalable pipelines.
• Integrate data from multiple sources into governed, analytics-ready datasets.
• Develop and optimize ETL/ELT workflows for ingestion, transformation, and delivery.
• Manage data processing workloads in modern data lake/lakehouse environments.
• Implement data quality, monitoring, lineage, and reconciliation processes.
• Create curated datasets and outputs to support dashboards and analytics.
• Optimize architectures for scalability, performance, resilience, and cost.
• Develop reusable frameworks, CI/CD pipelines, and automated testing.
• Collaborate with stakeholders to establish data definitions, metrics, and access controls.
• Participate in Agile delivery activities including sprint planning and reviews.

Skills

Apache Spark

Data Engineering

Data ETL

Python

SQL

CI/CD

Tools

Microsoft Fabric

OneLake

Job description

Allstate seeks a Data Engineer to design and scale risk data pipelines and analytics-ready datasets. You will collaborate with analytics engineers, product teams, risk partners, and governance stakeholders to reduce manual reporting and enable dashboards and AI-ready capabilities.

The role emphasizes data quality, lineage, and automated testing within modern data lake/lakehouse environments, including Fabric/OneLake, with a strong focus on scalability and cost efficiency.