Lead Data Engineer (AWS) - Hybrid Data Pipelines

State Farm; Tempe (AZ); Hybrid; USD 110,000 - 160,000; Full time

Job summary

State Farm is seeking a Data Engineer or Lead Data Engineer to join the Innovation Group’s SAIL lab, focusing on Agency and Digital. You will work with software engineers, product teams, and business partners to design, build, and operate data capabilities for experimentation and product development.

The role emphasizes turning ambiguous data needs into scalable, production-quality solutions, with opportunities to mentor engineers and shape data architectures, while supporting ML and AI

Qualifications

• 3+ years with Python development for ETL data
• Proficiency in SQL and relational databases
• Experience building ETL/ELT data pipelines
• Experience with structured, semi-structured, and unstructured data (CSV, JSON, text)
• Experience with AWS: S3, Glue, Lambda
• Git and modern version-control practices
• Familiarity with CI/CD and deploying to production
• Comfort with Linux and CLI
• Understanding of testing, maintainability, observability, reusable design
• Troubleshooting data quality, pipelines, integrations, performance

Responsibilities

• Move and transform data via code and pipelines
• Collaborate with software engineers and product teams to determine data access patterns
• Design and maintain data platforms for structured and unstructured data
• Provide technical leadership, mentoring, and influence architecture
• Lead data science/ML efforts, supporting lifecycle from exploration to deployment and operation

Skills

Python

SQL

ETL/ELT

Data pipelines

CI/CD

Linux

Observability

Troubleshooting

Agile

Communication

Tools

AWS

S3

Glue

Lambda

Git

CI/CD tools

Job description

State Farm is seeking a Data Engineer or Lead Data Engineer to join the Innovation Group’s SAIL lab, focusing on Agency and Digital. You will work with software engineers, product teams, and business partners to design, build, and operate data capabilities for experimentation and product development.

The role emphasizes turning ambiguous data needs into scalable, production-quality solutions, with opportunities to mentor engineers and shape data architectures, while supporting ML and AI