Lead Data Engineer — Hybrid Cloud Data Pipelines

State Farm; Richardson (TX); Hybrid; USD 86,000 - 135,000; Full time

Benefits offered by this job

Annual incentive up to 15%

Competitive salary up to $135,000

Comprehensive health benefits

Tuition assistance

Educational programs and mentoring

Generous time off and parental leave

Retirement plan with company match

Diverse and inclusive culture

Job summary

State Farm is hiring a Data Engineer to build and scale data products and pipelines that power better business decisions and customer retention. The role sits in Enterprise Technology and offers a collaborative environment focused on meaningful business impact.

You will partner across teams, tackle complex data challenges, and contribute to modern data solutions aligned with enterprise goals. This position suits someone eager to expand technical skills in a fast-moving setting.

Qualifications

• Professional experience as a Data Engineer.
• Proficiency in Python, Scala, Java, Bash.
• Hands-on with AWS services including ETL tools (Glue, EMR Serverless), Lambda, Step Functions, EventBridge, S3, Kinesis Firehose, Redshift, and Iceberg.
• Experience with distributed data processing frameworks such as Apache Spark, Databricks, and Spring Batch.
• Infrastructure as code tools such as OpenTofu for managing cloud resources and deployments.
• Familiarity with CI/CD pipelines including automated testing and security scans.
• Knowledge of auditing and monitoring tools: Dynatrace, AWS QuickSight and CloudWatch.

Responsibilities

• Uses industry-standard languages and frameworks for coding, testing, security, DevOps, and DataOps practices.
• Develops and maintains scalable, reusable data solutions across platforms and compute environments.
• Handles ETL data ingestion, cleansing, profiling, and deployment for analytics and production.
• Builds domain knowledge for State Farm data sources and investigates new data resources.
• Explores emerging technologies and critical systems in data engineering.
• Works with structured and unstructured data; applies DataOps mindset for quality data products.
• Collects requirements and designs implementation and integration plans.
• Ensures deployment and security of technology to support data assets in compliance with best practices.
• Applies complex computer science principles to data engineering problems.

Skills

Python

Scala

Java

Bash

AWS (Glue, EMR, Lambda)

Spark

Databricks

OpenTofu

CI/CD

Dynatrace

CloudWatch

SQL

Athena

PostgreSQL

Redshift

GitHub/GitLab

Tools

Spark (Framework)

Databricks

OpenTofu

Terraform/OpenTofu

GitHub

GitLab

PostgreSQL

Redshift

Athena

Job description

State Farm is hiring a Data Engineer to build and scale data products and pipelines that power better business decisions and customer retention. The role sits in Enterprise Technology and offers a collaborative environment focused on meaningful business impact.

You will partner across teams, tackle complex data challenges, and contribute to modern data solutions aligned with enterprise goals. This position suits someone eager to expand technical skills in a fast-moving setting.