Hybrid Data Engineer: Python, AWS & Spark

Information Technology Senior Management Forum; Richardson (TX); Hybrid; USD 86,000 - 130,000; Full time

Benefits offered by this job

Health and wellness benefits

Educational benefits (tuition)

Job summary

State Farm's Property & Casualty Data Pipeline team is hiring an Experienced Data Engineer to build and scale data products and pipelines powering better business decisions, growth, and retention.

This role sits in Enterprise Technology in a collaborative environment, partnering across teams to solve data challenges and shape modern data solutions aligned to enterprise goals. Strong candidate will have Python/Spark SQL, AWS data services, Databricks, and IaC/CI/CD experience.

Qualifications

• Experience as a Data Engineer.
• Proficiency in Python, Spark SQL (or PySpark), R, Java, Bash.
• Hands-on experience with AWS data services including Glue, EMR Serverless, Lambda, Step Functions, S3, DynamoDB, Kinesis Firehose, Redshift, Iceberg, and SageMaker.
• Experience with Apache Spark and Databricks.
• Experience with infrastructure as code tools for cloud deployments (OpenTofu/Terraform).
• Familiarity with CI/CD pipelines and Airflow.

Responsibilities

• Uses languages and frameworks across coding, testing, security, DevOps, and DataOps in data engineering practices.
• Develops scalable, reusable data solutions across platforms and compute environments.
• Handles ETL: acquisition, cleansing, profiling, and loading for analytics and deployment.
• Gathers domain knowledge of State Farm data sources and explores data resource acquisition.
• Identifies emerging technologies and core data systems and applicable tools.
• Manages datasets with structured and unstructured data.
• Applies DataOps practices to ensure data aligns with enterprise needs.

Skills

Python

Spark SQL / PySpark

R

Java

Bash

Tools

Glue

EMR Serverless

Lambda

Step Functions

EventBridge

S3

DynamoDB

Kinesis Firehose

Redshift

Iceberg

SageMaker

Databricks

OpenTofu

Terraform

Airflow

Job description

State Farm's Property & Casualty Data Pipeline team is hiring an Experienced Data Engineer to build and scale data products and pipelines powering better business decisions, growth, and retention.

This role sits in Enterprise Technology in a collaborative environment, partnering across teams to solve data challenges and shape modern data solutions aligned to enterprise goals. Strong candidate will have Python/Spark SQL, AWS data services, Databricks, and IaC/CI/CD experience.