AWS Event-Driven Data Engineer (Hybrid)

State Farm; Atlanta (GA); Hybrid; USD 81,000 - 142,000; Full time

Benefits offered by this job

Hybrid work environment

Health benefits

Education and tuition assistance

401(k) with company contributions

Job summary

State Farm is seeking a data engineering lead to design and extend an event-driven data platform. You will build scalable pipelines, CDC flows, and lakehouse transitions while enabling AI/ML data readiness. The role requires strong SQL, PySpark, and AWS expertise within a hybrid work environment.

You will partner with cross-functional teams to deliver governed data products, implement CI/CD, and ensure data quality and security across streaming and batch workloads.

Qualifications

• Experience designing and building event-driven data pipelines with idempotent consumers and schema evolution
• Experience designing shared data models across multiple consumers and versions
• Experience with CDC patterns from operational stores into event streams
• Experience building materialized read models for sub-second queries
• Knowledge of data governance and security for sensitive enterprise data

Responsibilities

• Define Canonical Data and Event Schemas - contribute to versioned domain event types
• Design and Deliver Event-Driven Architectures using AWS messaging and streaming services (Kinesis, SQS, SNS, EventBridge)
• Design and Build Data Pipelines with Kinesis, Kafka, Lambda, AWS Glue, Step Functions
• Build Change-Data-Capture (CDC) pipelines from operational stores into domain events
• Build Materialized Read Models in DynamoDB, OpenSearch, and Aurora
• Model Enterprise Data with dimensional models and dbt semantic layer definitions
• Engineer the Lakehouse Transition toward Databricks, Delta Lake, Iceberg
• Develop Data Access APIs exposing governed data products
• Engineer Derived-Fact Write-Back to the event backbone
• Enable AI/ML Data Readiness with feature datasets and embeddings
• Champion DevSecOps for Data with CI/CD integration and observability
• Document data dictionaries and runbooks; mentor and guide teams
• Mentor and collaborate to uplift data engineering practices

Skills

Data modeling

PySpark

SQL

AWS

Event-driven architecture

GitLab

Terraform

Delta Lake

Databricks

Analytical thinking

Tools

Kinesis

Kafka

OpenSearch

Job description

State Farm is seeking a data engineering lead to design and extend an event-driven data platform. You will build scalable pipelines, CDC flows, and lakehouse transitions while enabling AI/ML data readiness. The role requires strong SQL, PySpark, and AWS expertise within a hybrid work environment.

You will partner with cross-functional teams to deliver governed data products, implement CI/CD, and ensure data quality and security across streaming and batch workloads.