Lead AWS Event-Driven Data Engineer

State Farm; Richardson (TX); Hybrid; USD 81,000 - 142,000; Full time

Benefits offered by this job

Tuition assistance

401(k) with company contributions

Generous time off

Health and wellness programs

Matching Gift Program

Educational benefits

Job summary

State Farm is seeking a data engineer in the Richardson, TX area for a hybrid role focused on building scalable data pipelines and an enterprise lakehouse on Databricks. You will implement CDC patterns, design canonical data models, and enable AI/analytics by delivering trusted data products.

The role emphasizes hands-on work with Kafka/Kinesis, PySpark/SQL, and cloud data services, collaborating with analytics teams to support business decisions.

Qualifications

• Hands-on experience designing and building event-driven data pipelines with idempotent consumers and schema evolution.
• Experience designing canonical/shared data models across multiple consumers/versions.
• Experience with change-data-capture (CDC) from operational stores into event streams.

Responsibilities

• Define Canonical Data and Event Schemas for domain events across APIs/systems.
• Design and Deliver Event-Driven Architectures using AWS-native services including Kinesis, SQS, SNS, EventBridge.
• Design and Build Data Pipelines using Kinesis, Kafka, Lambda, AWS Glue, Step Functions for AI/Analytics use cases.
• Build CDC pipelines from stores like DynamoDB/RDS into canonical domain events on the event backbone.
• Create Materialized Read Models in DynamoDB/OpenSearch/Aurora for near real-time dashboards.
• Model Enterprise Data with dimensional models and dbt-based semantics for analytics workloads.
• Lead Lakehouse transition toward Databricks Delta Lake/Iceberg architectures.
• Develop Data Access APIs exposing governed data products to analysts and AI systems.
• Engineer data pipelines for AI readiness and data governance.
• Collaborate on CI/CD with GitLab and Terraform; mentor engineers on data engineering best practices.

Skills

Event-driven pipelines

Canonical data models

CDC patterns

Databricks Lakehouse

PySpark/SQL

Data modeling

CI/CD with GitLab

Tools

Kafka or Kinesis

AWS services (Kinesis, S3, Lambda)

Databricks

OpenSearch

Job description

State Farm is seeking a data engineer in the Richardson, TX area for a hybrid role focused on building scalable data pipelines and an enterprise lakehouse on Databricks. You will implement CDC patterns, design canonical data models, and enable AI/analytics by delivering trusted data products.

The role emphasizes hands-on work with Kafka/Kinesis, PySpark/SQL, and cloud data services, collaborating with analytics teams to support business decisions.