Senior AI Data & State Platform Engineer

EY; San Mateo (CA); Hybrid; USD 107,000 - 177,000; Full time

Job summary

EY is seeking an AI Systems Engineer to own the stateful backbone of EY’s AI-native platform, including data stores, memory tiers, and event streaming that hold and move state across cloud, on-prem, edge, client-managed, and air-gapped environments.

This role covers relational, vector/graph stores, durable workflows, and streaming pipelines, ensuring durability, consistency, and multi-tenant portability while enabling governance and lineage across teams.

Qualifications

• Bachelor's or Master's degree in Computer Science or related technical field.
• 8+ years operating production data infrastructure, streaming systems, or database platforms at scale.
• Hands-on expertise with PostgreSQL and caching technologies (Redis/Valkey) including HA, replication, and backup/recovery.
• Exposure to AI/ML data patterns — embeddings, retrieval, feature/state stores for agentic workloads.
• Production experience with vector/graph databases (Qdrant, Milvus, PGVector, Neo4j) in AI/ML contexts.
• Deep experience with event streaming and messaging (Kafka/Strimzi, NATS) and change data capture (Debezium).
• Experience with stream processing (Flink) and event/schema governance (CloudEvents, schema registry).
• Experience running stateful systems on Kubernetes (operators, persistent volumes, MinIO/OpenEBS).
• Ability to define clean ownership boundaries and data/schema contracts with platform, trust, runtime, and delivery teams.

Responsibilities

• Own the memory and data stores: relational and durable state, caching, vector/graph stores, and object storage across environments.
• Own event streaming and async messaging: Kafka, NATS JetStream, Debezium, Flink, and CloudEvents.
• Own data durability, consistency, and recoverability: replication, backup/restore, and cross-environment data movement.
• Build and operate streaming and CDC pipelines moving data reliably with schema governance.
• Make state multi-tenant and portable with isolation and performance across cloud, on-prem, edge, and air-gapped environments.
• Provide data and lineage substrate for governance, observability, and AI knowledge capabilities; integrate lineage tooling.

Skills

Production databases

Streaming architectures

Durability mindset

Stateful systems

Schema governance

Communication

Infrastructure as Code

Education

Bachelor's or Master's in CS

Tools

PostgreSQL

Redis

Valkey

Qdrant

Milvus

PGVector

Neo4j

MinIO

OpenEBS

Kafka

NATS

Debezium

Flink

CloudEvents

Job description

EY is seeking an AI Systems Engineer to own the stateful backbone of EY’s AI-native platform, including data stores, memory tiers, and event streaming that hold and move state across cloud, on-prem, edge, client-managed, and air-gapped environments.

This role covers relational, vector/graph stores, durable workflows, and streaming pipelines, ensuring durability, consistency, and multi-tenant portability while enabling governance and lineage across teams.