Stateful AI Data & Streaming Architect

EY; Tallahassee (FL); Hybrid; USD 107,000 - 177,000; Full time

Job summary

EY seeks an AI Systems Engineer to own the stateful backbone of EY’s AI-native platform, managing relational, vector, graph stores, and durable object storage across cloud, on‑prem, edge, and air‑gapped environments.

You will ensure data durability, consistency, and recoverability, overseeing streaming pipelines (Kafka, NATS) and change data capture, with schema governance and cross‑environment data movement. The role emphasizes multi‑tenant isolation and automation.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or related technical field.
• 8+ years operating production data infrastructure, streaming systems, or database platforms at scale.
• Hands-on expertise with relational databases (PostgreSQL) and caching (Redis/Valkey), including HA, replication, and backup/recovery.
• Exposure to AI/ML data patterns — embeddings, retrieval, feature/state stores for agentic workloads.
• Production experience with vector and/or graph databases (Qdrant, Milvus, PGVector, Neo4j) in AI/ML contexts.
• Deep experience with event streaming and messaging (Apache Kafka/Strimzi, NATS) and change data capture (Debezium).
• Experience with stream processing (Apache Flink) and event/schema governance (CloudEvents, schema registry).
• Experience running stateful systems on Kubernetes (operators, persistent volumes, object/storage).
• Ability to define clean ownership boundaries and data/schema contracts with platform, trust, runtime, and delivery teams.

Responsibilities

• Own the stateful data stores and memory tiers across environments (PostgreSQL, Redis/Valkey, vector stores, graph stores, object storage).
• Own event streaming and async messaging (Kafka, NATS JetStream, Debezium, Flink, CloudEvents).
• Ensure data durability, consistency, and recoverability across cross-environment data movement.
• Build and operate streaming and CDC pipelines with schema governance to prevent breaking changes.
• Make state multi-tenant and portable with isolation and performance guarantees.
• Provide data substrates and lineage support for governance and AI capabilities.

Skills

Data infrastructure

Streaming architectures

Kubernetes

Data durability

Multi-tenant isolation

Schema governance

Cloud and on-prem

Education

Bachelor’s or Master’s degree in Computer Science or related technical field

Tools

PostgreSQL

Redis/Valkey

Qdrant/Milvus/PGVector

Neo4j

MinIO/OpenEBS

Kafka/Strimzi

NATS JetStream

Debezium

Apache Flink

CloudEvents/Schema Registry

OpenLineage/Marquez

Temporal/DBOS

Kubernetes operators

Job description

EY seeks an AI Systems Engineer to own the stateful backbone of EY’s AI-native platform, managing relational, vector, graph stores, and durable object storage across cloud, on‑prem, edge, and air‑gapped environments.

You will ensure data durability, consistency, and recoverability, overseeing streaming pipelines (Kafka, NATS) and change data capture, with schema governance and cross‑environment data movement. The role emphasizes multi‑tenant isolation and automation.