Senior AI Data Infrastructure Engineer (State & Streaming)

EY; Hoboken (NJ); Hybrid; USD 107,000 - 177,000; Full time

Job summary

EY is seeking an AI Systems Engineer to own the stateful backbone of EY’s AI-native platform, including data stores, memory tiers, and event streaming systems across cloud, on-prem, edge, and air-gapped environments.

The role blends data infrastructure, database platforms, and streaming expertise to ensure durable, consistent access to state for agentic workloads in regulated client contexts, with a focus on reliability and scalable architectures.

Qualifications

• Bachelor's or Master’s degree in Computer Science or related field.
• 8+ years operating production data infrastructure, streaming systems, or database platforms at scale.
• Hands-on expertise with PostgreSQL and caching (Redis/Valkey), including HA, replication, and backup/recovery.
• Exposure to AI/ML data patterns — embeddings, retrieval, feature/state stores for agentic workloads.
• Production experience with vector and/or graph databases (Qdrant, Milvus, PGVector, Neo4j) in AI/ML contexts.
• Deep experience with event streaming and messaging (Kafka/Strimzi, NATS) and change data capture (Debezium).
• Experience with stream processing (Flink) and event/schema governance (CloudEvents, schema registry).
• Experience running stateful systems on Kubernetes (operators, persistent volumes, object/storage such as MinIO/OpenEBS).
• Ability to define clean ownership boundaries and data/schema contracts with platform, trust, runtime, and delivery teams.

Responsibilities

• Own the memory and data stores: relational and durable state; caching; vector stores; knowledge graphs; object storage across environments.
• Own event streaming and async messaging: Kafka, NATS JetStream, Debezium, Flink, CloudEvents.
• Own data durability, consistency, and recoverability: replication, backup/restore, point-in-time recovery, cross-environment data movement.
• Build and operate streaming and CDC pipelines with schema governance to prevent breaking changes.
• Make state multi-tenant and portable with isolation, performance, and consistent semantics across environments.
• Provide the data and lineage substrate for governance, observability, and AI knowledge capabilities; integrate with lineage tooling.

Skills

Production databases

Streaming systems

Stateful systems

Cloud & on-prem

Data governance

Schema evolution

Cross-team communication

Automation & IaC

Education

Bachelor/Master CS

Tools

PostgreSQL

Redis

Milvus

Neo4j

MinIO

OpenEBS

Kafka

Debezium

Flink

CloudEvents

Job description

EY is seeking an AI Systems Engineer to own the stateful backbone of EY’s AI-native platform, including data stores, memory tiers, and event streaming systems across cloud, on-prem, edge, and air-gapped environments.

The role blends data infrastructure, database platforms, and streaming expertise to ensure durable, consistent access to state for agentic workloads in regulated client contexts, with a focus on reliability and scalable architectures.