AI Systems Engineer: DevOps, Model Serving & Observability

EY; Tulsa (OK); Hybrid; USD 107,000 - 177,000; Full time

Benefits offered by this job

Hybrid work model

Total Rewards package: medical, dental

Paid time off

Job summary

EY seeks an AI Systems Engineer to own delivery, model-serving, routing, and observability across EY’s AI-native platform. You will shape CI/CD/CV pipelines, secure model execution, and governance across cloud, on-prem, edge, and air-gapped environments.

You will work at the intersection of DevOps, MLOps, FinOps, and observability, ensuring repeatable deployments, cost-aware serving, and transparent telemetry in regulated contexts.

Qualifications

• 8+ years in DevOps, MLOps, platform, or observability engineering with hands-on AI or high-throughput services.
• Hands-on DevOps experience including CI/CD/CV pipelines and GitOps tooling for automated build, test, release, and rollback.
• Hands-on expertise operating inference/model-serving frameworks (Ray Serve, vLLM, Triton, or NIM) on GPU infrastructure.
• Strong experience with observability stacks (Prometheus, Grafana, Loki, Tempo/Jaeger) and OpenTelemetry.
• Experience with API gateways and request routing (Envoy or equivalent), including streaming responses.
• Experience with cost management / FinOps tooling (OpenCost, Kubecost) and quota/rate-limit enforcement.
• Familiarity with model/artifact registries and supply-chain scanning (Harbor, MLflow, Trivy/SBOM).
• Proven track record operating AI or service infrastructure under compliance, security, or regulatory constraints.
• Ability to define clean ownership boundaries and consumption contracts with platform, trust, and data teams.

Responsibilities

• Support DevOps and delivery for AI workloads: build and operate the CI/CD/CV pipelines that ship AI services, agents, and runtime components, including automated build, test, continuous verification, release, and rollback, so AI workloads are delivered repeatably and safely into every environment.
• Own governance and discovery for AI assets, including service catalog/registry (Artifactory/Nexus, Harbor), experiment tracking and model metadata (MLflow), upstream registries/mirrors (HuggingFace/NGC), CVE/SBOM scanning (Trivy), lineage contracts (OpenLineage), and license management.
• Own resource and cost management, including quotas and rate limits, cost attribution and utilization (Apptio/OpenCost/Kubecost), so AI execution stays economically bounded and controllable per tenant and engagement.
• Own the full observability stack, including metrics (Prometheus/Mimir), logs (Loki), traces (Tempo/Jaeger), dashboards (Grafana), LLM debugging and evaluation (LangSmith/Langfuse), and SLA/alert notifications.
• Own the OpenTelemetry collection layer, including multi-tenant receiver, exporters and queues (Kafka sink), DCGM exporter for GPU telemetry, processor batching, and dynamic filtering, so every signal is captured and routed reliably.
• Automate GitOps-based delivery and continuous verification; embedding quality, integrity, and cost gates into pipelines so releases are policy-compliant by default rather than by manual review.
• Close the loop between delivery and observability by using telemetry, evaluation, and cost signals to drive deployment decisions, progressive rollout, and automated rollback of AI workloads.
• Ensure cost and telemetry are identity-stamped and per-tenant, so consumption and behavior are attributable end-to-end, keeping FinOps and observability tied to the workloads that generate the load.

Skills

CI/CD/CV pipelines

GitOps tooling

GPU inference systems

Observability

FinOps / cost attribution

Model governance & registries

Communication with leadership

Education

Bachelor's degree in Computer Science or related field

Tools

Ray Serve

vLLM/Triton/NIM

Prometheus/Grafana

OpenTelemetry

Harbor/MLflow/Trivy

LangSmith/Langfuse

Envoy

Job description

EY seeks an AI Systems Engineer to own delivery, model-serving, routing, and observability across EY’s AI-native platform. You will shape CI/CD/CV pipelines, secure model execution, and governance across cloud, on-prem, edge, and air-gapped environments.

You will work at the intersection of DevOps, MLOps, FinOps, and observability, ensuring repeatable deployments, cost-aware serving, and transparent telemetry in regulated contexts.