AI Systems Engineer: DevOps, Delivery & Observability

EY; Alpharetta (GA); Hybrid; USD 107,000 - 177,000; Full time

Benefits offered by this job

Hybrid work model

Total Rewards package

Medical and dental coverage

Pension and 401(k)

Job summary

EY seeks an AI Systems Engineer to own delivery, model-serving, routing, and observability for EY’s AI-native platform. You will shape how AI services are built and deployed across cloud, on‑prem, edge, and air‑gapped environments in a hybrid setup.

The role emphasizes CI/CD, governance, cost attribution, and deep observability using Prometheus, Grafana, Loki, Tempo/Jaeger, and OpenTelemetry, with GPU inference frameworks like Ray Serve, vLLM, Triton, or NIM. 8+ years in DevOps/MLOps preferred.

Qualifications

• 8+ years in DevOps, MLOps, platform, or observability engineering with hands-on production ownership of AI or high-throughput services.
• Strong hands-on DevOps experience with CI/CD/CV pipelines and GitOps tooling (ArgoCD, Helm, GitHub Actions/GitLab CI) for automated build, test, release, and rollback.
• Hands-on expertise operating inference/model-serving frameworks on GPU infrastructure (Ray Serve, vLLM, Triton, NIM).
• Strong experience with observability stacks (Prometheus, Grafana, Loki, Tempo/Jaeger) and OpenTelemetry.
• Experience with API gateways and request routing (Envoy) including streaming responses.
• Experience with cost management/FinOps tooling (OpenCost, Kubecost) and quota/rate-limit enforcement.
• Familiarity with model/artifact registries and supply-chain scanning (Harbor, MLflow, Trivy/SBOM).
• Proven track record operating AI/service infrastructure under compliance and regulatory constraints.

Responsibilities

• Supports DevOps and delivery for AI workloads: CI/CD/CV pipelines and automated release into every environment.
• Own governance and discovery for AI assets including registries and model metadata.
• Own resource and cost management with quotas and cost attribution per tenant and engagement.
• Own the full observability stack: metrics, logs, traces, dashboards, and LLM evaluation tooling.
• Own OpenTelemetry collection, including multi-tenant receivers and GPU telemetry exporters.
• Automate GitOps-based delivery and continuous verification with policy gates.
• Close the loop between delivery and observability to drive deployment decisions and automated rollback.
• Ensure consumption is identity-stamped and per-tenant for FinOps alignment.

Skills

DevOps

AI systems/MLops

Observability

FinOps

Governance/registries

Multi-environment

Communication

Education

Bachelor’s or Master’s in Computer Science

Tools

Ray Serve

vLLM

Triton

NIM

Prometheus

Grafana

Loki

Tempo/Jaeger

OpenTelemetry

Envoy

OpenCost

Kubecost

Harbor

MLflow

Trivy/SBOM

Job description

EY seeks an AI Systems Engineer to own delivery, model-serving, routing, and observability for EY’s AI-native platform. You will shape how AI services are built and deployed across cloud, on‑prem, edge, and air‑gapped environments in a hybrid setup.

The role emphasizes CI/CD, governance, cost attribution, and deep observability using Prometheus, Grafana, Loki, Tempo/Jaeger, and OpenTelemetry, with GPU inference frameworks like Ray Serve, vLLM, Triton, or NIM. 8+ years in DevOps/MLOps preferred.