AI Platform Delivery & Observability Lead

EY; Boston (MA); On-site; USD 126,000 - 230,000; Full time

Benefits offered by this job

Medical coverage

Dental coverage

401(k) plan

Paid time off

Hybrid/remote work options

Job summary

EY seeks an AI Systems Engineer to own delivery, model-serving, routing, and observability for EY’s AI-native platform. You will manage CI/CD/CV pipelines, governance of AI assets, and cost attribution across cloud, on-prem, edge, and air‑gapped environments.

You will lead the full observability stack, model-serving frameworks on GPUs, and secure, compliant delivery across tenants, with strong emphasis on open telemetry, governance, and scalable infrastructure.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or related technical field.
• 8+ years in DevOps, MLOps, platform, or observability engineering with hands-on ownership of AI or high-throughput services.
• Hands-on DevOps experience with CI/CD/CV pipelines and GitOps tooling (ArgoCD, Helm, GitHub Actions/GitLab CI, or equivalents).
• Hands-on expertise operating inference/model-serving frameworks (Ray Serve, vLLM, Triton, or NIM) on GPU infrastructure.
• Experience with observability stacks (Prometheus, Grafana, Loki, Tempo/Jaeger) and OpenTelemetry.
• Experience with API gateways and request routing (Envoy or equivalent), including streaming responses.
• Experience with cost management / FinOps tooling (OpenCost, Kubecost, or equivalent) and quota/rate-limit enforcement.
• Familiarity with model/artifact registries and supply-chain scanning (Harbor, MLflow, Trivy/SBOM).
• Proven track record operating AI or service infrastructure under compliance, security, or regulatory constraints.
• Ability to define clean ownership boundaries and consumption contracts with platform, trust, and data teams.

Responsibilities

• Own DevOps and delivery for AI workloads: CI/CD/CV pipelines to ship AI services and runtime components.
• Own governance and discovery for AI assets (registries, tracking, metadata).
• Own resource and cost management to keep AI execution bounded per tenant.
• Own the full observability stack: metrics, logs, traces, dashboards, and AI debugging tools.
• Own the OpenTelemetry collection layer, including multi-tenant receivers and GPU telemetry.
• Automate GitOps-based delivery and continuous verification with policy gates.
• Close the loop between delivery and observability using telemetry and cost signals for deployment decisions.
• Ensure per-tenant telemetry and cost attribution across workloads.

Skills

DevOps

MLOps

Observability

FinOps

OpenTelemetry

GitOps

CI/CD

Education

Bachelor’s or Master’s in Computer Science

Tools

Envoy

Prometheus

Grafana

Loki

Trivy

Job description

EY seeks an AI Systems Engineer to own delivery, model-serving, routing, and observability for EY’s AI-native platform. You will manage CI/CD/CV pipelines, governance of AI assets, and cost attribution across cloud, on-prem, edge, and air‑gapped environments.

You will lead the full observability stack, model-serving frameworks on GPUs, and secure, compliant delivery across tenants, with strong emphasis on open telemetry, governance, and scalable infrastructure.