AI Systems Engineer: DevOps, Model Serving & Observability

Ernst & Young Oman; Southfield (MI); On-site; USD 107,000 - 177,000; Full time

Benefits offered by this job

Hybrid work model

Medical and dental coverage

Pension and 401(k)

Job summary

EY is seeking an AI Systems Engineer to own delivery, model-serving, routing and observability for the AI-native platform across cloud, on-prem, edge, and air-gapped environments. You will define governance, discovery, and cost management while maintaining end-to-end observability and secure model execution.

The role spans DevOps, MLOps, FinOps, and observability, requiring deep experience with GPUs, AI workloads, and multi-tenant environments.

Qualifications

• 8+ years in DevOps, MLOps, platform, or observability engineering with hands-on ownership of AI workloads.
• Hands-on CI/CD/CV pipelines and GitOps tooling (ArgoCD, Helm, GitHub Actions/GitLab CI).
• Experience operating AI/model-serving frameworks on GPU infrastructure (Ray Serve, vLLM, Triton, NIM).
• Strong observability stack experience (Prometheus, Grafana, Loki, Tempo/Jaeger) and OpenTelemetry.
• Familiarity with model/artifact registries and supply-chain scanning (Harbor, MLflow, Trivy/SBOM).
• Experience with API gateways and request routing (Envoy).
• Cost attribution tooling (OpenCost, Kubecost) and quota enforcement.
• Ability to define ownership boundaries and consumption contracts across teams.

Responsibilities

• Own the delivery and runtime surface for AI workloads including CI/CD/CV pipelines, model execution, and governance.
• Manage AI asset discovery/registry, experiment tracking, and lineage contracts.
• Oversee resource/cost management with per-tenant attribution and governance controls.
• Own the full observability stack: metrics, logs, traces, dashboards, and SLA/alerts.
• Automate GitOps-based delivery with quality gates integrated into pipelines.
• Drive deployment decisions via telemetry, evaluation, and automated rollbacks.
• Ensure AI workloads are compliant with security, regulatory, and governance requirements.

Skills

DevOps

MLOps

Observability

FinOps

Governance

Communication

Multi-cloud

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Ray Serve

vLLM

Triton

NIM

MLflow

Prometheus

Grafana

Loki

Tempo

Jaeger

Envoy

OpenLineage

Kubecost

OpenCost

Job description

EY is seeking an AI Systems Engineer to own delivery, model-serving, routing and observability for the AI-native platform across cloud, on-prem, edge, and air-gapped environments. You will define governance, discovery, and cost management while maintaining end-to-end observability and secure model execution.

The role spans DevOps, MLOps, FinOps, and observability, requiring deep experience with GPUs, AI workloads, and multi-tenant environments.