Senior Platform Engineer — AI Platform on Kubernetes

EPAM Systems, Inc.; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Learning courses

Global opportunities

Job summary

EPAM Systems, Inc. is seeking a Senior Platform Engineer to build shared AI platform services powering LLM integrations and agent runtimes.

You will manage Kubernetes-based gateways, MCP layers, and telemetry to support application teams across environments. Responsibilities include model onboarding with routing quotas, end-to-end observability with OpenTelemetry, Langfuse telemetry, and robust security via OAuth2/OIDC, SSO, and JWT.

Qualifications

• 3+ years of senior platform engineering experience with AI platform services.
• Strong Kubernetes experience operating production services.
• Hands-on Langfuse experience implementing AI observability patterns.
• Strong MCP server/registry experience for tool registration and discovery.
• Experience with OpenTelemetry for distributed tracing and metrics.
• Production GitOps using Helm and Kustomize for declarative delivery.
• Strong IAM/security knowledge: OAuth2/OIDC, SSO, JWT, secrets handling.

Responsibilities

• Build and operate shared AI platform services including LLM gateways, routing, fallback, MCP layers, and agent runtimes.
• Design and run model onboarding with routing rules, quotas, tiering, and deprecation paths.
• Implement end-to-end observability using OpenTelemetry, metrics, and logs.
• Maintain Langfuse telemetry: traces, spans, prompts, completions, feedback, evaluation metadata.
• Design MCP layer for tool registration, discovery, sessions, permissions, tracing.
• Run platform services on Kubernetes with GitOps, Helm, Kustomize, autoscaling, health checks.
• Implement authentication, authorization, tenancy with OAuth2/OIDC, SSO, JWT, API keys.
• Apply guardrails including PII detection, auditing, retention rules.
• Build dashboards, alerts and reports for reliability, latency, errors, production behavior.
• Design cost/usage observability with attribution across providers and environments.
• Create showback-ready metrics for token usage, model mix, cache behavior, vendor spend.
• Support agent lifecycle: publishing, versioning, discovery, memory, scaling, resilience.
• Develop self-service workflows for onboarding, provisioning, templates, internal portals/CLIs.

Skills

Kubernetes

Langfuse

OpenTelemetry

MCP

GitOps

IAM & Security

OAuth2/OIDC

SSO

JWT

English (B2)

Tools

Helm

Kustomize

LangChain

LangGraph

Job description

EPAM Systems, Inc. is seeking a Senior Platform Engineer to build shared AI platform services powering LLM integrations and agent runtimes.

You will manage Kubernetes-based gateways, MCP layers, and telemetry to support application teams across environments. Responsibilities include model onboarding with routing quotas, end-to-end observability with OpenTelemetry, Langfuse telemetry, and robust security via OAuth2/OIDC, SSO, and JWT.