Senior Platform Engineer (AI Platforms)

EPAM Systems, Inc.; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Learning courses

Global opportunities

Job summary

EPAM Systems, Inc. is seeking a Senior Platform Engineer to build shared AI platform services powering LLM integrations and agent runtimes.

You will manage Kubernetes-based gateways, MCP layers, and telemetry to support application teams across environments. Responsibilities include model onboarding with routing quotas, end-to-end observability with OpenTelemetry, Langfuse telemetry, and robust security via OAuth2/OIDC, SSO, and JWT.

Qualifications

• 3+ years of senior platform engineering experience with AI platform services.
• Strong Kubernetes experience operating production services.
• Hands-on Langfuse experience implementing AI observability patterns.
• Strong MCP server/registry experience for tool registration and discovery.
• Experience with OpenTelemetry for distributed tracing and metrics.
• Production GitOps using Helm and Kustomize for declarative delivery.
• Strong IAM/security knowledge: OAuth2/OIDC, SSO, JWT, secrets handling.

Responsibilities

• Build and operate shared AI platform services including LLM gateways, routing, fallback, MCP layers, and agent runtimes.
• Design and run model onboarding with routing rules, quotas, tiering, and deprecation paths.
• Implement end-to-end observability using OpenTelemetry, metrics, and logs.
• Maintain Langfuse telemetry: traces, spans, prompts, completions, feedback, evaluation metadata.
• Design MCP layer for tool registration, discovery, sessions, permissions, tracing.
• Run platform services on Kubernetes with GitOps, Helm, Kustomize, autoscaling, health checks.
• Implement authentication, authorization, tenancy with OAuth2/OIDC, SSO, JWT, API keys.
• Apply guardrails including PII detection, auditing, retention rules.
• Build dashboards, alerts and reports for reliability, latency, errors, production behavior.
• Design cost/usage observability with attribution across providers and environments.
• Create showback-ready metrics for token usage, model mix, cache behavior, vendor spend.
• Support agent lifecycle: publishing, versioning, discovery, memory, scaling, resilience.
• Develop self-service workflows for onboarding, provisioning, templates, internal portals/CLIs.

Skills

Kubernetes

Langfuse

OpenTelemetry

MCP

GitOps

IAM & Security

OAuth2/OIDC

SSO

JWT

English (B2)

Tools

Helm

Kustomize

LangChain

LangGraph

Job description

We are looking for a Senior Platform Engineer to build shared AI platform services that power LLM integrations and agent runtimes, with observability, guardrails, and cost attribution designed in from day one. You will run and evolve Kubernetes-based gateway, MCP, and telemetry capabilities that application teams rely on.ResponsibilitiesBuild and operate shared AI platform services including LLM gateways, routing, fallback, MCP layers, and agent runtimesDesign and run model onboarding across environments with routing rules, quotas, tiering, and version deprecation pathsImplement end-to-end observability for platform components using OpenTelemetry, metrics, and structured logsImplement and maintain Langfuse telemetry including traces, spans, prompts, completions, feedback, and evaluation metadataDesign and operate the MCP layer for tool registration, discovery, session handling, permission boundaries, and tool-call tracingRun platform services on Kubernetes with GitOps using Helm and Kustomize, autoscaling, health checks, and progressive rolloutImplement authentication, authorization, and tenancy with OAuth2/OIDC, SSO, JWT validation, and API key managementApply guardrails and policy enforcement including PII detection, filtering, audit logging, and retention rulesBuild dashboards, alerts, and reports for reliability, latency, errors, quality signals, and production behaviorDesign cost and usage observability across model providers with attribution by app, team, user, model, and environmentCreate showback-ready metrics for token usage, model mix, cache behavior, and vendor spend to guide routing and capacitySupport agent lifecycle capabilities including publishing, versioning, discovery, memory, scaling, and resilienceBuild developer self-service workflows for onboarding, provisioning, templates, and internal portals or CLIsRequirementsSenior-level platform engineering experience (3+ years) with AI platform servicesStrong Kubernetes experience (3+ years) operating production servicesHands-on Langfuse experience implementing AI observability patternsStrong Model Context Protocol (MCP) experience building or operating MCP servers and registriesStrong OpenTelemetry experience for distributed tracing, metrics, and structured logsProduction GitOps experience using Helm and Kustomize for declarative deliveryStrong IAM and security knowledge covering OAuth2/OIDC, SSO, JWT, and secrets handlingStrong communication skills to partner with engineering and developer teamsUpper-Intermediate English proficiency (B2) for collaboration and documentationNice to haveHands-on LangChain experience for agent or tool integration patternsHands-on LangGraph experience for graph-based agent workflowsRetrieval-Augmented Generation (RAG) experience with evaluation and retrieval quality tuningWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedIn