Senior Platform Engineer (AI Platforms)

EPAM Systems; United States; Remote; USD 140,000 - 210,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off and sick leave

Upskilling and certification programs

LinkedIn Learning access

Job summary

EPAM Systems, Inc. is seeking a Senior Platform Engineer to build shared AI platform services powering LLM integrations and agent runtimes. You will operate Kubernetes-based gateway, MCP layers, and telemetry capabilities relied upon by product teams.

You will implement model onboarding, routing, quotas, and version deprecation, while delivering end-to-end observability with OpenTelemetry. Strong security and IAM practices are essential, with a focus on governance and cost attribution.

Qualifications

• 3+ years of senior platform engineering experience with AI platform services.
• Strong Kubernetes experience operating production services.
• Hands-on Langfuse experience implementing AI observability patterns.
• Strong MCP (Model Context Protocol) experience building or operating MCP servers and registries.
• OpenTelemetry experience for distributed tracing, metrics, and structured logs.

Responsibilities

• Build and operate shared AI platform services including LLM gateways, routing, and agent runtimes.
• Design and run model onboarding with routing rules, quotas, tiering, and version deprecation paths.
• Implement end-to-end observability using OpenTelemetry, metrics and structured logs.
• Implement authentication, authorization, tenancy, OAuth2/OIDC, SSO, and API key management.
• Apply guardrails, data privacy, auditing, and retention rules across platforms.

Skills

Kubernetes production

Langfuse observability

MCP (Model Context Protocol)

OpenTelemetry

OAuth2 / OIDC / SSO / JWT

GitOps with Helm

Kubernetes tooling / IAM security

RAG & evaluation tooling

Distributed tracing & metrics

English communication (B2+)

Tools

Helm

Kustomize

LangChain

LangGraph

OAuth tooling

Job description

We are looking for a Senior Platform Engineer to build shared AI platform services that power LLM integrations and agent runtimes, with observability, guardrails, and cost attribution designed in from day one. You will run and evolve Kubernetes-based gateway, MCP, and telemetry capabilities that application teams rely on.ResponsibilitiesBuild and operate shared AI platform services including LLM gateways, routing, fallback, MCP layers, and agent runtimesDesign and run model onboarding across environments with routing rules, quotas, tiering, and version deprecation pathsImplement end-to-end observability for platform components using OpenTelemetry, metrics, and structured logsImplement and maintain Langfuse telemetry including traces, spans, prompts, completions, feedback, and evaluation metadataDesign and operate the MCP layer for tool registration, discovery, session handling, permission boundaries, and tool-call tracingRun platform services on Kubernetes with GitOps using Helm and Kustomize, autoscaling, health checks, and progressive rolloutImplement authentication, authorization, and tenancy with OAuth2/OIDC, SSO, JWT validation, and API key managementApply guardrails and policy enforcement including PII detection, filtering, audit logging, and retention rulesBuild dashboards, alerts, and reports for reliability, latency, errors, quality signals, and production behaviorDesign cost and usage observability across model providers with attribution by app, team, user, model, and environmentCreate showback-ready metrics for token usage, model mix, cache behavior, and vendor spend to guide routing and capacitySupport agent lifecycle capabilities including publishing, versioning, discovery, memory, scaling, and resilienceBuild developer self-service workflows for onboarding, provisioning, templates, and internal portals or CLIsRequirementsSenior-level platform engineering experience (3+ years) with AI platform servicesStrong Kubernetes experience (3+ years) operating production servicesHands-on Langfuse experience implementing AI observability patternsStrong Model Context Protocol (MCP) experience building or operating MCP servers and registriesStrong OpenTelemetry experience for distributed tracing, metrics, and structured logsProduction GitOps experience using Helm and Kustomize for declarative deliveryStrong IAM and security knowledge covering OAuth2/OIDC, SSO, JWT, and secrets handlingStrong communication skills to partner with engineering and developer teamsUpper-Intermediate English proficiency (B2) for collaboration and documentationNice to haveHands-on LangChain experience for agent or tool integration patternsHands-on LangGraph experience for graph-based agent workflowsRetrieval-Augmented Generation (RAG) experience with evaluation and retrieval quality tuningWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedIn