Senior Platform Engineer

Value Innovation Labs; Oaks (PA); On-site; USD 140,000 - 190,000; Full time

Job summary

Value Innovation Labs is seeking a Senior Platform Engineer to build and operate the control plane for an enterprise Agentic AI Platform in the financial services sector. The role centers on AI Gateway, Identity & Registration, and Observability, within a small, senior delivery team.

Responsibilities include routing model traffic across clouds/on-prem, enforcing gateway policies, and hardening services for production.

Qualifications

• 5+ years hands-on engineering across API platforms, identity, or observability.
• 4+ years building and operating production API gateways.
• 3+ years enterprise identity and access engineering.
• 3+ years OpenTelemetry / Distributed Tracing.
• 3+ years SRE or platform operations.
• 2+ years working with LLM/AI workloads in production.
• Experience with Langfuse, LangSmith, Arize, or equivalent LLM observability platforms.
• Production API operations incl. failover, caching, load testing, incident response.

Responsibilities

• Build and operate an AI gateway routing model traffic across cloud, third-party, and on-premises environments.
• Implement gateway policies including routing, rate limiting, quotas, retries, failover, circuit breaking, caching, and transformations.
• Harden existing services for production and manage migrations without changing consumer contracts.
• Build identity and registration workflows for workloads/services.
• Design and implement OpenTelemetry schemas and telemetry pipelines across agent runtimes.
• Develop fleet visibility for agent ownership, version, environment, health, and consumption.

Skills

API platforms

Identity engineering

Observability

OpenTelemetry

SRE

Cloud infrastructure

Python

C#

OAuth2/OIDC

LLM workloads

Tools

Langfuse

LangSmith

Arize

Job description

Hiring: Senior Platform Engineer – AI Gateway, Identity & Observability

Platform: Agentic AI Platform | Financial Services

Experience: 8+ Years Overall | 5+ Years Hands-on API Platform, Identity, or Observability Engineering

We are looking for a highly experienced Senior Platform Engineer to build and operate the control plane for an enterprise Agentic AI Platform in the financial services industry.

The role focuses on three interconnected areas: AI Gateway, Identity & Registration, and Observability. This is a hands-on engineering position within a small, senior delivery team.

Key Responsibilities

• Build and operate an AI gateway routing model traffic across cloud, third-party, and on-premises inference environments
• Implement gateway policies including routing, rate limiting, quotas, retries, failover, circuit breaking, caching, and transformations
• Harden existing services for production and manage controlled migrations without changing existing consumer contracts
• Build agent identity and registration workflows using workload/service identity
• Design and implement OpenTelemetry schemas and telemetry pipelines across agent runtimes
• Build fleet visibility for agent ownership, version, environment, health, and consumption
• Define SLI/SLOs, alerting, reliability metrics, and cost/usage attribution
• Implement gateway guardrails, content safety policies, and custom filtering integrations
• Establish secure network and integration paths for observability and third-party services
• Load test the gateway and validate production capacity
• Develop operational runbooks and technical documentation

Must Have

• 5+ years hands-on engineering across API platforms, identity, or observability
• 4+ years building and operating production API gateways
• 3+ years enterprise identity and access engineering
• Strong OAuth2, OIDC, JWT, Service Principals, Managed/Workload Identity experience
• 3+ years hands-on OpenTelemetry / Distributed Tracing
• 3+ years SRE, production engineering, or platform operations experience
• 2+ years working with LLM/AI workloads in production
• Experience with Langfuse, LangSmith, Arize, or equivalent LLM observability platforms
• Production API operations experience including failover, circuit breaking, caching, load testing, and incident response
• Enterprise networking, private connectivity, DNS, TLS, certificates, and security review experience
• Cloud infrastructure experience on at least one major hyperscaler
• Programming experience with Python, C#, or equivalent

Preferred

• Microsoft Entra ID, Managed Identity, App Registrations, and Conditional Access
• AI/ML reliability engineering experience
• Agent identity and agent-to-service authentication experience
• Chargeback/showback experience for shared platforms
• Azure AI Content Safety or custom filtering services
• Financial services or regulated-industry experience
• Agent frameworks and AI platform experience

#Hiring #SeniorPlatformEngineer #PlatformEngineer #AIGateway #AzureAPIM #Azure #APIManagement #OpenTelemetry #Observability #LLM #GenerativeAI #AgenticAI #OAuth2 #OIDC #EntraID #SRE #CloudEngineering #Python #AIInfrastructure #FinancialServices #PennsylvaniaJobs #OaksPA #TechJobs