AI Systems Architect & Principal Engineer

EPAM Systems; New York (NY); On-site; USD 140,000 - 165,000; Full time

Benefits offered by this job

Medical, Dental and Vision Insurance

Health Savings Account

Flexible Spending Accounts (Healthcare

Dependent Care

Commuter

Life and AD&D Insurance

Employee Assistance Program

401(k) Retirement Savings Plan

Paid Time Off

Paid Holidays

Pet Insurance

Employee Stock Purchase Program

Annual bonus program

Long-Term Incentive Program

Job summary

EPAM Systems is seeking an Architect / Principal Software Engineer to lead the technical direction of our AI Engineering pods, splitting time between prototyping (30–40%) and high-level system architecture and mentorship across US teams.

You will design multi-agent workflows, low-latency LLM orchestration, and production-grade retrieval pipelines, collaborating with product, security, data, and infrastructure to deliver reliable and secure agentic systems.

Qualifications

• 10+ years of software engineering experience in distributed backend systems.
• 6+ years of Python experience with FastAPI, AsyncIO, and typing.
• 3+ years building production systems with LLMs (LangChain / LangGraph or similar).
• Hands-on with LangGraph stateful agent frameworks and state graphs.
• 3–4 years of React and TypeScript; review UI code and streaming APIs.
• AWS prod experience: EKS/ECS, Lambda, Bedrock, S3, Docker, Terraform or OpenTofu.
• OAuth 2.0 / OIDC authentication and secure credential handling.

Responsibilities

• Set architectural direction across AI pods with trade-offs on model capabilities, cost, latency, and complexity.
• Spend roughly a third of time coding backend AI services and integration contracts.
• Design agent workflows with streaming responses and low latency for chat and tasks.
• Establish shared standards for MCP servers, tool schemas, and context management.
• Work with security and infrastructure to enforce least-privilege tool execution and secret handling.
• Define observability requirements and SLAs with tracing tools.
• Mentor Staff and Senior engineers through design reviews.

Skills

Distributed backend systems

Python (FastAPI)

LLM production systems

LangGraph / LangChain

React & TypeScript

AWS (EKS/ECS, Lambda, S3)

OAuth 2.0 / OIDC

Tools

Docker

Terraform / OpenTofu

LangGraph

Job description

EPAM Systems is seeking an Architect / Principal Software Engineer to lead the technical direction of our AI Engineering pods, splitting time between prototyping (30–40%) and high-level system architecture and mentorship across US teams.

You will design multi-agent workflows, low-latency LLM orchestration, and production-grade retrieval pipelines, collaborating with product, security, data, and infrastructure to deliver reliable and secure agentic systems.