Architect / Principal Software Engineer

EPAM Systems; New York (NY); On-site; USD 140,000 - 165,000; Full time

Benefits offered by this job

Medical, Dental and Vision Insurance

Health Savings Account

Flexible Spending Accounts (Healthcare

Dependent Care

Commuter

Life and AD&D Insurance

Employee Assistance Program

401(k) Retirement Savings Plan

Paid Time Off

Paid Holidays

Pet Insurance

Employee Stock Purchase Program

Annual bonus program

Long-Term Incentive Program

Job summary

EPAM Systems is seeking an Architect / Principal Software Engineer to lead the technical direction of our AI Engineering pods, splitting time between prototyping (30–40%) and high-level system architecture and mentorship across US teams.

You will design multi-agent workflows, low-latency LLM orchestration, and production-grade retrieval pipelines, collaborating with product, security, data, and infrastructure to deliver reliable and secure agentic systems.

Qualifications

• 10+ years of software engineering experience in distributed backend systems.
• 6+ years of Python experience with FastAPI, AsyncIO, and typing.
• 3+ years building production systems with LLMs (LangChain / LangGraph or similar).
• Hands-on with LangGraph stateful agent frameworks and state graphs.
• 3–4 years of React and TypeScript; review UI code and streaming APIs.
• AWS prod experience: EKS/ECS, Lambda, Bedrock, S3, Docker, Terraform or OpenTofu.
• OAuth 2.0 / OIDC authentication and secure credential handling.

Responsibilities

• Set architectural direction across AI pods with trade-offs on model capabilities, cost, latency, and complexity.
• Spend roughly a third of time coding backend AI services and integration contracts.
• Design agent workflows with streaming responses and low latency for chat and tasks.
• Establish shared standards for MCP servers, tool schemas, and context management.
• Work with security and infrastructure to enforce least-privilege tool execution and secret handling.
• Define observability requirements and SLAs with tracing tools.
• Mentor Staff and Senior engineers through design reviews.

Skills

Distributed backend systems

Python (FastAPI)

LLM production systems

LangGraph / LangChain

React & TypeScript

AWS (EKS/ECS, Lambda, S3)

OAuth 2.0 / OIDC

Tools

Docker

Terraform / OpenTofu

LangGraph

Job description

We are looking for an Architect / Principal Software Engineer to lead the technical direction of our AI Engineering pods. You will split your time between hands-on engineering (about 30% to 40% prototyping core workflows, building reference implementations, and reviewing code) and higher-level system architecture, technical standards, and mentorship across teams. You will work closely with US-based Product, Security, Data, and Infrastructure teams to make sure our agentic systems are reliable, fast, and secure. Your main focus will be designing predictable multi-agent workflows, low-latency LLM orchestration services, and retrieval pipelines that run cleanly in production. Req.#1098579355

Responsibilities

• Set architectural direction across multiple AI pods, making clear trade-offs between model capabilities, cost, latency, and operational complexity
• Spend roughly a third of your time in code: writing backend AI services, building orchestration patterns, and defining integration contracts with frontend teams
• Design agent workflows with streaming responses, predictable error handling, and low latency for chat and automated tasks
• Establish shared standards for Model Context Protocol (MCP) servers, tool schemas, and context management
• Work with Okta Security and Infrastructure to enforce least-privilege tool execution, tenant isolation, and secret handling within agent loops
• Define observability requirements and service-level objectives, including tracing with tools like LangSmith or Arize Phoenix
• Mentor Staff and Senior engineers through direct design reviews and technical guidance

Requirements

• 10+ years of software engineering experience, with a background in designing, scaling, and operating distributed backend systems
• 6+ years of Python experience, especially with FastAPI, AsyncIO, and modern typing practices. Python is our primary language for backend and AI services
• 3+ years building production systems with LLMs, including LangChain, LangGraph, AWS Bedrock, Anthropic Claude, or OpenAI APIs. Candidates with classical ML backgrounds (PyTorch, scikit-learn) who moved into generative AI architectures are welcome
• Hands-on experience with LangGraph or similar stateful agent frameworks, particularly managing state graphs, loops, checkpointing, and human-in-the-loop steps
• 3 to 4 years of React and TypeScript experience, enough to review UI code, understand frontend state, and ensure streaming APIs work cleanly with the web client
• Production AWS experience, including EKS or ECS, Lambda, Bedrock, S3, Docker, and Terraform or OpenTofu
• Authentication and security fundamentals, specifically OAuth 2.0, OIDC, and secure credential handling

Nice to have

• Practical experience building custom MCP servers and defining tool discovery patterns
• Experience migrating from older retrieval systems (such as Amazon Kendra) to vector platforms like OpenSearch, Pinecone, or Qdrant
• Experience handling multimodal data (images, documents) within LangGraph state
• Background in Identity and Access Management (IAM) or Customer Identity (CIAM)

We offer

• Medical, Dental and Vision Insurance (Subsidized)
• Health Savings Account
• Flexible Spending Accounts (Healthcare, Dependent Care, Commuter)
• Short-Term and Long-Term Disability (Company Provided)
• Life and AD&D Insurance (Company Provided)
• Employee Assistance Program
• Unlimited access to LinkedIn learning solutions
• Matched 401(k) Retirement Savings Plan
• Paid Time Off - the employee will be eligible to accrue 15-25 paid days, depending on specific level and tenure with EPAM (accrual eligibility may change over time)
• Paid Holidays - nine (9) total per year
• Legal Plan and Identity Theft Protection
• Accident Insurance
• Employee Discounts
• Pet Insurance
• Employee Stock Purchase Program
• If otherwise eligible, participation in the discretionary annual bonus program
• If otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program

For remote work in New York City only.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our clients, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Engineer the Future with a Career at EPAM

Salary range: $140,000 - $165,000. In addition, the details highlighted in this job posting above are a general description of all other expected benefits and compensation for the position.

EPAM will not provide new H-1B visa sponsorship for this position. Candidates with existing transferable H-1B status may be considered.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.