Principal AI Infrastructure & Accelerator Architect

EPAM Systems; United States; Remote; USD 180,000 - 220,000; Full time

Benefits offered by this job

Medical Insurance

Dental & Vision Insurance

401(k) Matching

Paid Time Off

Job summary

EPAM Systems in the United States seeks a Principal Architect to define the architectural vision for high-performance AI training and serving infrastructure, focusing on TPU and GPU architectures and hardware-software co-design.

You will lead the Center of Excellence, mentor teams, and drive multi-year roadmaps that enable scalable AI across Google Cloud, enterprise customers, and open-source ecosystems. Strong leadership and kernel-level design experience are required.

Qualifications

• BS in Computer Science, Electrical Engineering, or equivalent practical experience (MS/Ph.D. preferred).
• 15+ years of software engineering experience, 8+ years in distributed systems/AI infrastructure/HPC architecture.
• 7+ years designing and developing complex software in C++ or Python.
• 5+ years leading architecture and delivery of large-scale software products or ecosystems.
• Proven kernel-level performance-focused architecture bridging hardware and software.

Responsibilities

• Define multi-year roadmap for high-performance AI kernels and hardware-software co-design for TPU/GPU architectures.
• Scale and mentor a technical practice with governance, standards, and best practices.
• Collaborate with ML researchers, framework architects (JAX, PyTorch), and compiler teams (XLA, MLIR).
• Architect foundational infrastructure incl. benchmarking, autotuning, regression pipelines, and docs.
• Track hardware/ML trends to unlock efficiency in training and inference.

Skills

Distributed systems

AI infrastructure

C++/Python

Architecture leadership

Hardware-software co-design

Education

Bachelor's degree in CS/EE

MS/PhD preferred

Tools

JAX

PyTorch

XLA

MLIR

CUDA

Job description

Are you a visionary technical leader passionate about defining the future of AI infrastructure and squeezing every drop of performance out of advanced hardware accelerators at scale? We are seeking a Principal Architect to lead, shape, and execute our technical strategy for AI performance, optimization, and hardware-software co-design.

In this elite, highly visible role, you will define the architectural vision for both AI training and serving infrastructure, delivering massive industry-wide impact. You will spearhead our Center of Excellence (CoE), scaling our practice and guiding the technical roadmap across next-generation Tensor Processing Units (TPUs), Graphics Processing Unit (GPU) fleets, state-of-the-art ML models, and advanced compiler toolchains.

Your architectural decisions will directly enable cutting-edge AI research and large-scale production deployments across Google Cloud, major enterprise customers, and the broader open-source ecosystem. If you thrive on solving intractable performance bottlenecks and redefining what is physically and computationally possible in AI infrastructure, this is your platform.

Req# 1067369299

Responsibilities

• Define and drive the multi-year technical roadmap for high-performance AI kernels, custom operations, and hardware-software co-design targeting TPU and GPU architectures
• Scale and mentor a world-class technical practice, establishing architectural governance, engineering standards, and best practices across the organization
• Act as the principal technical liaison partnering with ML researchers, core framework architects (JAX, PyTorch), and compiler engineering teams (XLA, MLIR) to eliminate systemic bottlenecks and shape future hardware/software requirements
• Architect foundational infrastructure— including enterprise-grade benchmarking suites, automated autotuning frameworks, regression analysis pipelines, and comprehensive documentation—empowering the global developer community
• Anticipate industry shifts by tracking advancements in hardware architectures, emerging model topologies, and compiler innovations to unlock step-changes in AI training and inference efficiency

Requirements

• Bachelor's degree in Computer Science, Electrical Engineering, or equivalent practical experience (Master's or Ph.D. preferred)
• 15+ years of software engineering experience, with 8+ years focused on distributed systems, AI infrastructure, or high-performance computing (HPC) architecture
• 7+ years of experience designing and developing complex software systems in C++ or Python
• 5+ years of experience leading the architecture, design, and delivery of large-scale software products, frameworks, or developer ecosystems from inception to production
• Proven track record of architecting performance-critical systems at the kernel level, bridging hardware accelerators and high-level software frameworks

Nice to have

• Deep expertise in optimizing TPU/GPU execution, leveraging low-level kernel languages/abstractions such as Pallas, Mosaic, Triton, or CUDA
• Comprehensive knowledge of modern ML frameworks (JAX, PyTorch), attention mechanisms, Mixture of Experts (MoEs), model quantization, and low-precision arithmetic
• Advanced understanding of modern accelerator architectures, including heterogeneous compute, complex memory hierarchies, data movement optimization, and multi-node scale-out fabrics
• Deep familiarity with compiler principles, code generation, and modern toolchains such as MLIR, OpenXLA, and LLVM
• Demonstrated leadership in building and scaling developer infrastructure, widely adopted Open-Source Software (OSS) libraries, and extensible high-performance APIs
• Exceptional strategic communication and stakeholder management skills, with a history of influencing cross-functional engineering teams, researchers, and executive leadership

We offer

• Medical, Dental and Vision Insurance (Subsidized)
• Health Savings Account
• Flexible Spending Accounts (Healthcare, Dependent Care, Commuter)
• Short-Term and Long-Term Disability (Company Provided)
• Life and AD&D Insurance (Company Provided)
• Employee Assistance Program
• Unlimited access to LinkedIn learning solutions
• Matched 401(k) Retirement Savings Plan
• Paid Time Off – the employee will be eligible to accrue 15-25 paid days, depending on specific level and tenure with EPAM (accrual eligibility may change over time)
• Paid Holidays - nine (9) total per year
• Legal Plan and Identity Theft Protection
• Accident Insurance
• Employee Discounts
• Pet Insurance
• Employee Stock Purchase Program
• If otherwise eligible, participation in the discretionary annual bonus program
• If otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program

This Remote Position Cannot be Performed in New York City.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our clients, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Engineer the Future with a Career at EPAM

This posting includes a good faith range of the salary EPAM would reasonably expect to pay the selected candidate. The range provided reflects base salary only. Individual compensation offers within the range are based on a variety of factors, including, but not limited to: geographic location, experience, credentials, education, training; the demand for the role; and overall business and labor market considerations. Most candidates are hired at a salary within the range disclosed. Salary range: $180,000 - $220,000. In addition, the details highlighted in this job posting above are a general description of all other expected benefits and compensation for the position.

In accordance with the LA County Fair Chance Ordinance, you may find a copy of the Notice containing a summary of the Ordinance’s key provisions here: Concept FCO Posting 8 27 24 (lacounty.gov)

EPAM will not provide new H-1B visa sponsorship for this position. Candidates with existing transferable H-1B status may be considered.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.