Principal AI Infrastructure & Accelerator Architect

EPAM Systems, Inc.; United States; Remote; USD 180,000 - 220,000; Full time

Benefits offered by this job

Health Insurance

401(k) Retirement Plan

Paid Time Off

Job summary

EPAM Systems, Inc. seeks a Principal Architect to lead the architectural vision for AI training and serving infrastructure, driving performance at scale across TPU/GPU architectures and compiler toolchains.

You will spearhead the Center of Excellence, mentor teams, and collaborate with ML researchers and framework engineers to set standards and accelerate production deployment. The role entails shaping enterprise-grade benchmarking, autotuning, and tooling while tracking industry advances in

Qualifications

• 15+ years of software engineering experience, incl. distributed systems, AI infrastructure, or HPC architecture.
• 7+ years designing and developing complex software systems in C++ or Python.
• 5+ years leading architecture, design, and delivery of large-scale software products or developer ecosystems.
• 8+ years focused on AI infrastructure or HPC architecture.

Responsibilities

• Define and drive the multi-year technical roadmap for high-performance AI kernels and hardware-software co-design.
• Scale and mentor a world-class technical practice, establish governance and engineering standards.
• Liaise with ML researchers, core framework architects (JAX, PyTorch), and compiler teams (XLA, MLIR) to remove bottlenecks.
• Architect foundational infrastructure including benchmarking suites, autotuning, and regression analysis pipelines.

Skills

Distributed systems

AI infrastructure

C++

Python

Kernel-level programming

Performance optimization

Hardware-software co-design

Education

Bachelor's degree in CS/EE or equivalent

Master's or Ph.D. preferred

Tools

MLIR

OpenXLA

LLVM

Job description

Are you a visionary technical leader passionate about defining the future of AI infrastructure and squeezing every drop of performance out of advanced hardware accelerators at scale? We are seeking a Principal Architect to lead, shape, and execute our technical strategy for AI performance, optimization, and hardware-software co-design.In this elite, highly visible role, you will define the architectural vision for both AI training and serving infrastructure, delivering massive industry-wide impact. You will spearhead our Center of Excellence (CoE), scaling our practice and guiding the technical roadmap across next-generation Tensor Processing Units (TPUs), Graphics Processing Unit (GPU) fleets, state-of-the-art ML models, and advanced compiler toolchains.Your architectural decisions will directly enable cutting-edge AI research and large-scale production deployments across Google Cloud, major enterprise customers, and the broader open-source ecosystem. If you thrive on solving intractable performance bottlenecks and redefining what is physically and computationally possible in AI infrastructure, this is your platform.Req# 1067369299ResponsibilitiesDefine and drive the multi-year technical roadmap for high-performance AI kernels, custom operations, and hardware-software co-design targeting TPU and GPU architecturesScale and mentor a world-class technical practice, establishing architectural governance, engineering standards, and best practices across the organizationAct as the principal technical liaison partnering with ML researchers, core framework architects (JAX, PyTorch), and compiler engineering teams (XLA, MLIR) to eliminate systemic bottlenecks and shape future hardware/software requirementsArchitect foundational infrastructure—including enterprise-grade benchmarking suites, automated autotuning frameworks, regression analysis pipelines, and comprehensive documentation—empowering the global developer communityAnticipate industry shifts by tracking advancements in hardware architectures, emerging model topologies, and compiler innovations to unlock step-changes in AI training and inference efficiencyRequirementsBachelor's degree in Computer Science, Electrical Engineering, or equivalent practical experience (Master's or Ph.D. preferred)15+ years of software engineering experience, with 8+ years focused on distributed systems, AI infrastructure, or high-performance computing (HPC) architecture7+ years of experience designing and developing complex software systems in C++ or Python5+ years of experience leading the architecture, design, and delivery of large-scale software products, frameworks, or developer ecosystems from inception to productionProven track record of architecting performance-critical systems at the kernel level, bridging hardware accelerators and high-level software frameworksNice to haveDeep expertise in optimizing TPU/GPU execution, leveraging low-level kernel languages/abstractions such as Pallas, Mosaic, Triton, or CUDAComprehensive knowledge of modern ML frameworks (JAX, PyTorch), attention mechanisms, Mixture of Experts (MoEs), model quantization, and low-precision arithmeticAdvanced understanding of modern accelerator architectures, including heterogeneous compute, complex memory hierarchies, data movement optimization, and multi-node scale-out fabricsDeep familiarity with compiler principles, code generation, and modern toolchains such as MLIR, OpenXLA, and LLVMDemonstrated leadership in building and scaling developer infrastructure, widely adopted Open-Source Software (OSS) libraries, and extensible high-performance APIsExceptional strategic communication and stakeholder management skills, with a history of influencing cross-functional engineering teams, researchers, and executive leadershipWe offerMedical, Dental and Vision Insurance (Subsidized)Health Savings AccountFlexible Spending Accounts (Healthcare, Dependent Care, Commuter)Short-Term and Long-Term Disability (Company Provided)Life and AD&D Insurance (Company Provided)Employee Assistance ProgramUnlimited access to LinkedIn learning solutionsMatched 401(k) Retirement Savings PlanPaid Time Off – the employee will be eligible to accrue 15-25 paid days, depending on specific level and tenure with EPAM (accrual eligibility may change over time)Paid Holidays - nine (9) total per yearLegal Plan and Identity Theft ProtectionAccident InsuranceEmployee DiscountsPet InsuranceEmployee Stock Purchase ProgramIf otherwise eligible, participation in the discretionary annual bonus programIf otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) ProgramThis Remote Position Cannot be Performed in New York City.EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our clients, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.Engineer the Future with a Career at EPAMThis posting includes a good faith range of the salary EPAM would reasonably expect to pay the selected candidate. The range provided reflects base salary only. Individual compensation offers within the range are based on a variety of factors, including, but not limited to: geographic location, experience, credentials, education, training; the demand for the role; and overall business and labor market considerations. Most candidates are hired at a salary within the range disclosed. Salary range: $180,000 - $220,000. In addition, the details highlighted in this job posting above are a general description of all other expected benefits and compensation for the position.Applications will be accepted on a rolling basis.In accordance with the LA County Fair Chance Ordinance, you may find a copy of the Notice containing a summary of the Ordinance’s key provisions here: Concept FCO Posting 8 27 24 (lacounty.gov)EPAM will not provide new H-1B visa sponsorship for this position. Candidates with existing transferable H-1B status may be considered.It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.