Senior AI Infrastructure Engineer

EPAM Systems; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

EPAM Systems is seeking a Senior AI Infrastructure Engineer to design and operate scalable infrastructure for enterprise AI/ML workloads. You will manage containerized applications with Kubernetes, drive automation, and ensure reliability across cloud environments.

The role emphasizes observability, incident response, and platform engineering best practices. Candidates should have 5+ years in infrastructure roles, strong Linux and scripting skills, and hands-on experience with cloud platforms,

Qualifications

• Bachelor's degree in Engineering or related field with 5+ years of relevant experience.
• Hands-on Kubernetes, Docker, and containerized application management.
• Experience on at least one major cloud platform (Azure, AWS, or GCP).
• Proficiency in scripting and automation using Python, Shell/Bash, or similar.
• Familiarity with monitoring/observability tools such as Prometheus, Grafana, ELK/OpenSearch, or Datadog.
• Strong Linux systems administration and platform operations knowledge.
• Background in AI/ML production workloads and reliability engineering.
• Experience with service mesh (Istio/Linkerd) and API gateways, plus networking basics.

Responsibilities

• Design, build, operate, and optimize scalable infrastructure supporting enterprise AI/ML platforms and workloads.
• Manage and maintain containerized applications and orchestration platforms for high availability.
• Provide operational support for mission-critical production environments.
• Troubleshoot infrastructure, platform, application, and container issues.
• Perform log analysis, RCA, and incident management.
• Implement and maintain monitoring, logging, and observability across platforms.
• Enable AI/ML teams through automation, tooling, and efficient infrastructure practices.
• Collaborate on networking configurations including load balancing, ingress, and traffic management.
• Apply cloud-native architecture patterns and platform engineering best practices.

Skills

Kubernetes

Docker

Python scripting

Linux admin

Observability

SRE practices

Networking basics

Cloud platforms

Automation

Education

Bachelor's degree in Engineering (B.E/B.Tech) or related field

Tools

Terraform

Helm

Prometheus

Grafana

ELK/OpenSearch

Datadog

Istio/Linkerd

Job description

We are looking for a highly skilled Senior AI Infrastructure Engineer to design, build, operate, and optimize scalable infrastructure that supports enterprise AI/ML platforms and workloads. The ideal candidate will have deep expertise in cloud-native technologies, Kubernetes, observability, automation, and platform reliability, with a strong focus on enabling AI/ML teams through robust and efficient infrastructure.

Responsibilities

• Design, build, operate, and optimize scalable infrastructure supporting enterprise AI/ML platforms and workloads
• Manage and maintain containerized applications and orchestration platforms to ensure high availability and performance
• Provide operational support for mission-critical production environments
• Troubleshoot infrastructure, platform, application, and container orchestration issues
• Perform log analysis, root cause analysis (RCA), and incident management
• Implement and maintain monitoring, logging, and observability solutions across platforms
• Enable AI/ML teams through robust automation, tooling, and efficient infrastructure practices
• Optimize system design for scalability, performance, and reliability
• Collaborate on networking configurations, including load balancing, ingress, and traffic management
• Apply cloud-native architecture patterns and platform engineering best practices across environments

Requirements

• Bachelor's degree in Engineering (B.E/B.Tech) or a related field with 5+ years of relevant experience
• Hands-on expertise in Kubernetes, Docker, and containerized application management
• Experience managing infrastructure on at least one major cloud platform (Azure, AWS, or GCP)
• Proficiency in scripting and automation using Python, Shell/Bash, or similar languages
• Familiarity with monitoring, logging, and observability tools such as Prometheus, Grafana, and ELK/OpenSearch or Datadog
• Strong understanding of Linux systems administration and platform operations
• Knowledge of system design, scalability, performance optimization, and reliability engineering
• Capability to perform troubleshooting, log analysis, root cause analysis (RCA), and incident management
• Understanding of networking fundamentals, including DNS, load balancing, and ingress and security concepts
• Background in supporting AI/ML platforms or GPU-based workloads in production environments
• Expertise in advanced networking concepts such as service mesh (Istio/Linkerd), network policies, and API gateways
• Skills in cloud-native architecture patterns combined with strong analytical and troubleshooting abilities
• English proficiency at an Upper-Intermediate level (B2) or higher
• Nice to have
• Familiarity with SRE practices (SLIs, SLOs, error budgets, disaster recovery)
• Hands-on experience with Infrastructure as Code tools such as Terraform and Helm Charts
• Experience in cloud cost monitoring and optimization for high-compute AI workloads