Senior AI Inference Platform Engineer

SambaNova; San Jose (CA); On-site; USD 144,000 - 189,000; Full time

Benefits offered by this job

Health insurance

HSA

Gympass+

Headspace subscription

Job summary

SambaNova Systems in San Jose, California, is seeking a Principal Cloud Platform Engineer to own the reliability and scalability of our AI inference service across regions. You will bridge software development and operations, shaping uptime, latency and resource usage in our on-prem/cloud deployments.

You will build monitoring, auto-scaling, and IaC pipelines while leading on-call incident response and collaborating with finance on cloud spend.

Qualifications

• BS in Computer Science, Computer Engineering, or related field
• 3+ years of experience in a Site Reliability Engineering, DevOps
• Experience supporting a large-scale, customer-facing service in a public cloud environment (AWS, GCP, Azure)
• Strong programming and scripting skills in languages like Python, Go, Rust, or Java
• Proven experience with containerization and orchestration technologies (Docker and Kubernetes)
• Deep understanding of monitoring and observability principles and tools (e.g., Prometheus, Grafana, ELK Stack, Datadog)
• Experience with Infrastructure as Code (e.g., Terraform, CloudFormation)
• Experience with CI/CD principles and tools (e.g., Jenkins, GitHub Actions, ArgoCD)
• Strong Linux/Unix system administration fundamentals

Responsibilities

• Shared ownership of the production inferencing service across regions
• Standing-up and automating AI infrastructure in new regions
• Participating in a shared primary/secondary on-call rotation, and leading incident response
• Building monitoring, alerting, and dashboards in Prometheus, Grafana, and Datadog
• Finding and eliminating performance bottlenecks
• Designing auto-scaling policies that handle variable inference loads
• Managing cloud and on-prem infrastructure as code in Terraform and Ansible
• Building CI/CD pipelines that safely deploy new model versions and service updates
• Forecasting infrastructure needs against the product roadmap and usage trends
• Defining and reporting on SLOs and SLIs for the inferencing platform

Skills

Programming in Python/Go/Rust/Java

Linux/Unix administration

Monitoring & observability

CI/CD practices

Cloud platforms (AWS/GCP/Azure)

Education

BS in CS/CE or related field

Tools

Docker

Kubernetes

Terraform

Ansible

Prometheus

Grafana

Datadog

Jenkins

GitHub Actions

ArgoCD

CloudFormation

Job description

SambaNova Systems in San Jose, California, is seeking a Principal Cloud Platform Engineer to own the reliability and scalability of our AI inference service across regions. You will bridge software development and operations, shaping uptime, latency and resource usage in our on-prem/cloud deployments.

You will build monitoring, auto-scaling, and IaC pipelines while leading on-call incident response and collaborating with finance on cloud spend.