LLM Infra Engineer: Scale, Reliability & Innovation

Scale AI, Inc.; New York (NY); On-site; USD 180,000 - 225,000; Full time

Benefits offered by this job

Comprehensive health coverage

Equity compensation

Learning and development stipend

Generous PTO

Commuter stipend

Job summary

Scale AI, Inc. in New York is looking for a Software Engineer on the ML Infrastructure team to design and build robust platforms for serving LLMs. This role involves collaborating with researchers and applying backend system design principles to launch solutions that enhance innovation.

The ideal candidate will possess experience in developing high-performance systems while ensuring scalability and reliability. Benefits include comprehensive health coverage, equity compensation, and generous PTO.

Qualifications

• 4+ years of experience building large-scale, high-performance backend systems.
• Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++).
• Experience with LLM serving and routing fundamentals.
• Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code.

Responsibilities

• Build and maintain fault-tolerant, high-performance systems for serving LLMs workloads at scale.
• Collaborate with researchers and engineers to integrate and optimize models.
• Conduct architecture and design reviews to ensure best practices.
• Develop monitoring solutions to ensure system health.

Skills

Backend Systems

Python

Go

Rust

C++

Containers

Cloud Infrastructure

Terraform

Tools

Docker

Kubernetes

Job description

Scale AI, Inc. in New York is looking for a Software Engineer on the ML Infrastructure team to design and build robust platforms for serving LLMs. This role involves collaborating with researchers and applying backend system design principles to launch solutions that enhance innovation.

The ideal candidate will possess experience in developing high-performance systems while ensuring scalability and reliability. Benefits include comprehensive health coverage, equity compensation, and generous PTO.