Senior ML Infrastructure Engineer

SambaNova; Austin (TX); On-site; USD 170,000 - 240,000; Full time

Benefits offered by this job

Medical Insurance

Dependent Insurance

Health Savings Account (HSA)

Dental Insurance

Vision Insurance

Short Term Disability

Long Term Disability

Basic Life Insurance

Voluntary Life Insurance

AD&D Insurance

Flexible Spending Account (FSA)

Headspace Subscription

Gympass+ Membership

One Medical Membership

Employee Assistance Program

Equity

Job summary

SambaNova in Austin, Texas, seeks a senior engineer to design and operate production-grade inference infrastructure on our RDU architecture, focusing on performance and cost optimization while owning the public API surface.

You will build accuracy verification systems to ensure the reliability of inference features and collaborate with cross-functional teams to deliver scalable ML deployments.

Qualifications

• B.S. in Computer Science or related field
• 5+ years of experience building large-scale distributed systems
• Proficiency in Python and familiarity with modern LLM serving stacks like vLLM or TensorRT-LLM

Responsibilities

• Design and operate production-grade inference infrastructure on RDU architecture.
• Own the public API surface and build accuracy verification systems to ensure the reliability of inference features.

Skills

ML Infrastructure

Distributed Systems

LLM Inference

Python

Systems Design

vLLM

Speculative Decoding

Constrained Decoding

API Design

Concurrency

Algorithms

Data Structures

RDU Architecture

Prompt Caching

Long-context Serving

Accuracy Verification

Education

Bachelor's degree in Computer Science

Tools

vLLM

TensorRT-LLM

Job description

Design and operate production-grade inference infrastructure on RDU architecture to optimize performance and cost. Own the public API surface and build accuracy verification systems to ensure the reliability of inference features.

Requirements

Requires a B.S. in Computer Science or a related field and over 5 years of experience building large-scale distributed systems. Proficiency in Python and familiarity with modern LLM serving stacks like vLLM or TensorRT-LLM are essential.

Key Skills

• ML Infrastructure
• Distributed Systems
• LLM Inference
• Python
• Systems Design
• vLLM
• Speculative Decoding
• Constrained Decoding
• API Design
• Concurrency
• Algorithms
• Data Structures
• RDU Architecture
• Prompt Caching
• Long-context Serving
• Accuracy Verification

Benefits

• Medical Insurance
• Dependent Insurance
• Health Savings Account (HSA)
• Dental Insurance
• Vision Insurance
• Short Term Disability
• Long Term Disability
• Basic Life Insurance
• Voluntary Life Insurance
• AD&D Insurance
• Flexible Spending Account (FSA)
• Headspace Subscription
• Gympass+ Membership
• One Medical Membership
• Employee Assistance Program
• Equity