Senior ML Infra Engineer - Equity & Scalable Inference

SambaNova; Austin (TX); On-site; USD 170,000 - 240,000; Full time

Benefits offered by this job

Medical Insurance

Dependent Insurance

Health Savings Account (HSA)

Dental Insurance

Vision Insurance

Short Term Disability

Long Term Disability

Basic Life Insurance

Voluntary Life Insurance

AD&D Insurance

Flexible Spending Account (FSA)

Headspace Subscription

Gympass+ Membership

One Medical Membership

Employee Assistance Program

Equity

Job summary

SambaNova in Austin, Texas, seeks a senior engineer to design and operate production-grade inference infrastructure on our RDU architecture, focusing on performance and cost optimization while owning the public API surface.

You will build accuracy verification systems to ensure the reliability of inference features and collaborate with cross-functional teams to deliver scalable ML deployments.

Qualifications

• B.S. in Computer Science or related field
• 5+ years of experience building large-scale distributed systems
• Proficiency in Python and familiarity with modern LLM serving stacks like vLLM or TensorRT-LLM

Responsibilities

• Design and operate production-grade inference infrastructure on RDU architecture.
• Own the public API surface and build accuracy verification systems to ensure the reliability of inference features.

Skills

ML Infrastructure

Distributed Systems

LLM Inference

Python

Systems Design

vLLM

Speculative Decoding

Constrained Decoding

API Design

Concurrency

Algorithms

Data Structures

RDU Architecture

Prompt Caching

Long-context Serving

Accuracy Verification

Education

Bachelor's degree in Computer Science

Tools

vLLM

TensorRT-LLM

Job description

SambaNova in Austin, Texas, seeks a senior engineer to design and operate production-grade inference infrastructure on our RDU architecture, focusing on performance and cost optimization while owning the public API surface.

You will build accuracy verification systems to ensure the reliability of inference features and collaborate with cross-functional teams to deliver scalable ML deployments.