New-Grad ML Inference Systems Engineer

SambaNova, Inc.; Austin (TX); On-site; USD 131,000 - 197,000; Full time

Benefits offered by this job

Health insurance

Dental & Vision

HSA plan

Employee well-being

Job summary

SambaNova Systems in Austin, TX, is seeking a new grad ML Infrastructure Engineer to contribute to the inference stack on our Reconfigurable Dataflow Unit (RDU). You will write production Python for the inference serving stack, integrate vLLM, and help bring speculative decoding, prompt caching, and tool calling into production.

Join a team that builds request scheduling, caching, decoding techniques, and public APIs, with tests to ensure accuracy and regression coverage, while debugging latency

Qualifications

• B.S. in CS, EE or related field
• Strong programming in Python
• Understanding of algorithms, data structures, concurrency, and systems design
• Coursework or project experience in distributed systems or ML
• Experience with LLM serving frameworks (vLLM, SGLang, TensorRT-LLM)
• Experience with inference techniques (speculative decoding, KV caching, continuous batching)
• Experience with PyTorch

Responsibilities

• Implement inference features on the RDU (speculative decoding, constrained decoding, tool calling, prompt caching, long-context inference)
• Adapt vLLM and serving frameworks to the RDU architecture
• Build and maintain endpoints in the public inference API
• Write accuracy verification and regression tests for inference features
• Debug latency, throughput, and reliability in production services

Skills

Python

Algorithms & DS

Distributed systems

Machine learning

PyTorch

Concurrency

Education

B.S. in Computer Science or Electrical Engineering

M.S. or Ph.D. in CS/EE

Tools

vLLM

SGLang

TensorRT-LLM

PyTorch

Job description

SambaNova Systems in Austin, TX, is seeking a new grad ML Infrastructure Engineer to contribute to the inference stack on our Reconfigurable Dataflow Unit (RDU). You will write production Python for the inference serving stack, integrate vLLM, and help bring speculative decoding, prompt caching, and tool calling into production.

Join a team that builds request scheduling, caching, decoding techniques, and public APIs, with tests to ensure accuracy and regression coverage, while debugging latency