New Grad ML Infra Engineer - Inference Serving on RDU

sambanovasystems; Austin (TX); Sur place; USD 131 000 - 197 000; Plein temps

Avantages offerts par ce poste

Health insurance

Dependent coverage

HSA with employer contribution

Dental

Vision

Short/Long-term Disability

Life insurance

AD&D insurance

FSA options

Headspace subscription

Gympass+ membership

One Medical membership

Employee Assistance Program

Résumé du poste

SambaNova Systems is seeking a new grad ML Infrastructure Engineer to join our AI inference stack in Austin. You will write production Python for our vLLM-based serving stack, integrate features like speculative decoding and tool calling, and build robust APIs on the Reconfigurable Dataflow Unit platform.

You will adapt serving frameworks to the RDU architecture, write tests for accuracy and regression, and diagnose latency and reliability in production services to ensure scalable,

Qualifications

• B.S. in Computer Science, Electrical Engineering, or related field.
• Strong programming skills in Python.
• Understanding of algorithms, data structures, concurrency, and systems design.
• Coursework or project experience in distributed systems or machine learning.
• Experience with LLM serving frameworks such as vLLM, SGLang, or TensorRT-LLM.
• Experience with inference techniques such as speculative decoding, KV caching, or continuous batching.
• Experience with PyTorch.

Responsabilités

• Implement inference features on the RDU such as speculative decoding, constrained decoding, tool calling, prompt caching, and long-context inference.
• Adapt vLLM and related serving frameworks to the RDU architecture.
• Build and maintain endpoints in the public inference API.
• Write accuracy verification and regression tests for inference features.
• Debug latency, throughput, and reliability issues in production services.

Connaissances

Python

Algorithms

Concurrency

Systems design

Distributed systems

ML experience

Formation

B.S. in Computer Science, Electrical Engineering, or related field

M.S. or Ph.D. in Computer Science, Electrical Engineering, or related field

Outils

vLLM

SGLang

TensorRT-LLM

PyTorch

Description du poste

SambaNova Systems is seeking a new grad ML Infrastructure Engineer to join our AI inference stack in Austin. You will write production Python for our vLLM-based serving stack, integrate features like speculative decoding and tool calling, and build robust APIs on the Reconfigurable Dataflow Unit platform.

You will adapt serving frameworks to the RDU architecture, write tests for accuracy and regression, and diagnose latency and reliability in production services to ensure scalable,