Senior ML API Engineer - Real-Time AI Inference

Cerebras; Sunnyvale (CA); On-site; USD 180,000 - 250,000; Full time

Job summary

Cerebras Systems is hiring a Senior Software Engineer to build the ML API layer for our disaggregated AI inference system. You will own features across inference APIs, model integration, and request routing, delivering consistent behavior across accelerators and backends.

You will work on streaming, structured outputs, multimodal inputs, and model configuration, collaborating with product, cloud, and customer teams.

Qualifications

• Bachelor's degree or equivalent practical experience in CS/CE/EE or related field.
• 5+ years of software engineering with ownership of production services or distributed systems.
• Strong programming in Python; proficient in Go, C++, Rust, or similar.
• Experience designing and delivering multi-component features and APIs.
• Experience operating services on Linux with containers and CI/CD.
• Ability to diagnose correctness, reliability, and performance across distributed systems.

Responsibilities

• Own end-to-end features for chat completions, text generation, streaming, and model configuration.
• Lead multi-component delivery with design docs, tradeoffs, and milestones.
• Ensure clear, stable API behavior across multiple backends and interfaces.
• Integrate models, tokenizers, and configuration into the serving platform.
• Coordinate with teams across model enablement, compiler, runtime, and cloud infra.
• Improve reliability with metrics, tracing, logs, alerts, and incident response.

Skills

Python

Go

C++

Rust

Distributed systems

API design

REST/gRPC

Linux & containers

Education

Bachelor's degree in Computer Science/Engineering

Equivalent practical experience

Tools

Kubernetes

Torch/Hugging Face libraries

vLLM

Job description

Cerebras Systems is hiring a Senior Software Engineer to build the ML API layer for our disaggregated AI inference system. You will own features across inference APIs, model integration, and request routing, delivering consistent behavior across accelerators and backends.

You will work on streaming, structured outputs, multimodal inputs, and model configuration, collaborating with product, cloud, and customer teams.