Staff AI Inference API Engineer

Cerebras; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Cerebras Systems is hiring a Software Engineer to build and evolve the ML API layer for its disaggregated inference systems. You will work across our inference APIs, model integration layer, request-routing services and Cerebras inference platform to deliver a consistent experience across models and accelerator backends.

This hands-on software engineering role sits at the intersection of machine learning systems, API design, model serving, and distributed systems.

Qualifications

• 5+ years of software engineering experience with production software or distributed systems.
• Strong programming ability in Python and Go plus experience with performance-sensitive services in C++, Rust, or similar.
• Experience building stable APIs with validation, error handling, observability, compatibility, and versioning.

Responsibilities

• Build production ML inference APIs for chat completions, text generation, streaming, and model configuration.
• Deliver a unified serving experience across GPU prefill, Cerebras decode, and other backends.
• Enable new models and capabilities by integrating foundation models, tokenizers, prompt formats, and sampling methods.
• Own API compatibility and evolution with clear versioning, deprecation, and backward compatibility practices.
• Integrate with model-serving runtimes (vLLM, PyTorch, Hugging Face, ROCm) and Cerebras components.
• Improve serving performance including streaming, latency, throughput, and batching.
• Develop testing, conformance, and qualification infrastructure and tools.

Skills

Python

Go

C++

Rust

Education

Bachelor's degree in CS/CE/EE

Tools

vLLM

PyTorch

Hugging Face

Job description

Cerebras Systems is hiring a Software Engineer to build and evolve the ML API layer for its disaggregated inference systems. You will work across our inference APIs, model integration layer, request-routing services and Cerebras inference platform to deliver a consistent experience across models and accelerator backends.

This hands-on software engineering role sits at the intersection of machine learning systems, API design, model serving, and distributed systems.