Staff AI Inference API Engineer

Fdetimes; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Cerebras Systems is hiring a Software Engineer to evolve the ML API layer for a disaggregated AI inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtime components to deliver a consistent user experience across models and accelerators.

This hands-on role sits at the intersection of ML systems, API design, and distributed architectures, enabling new model architectures, streaming features, and tool use in production.

Qualifications

• 5+ years software engineering experience with production software or distributed systems.
• Strong Python and Go; experience with C++, Rust or similar systems language.
• Experience building stable APIs with validation, error handling, observability, compatibility, and versioning practices.

Responsibilities

• Build production ML inference APIs for chat completions, text generation, streaming, and model configuration.
• Deliver a unified serving experience across multiple backends and inference services.
• Integrate foundation models, tokenizers, prompt formats, sampling methods, and multimodal inputs.
• Maintain API compatibility and evolution with versioning, deprecation, validation, and backward compatibility.
• Integrate with model-serving runtimes such as vLLM, PyTorch, Hugging Face libraries, ROCm stack, and Cerebras runtime components.
• Support disaggregated inference via coordination of prefill and decode paths, including request routing and error handling.
• Improve streaming performance, latency, throughput, and scheduling across serving components.
• Develop validation systems for tokenization, sampling, logits, and determinism across backends.

Skills

Python

Go

C++

Rust

Distributed systems

APIs

Education

Bachelor's degree in CS/CE/EE or equivalent

Tools

vLLM

SGLang

PyTorch

Hugging Face

Transformers

Triton

TensorRT-LLM

Job description

Cerebras Systems is hiring a Software Engineer to evolve the ML API layer for a disaggregated AI inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtime components to deliver a consistent user experience across models and accelerators.

This hands-on role sits at the intersection of ML systems, API design, and distributed architectures, enabling new model architectures, streaming features, and tool use in production.