Senior Software Engineer, ML API – AI Inference

Cerebras; United States; Remote; USD 160,000 - 260,000; Full time

Job summary

Cerebras Systems, a leader in AI hardware, seeks a Senior Software Engineer to shape the ML API layer of our disaggregated inference stack. You will own features across chat completions, text generation, streaming, and model integration, delivering a reliable, scalable experience.

This hands-on role requires collaboration with model enablement, compiler, runtime, cloud, product, and customers, translating evolving capabilities into robust designs and production-ready code.

Qualifications

• Hands-on experience designing and implementing ML inference APIs.
• Proficiency in distributed systems and scalable services.
• Experience translating model capabilities into production features.

Responsibilities

• Own end-to-end features for chat, text generation, streaming, and model integration.
• Lead multi-component delivery with cross-team collaboration.
• Ensure consistent API behavior across backends and maintain backward compatibility.
• Enable new models and capabilities by integrating tokenizers, templates, and configurations.

Skills

API design

Distributed systems

ML systems

Hands-on coding

Job description

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

About the Role

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes this heterogeneous serving system accessible, reliable, and easy to use. You will own significant features and components across our inference APIs, model integration layer, and request-routing services, delivering a consistent experience across models and accelerator backends. This role sits at the intersection of machine learning systems, API design, model serving, and distributed systems. You will translate evolving model capabilities and customer requirements into clear designs, lead implementation across multiple components, and carry features through validation, rollout, and production operation. Your work will help ensure that streaming, sampling, tool use, structured outputs, multimodal inputs, and model configuration behave correctly and consistently. You will work closely with model enablement, compiler, runtime, cloud infrastructure, product, and customer-facing teams, as well as customers directly. This is a hands-on individual-contributor role with opportunities to lead small project teams, mentor engineers, and improve the quality and reliability of production inference software.

Responsibilities

• Own features end to end. Design, implement, test, and operate features for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference capabilities.
• Lead multi-component delivery. Write clear design documents, evaluate technical tradeoffs, break projects into deliverable milestones, and coordinate implementation with a small group of engineers and partner teams.
• Deliver consistent API behavior. Implement clear request and response semantics across GPU prefill, Cerebras decode, and other inference backends. Maintain compatibility with widely adopted inference interfaces and evolve owned APIs through validation, versioning, and deprecation best practices.
• Enable new models and capabilities. Integrate foundation models, tokenizers, chat templates, generation configuration, sampling methods, and model-specific feature