AI Engineer, Model Quality and Performance

Cerebras; Sunnyvale (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

Cerebras is looking for an individual to own model quality and performance of inference offerings in Sunnyvale, California. This role involves designing eval suites with AI agents, building custom evaluations for clients, automating eval execution, and synthesizing quality and performance data into user-friendly views. The ideal candidate should have experience with AI agents, strong math and statistics background, and comfort with tools like Docker and Git. Cerebras is committed to diversity and creating an inclusive environment.

Qualifications

• Experience shipping real systems using AI agents.
• Ability to build automated eval executions end-to-end.
• Familiar with performance tuning on silicon, GPUs, or FPGAs.

Responsibilities

• Design eval suites with AI agents.
• Build custom evals for target customers.
• Automate eval execution with AI-driven pipelines.
• Forecast and benchmark model performance for top customers.
• Build tooling that integrates quality and performance data.

Skills

Experience building AI agents

Strong math/stats background

Comfort with Docker, Git, and the standard automation stack

Taste for tooling design

Job description

Cerebras' current customers include top model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

Thanks to the groundbreaking wafer-scale architecture, Cerebras Inference offers the fastest Generative AI inference solution in the world, over 10 times faster than GPU-based hyperscale cloud inference services.

About The Role

You'll own model quality and performance for Cerebras' inference offerings. You will define what "good" looks like across the models we serve, building AI-driven systems to measure it at scale, and translating those signals into artifacts our customers and product team actually use.

You'll use AI agents to spin up custom eval suites per customer use case, mine trajectories for representative test data, automate the repetitive parts of release qual, and help build performance datasets and benchmarking workflows for customer use cases. We want someone whose first instinct is "how do I get an AI agent to do this on a loop."

You'll sit between engineering, product, and customer-facing teams.

What You'll Do

• Design eval suites with AI agents in the loop. For every model release, curate a thoughtful mix of advanced, basic, long-context, and customer-use-case-specific evals. Use Claude to generate, validate, and prune candidate test cases at speed.
• Build custom evals for target customers by orchestrating AI agents to mine trajectories from their workloads and synthesize representative eval sets.
• Automate eval execution end-to-end with AI-driven pipelines on top of standard tooling (Docker, Git, CI). The goal is a system that runs itself between releases, not a script you re-run by hand.
• Build automations to forecast and benchmark model performance on Cerebras for our top customers, including modeling how fast customer-specific workloads will run in production.
• Build product-quality tooling that synthesizes quality + performance data into a single, easy-to-use view.

Skills & Qualifications

• Experience building AI agents. You ship real systems with Claude (or equivalent) as a force multiplier. You've built things that would have been infeasible solo without AI agents in the loop.
• Strong math/stats background.
• Comfort with Docker, Git, and the standard automation stack
• A taste for tooling design. You've shipped something that a non-engineer used without complaining. Bonus if AI helped you ship it.

Assets

• Performance-tuning experience on custom silicon, GPUs, or FPGAs.
• Experience designing evals for agentic / coding / long-context / multimodal use cases.
• Familiarity with open-source eval frameworks (EvalScope, lm-eval-harness, etc.).
• Experience building AI agents.

Cerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them.