Staff Engineer, AI Inference Benchmarking

Fdetimes; San Francisco (CA); Presencial; USD 120 000 - 180 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Equity

Competitive compensation

Resumo da oferta

Artificial Analysis, based in San Francisco, is hiring a Member of Technical Staff to lead coverage of the serverless inference landscape. You will benchmark endpoints across providers, extend benchmarking dimensions, and collaborate directly with neoclouds to refine methodologies and measurements.

You will analyze market dynamics, productivity metrics, and cost-per-token economics, helping set industry standards while contributing to the roadmap of our inference benchmarking platform and

Qualificações

• 3+ years of professional experience, including at least 2 years at or with inference providers, neoclouds or production deployments.

Responsabilidades

• Benchmark the Inference Landscape: Own performance and price coverage of serverless API inference across the provider ecosystem.
• Extend Our Methodology: Drive new benchmarking dimensions including cached pricing, endpoint accuracy and agentic performance.
• Partner with Providers: Work directly with inference providers and neoclouds to benchmark their endpoints and shape measurement.
• Analyze the Market: Produce analysis on throughput, time-to-first-token, and price-performance frontiers.
• Drive Product Direction: Shape the roadmap of our inference benchmarking platform with engineers.
• Become AI-Native: Use cutting-edge AI tools to stay competitive in AI benchmarking.

Conhecimentos

Analytical thinking

Python

Data analysis

Critical thinking

Ferramentas

Model serving stack: vLLM

SGLang

TensorRT-LLM

Descrição da oferta de emprego

Artificial Analysis, based in San Francisco, is hiring a Member of Technical Staff to lead coverage of the serverless inference landscape. You will benchmark endpoints across providers, extend benchmarking dimensions, and collaborate directly with neoclouds to refine methodologies and measurements.

You will analyze market dynamics, productivity metrics, and cost-per-token economics, helping set industry standards while contributing to the roadmap of our inference benchmarking platform and