Staff, AI Inference Benchmarking (Equity, Onsite SF)

Artificial Analysis, Inc.; San Francisco (CA); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Equity

Job summary

Artificial Analysis, Inc. is seeking a Member of Technical Staff to own coverage of the serverless inference landscape. You will benchmark endpoints across providers, extend methodology to new dimensions, and work directly with neoclouds to understand performance and economics.

Your work will drive product direction and keep us ahead in frontier AI benchmarking. You will analyze metrics from tokens per second to price-performance frontiers, collaborating with engineers and industry leaders.

Qualifications

• 3+ years of professional experience in inference or related field.
• Experience with serverless inference and model serving stacks.
• Strong analytical and critical thinking skills.
• Proficiency in Python and data analysis.

Responsibilities

• Benchmark the serverless inference landscape across providers.
• Extend benchmarking methodology with new dimensions.
• Collaborate with providers and neoclouds to benchmark endpoints.
• Analyze throughput, latency, and cost metrics.
• Shape roadmap for the inference benchmarking platform.
• Embrace an AI-native workflow to stay ahead in AI benchmarks.

Skills

Python

Data analysis

Analytical thinking

Benchmarking

Inference market knowledge

Tools

vLLM

SGLang

TensorRT-LLM

Inference APIs

Job description

Artificial Analysis, Inc. is seeking a Member of Technical Staff to own coverage of the serverless inference landscape. You will benchmark endpoints across providers, extend methodology to new dimensions, and work directly with neoclouds to understand performance and economics.

Your work will drive product direction and keep us ahead in frontier AI benchmarking. You will analyze metrics from tokens per second to price-performance frontiers, collaborating with engineers and industry leaders.