Staff Engineer - LLM Benchmark Platform

Vals AI, Inc.; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Relocation assistance

Health insurance

Dental insurance

Lunch & snacks

Job summary

Vals AI, Inc. is seeking exceptional engineers to own and scale our benchmarking platform.

You’ll work across the stack—Python/Django backend, React/TypeScript frontend, and distributed cloud infrastructure—building reliable, high-performance systems for evaluating LLMs at scale. You will drive architecture decisions, mentor teammates, and collaborate with research to ensure infrastructure meets rigorous requirements.

Qualifications

• Strong engineering fundamentals.
• Python expertise in a professional setting.
• System Design Experience: familiarity with VMs, containers, load balancers, databases.
• Familiarity with LLM APIs and concepts like temperature, tokenization, reasoning.

Responsibilities

• Build distributed systems to run evaluations across models, benchmarks, and machines at scale.
• Deploy cloud infrastructure using IaC, including deployment pipelines, servers, logging, monitoring.
• Contribute to internal/external libraries, including public model library.
• Develop full-stack features with React/TypeScript frontend and Python/Django backend.
• Perform code and architecture reviews for team members.
• Help establish engineering best practices across the organization.
• Collaborate closely with the research team to meet infrastructure needs.

Skills

Python

System design

LLMs

Distributed systems

React/TypeScript

Django

AWS

CDK

Tools

AWS

CDK

Docker

Kubernetes

Job description

Vals AI, Inc. is seeking exceptional engineers to own and scale our benchmarking platform.

You’ll work across the stack—Python/Django backend, React/TypeScript frontend, and distributed cloud infrastructure—building reliable, high-performance systems for evaluating LLMs at scale. You will drive architecture decisions, mentor teammates, and collaborate with research to ensure infrastructure meets rigorous requirements.