Software Engineer, Evaluation Infrastructure (Scale ML)

Maven Ventures; Palo Alto (CA); On-site; USD 180,000 - 440,000; Full time

Job summary

SpaceXSI is seeking a Software Engineer for Evaluation Infrastructure to help build scalable evaluation pipelines for next-gen models in a cutting-edge AI environment. You will work closely with modeling and evaluation teams to design new evals, maintain signal quality, and create internal tooling to support model training.

Responsibilities include optimizing performance across the eval stack, such as inference, capacity management, and asynchronous orchestration, while ensuring reliable,

Qualifications

• Experience writing production-quality code in Python, PyTorch, JAX, Rust, or C++.
• Experience in building and optimizing distributed systems.
• Experience in LLM inference.
• Experience in GPU compute management and dynamic resource optimization.
• Experience in developing interfaces for comparing models and evals.

Responsibilities

• Build highly reliable, efficient, and easy to use infrastructure that runs all our evaluations at scale.
• Collaborate with modeling & evaluation teams on developing new evals, maintaining evaluation signal quality, and building internal tooling to support training our models.
• Identify and resolve performance bottlenecks in all layers of the eval infrastructure stack, including inference, capacity fleet management, and asynchronous eval orchestration.

Skills

Distributed systems

LLM inference

GPU compute management

Workload scheduling

Model eval interfaces

Tools

Python

PyTorch

JAX

Rust

C++

Job description

SpaceXSI is seeking a Software Engineer for Evaluation Infrastructure to help build scalable evaluation pipelines for next-gen models in a cutting-edge AI environment. You will work closely with modeling and evaluation teams to design new evals, maintain signal quality, and create internal tooling to support model training.

Responsibilities include optimizing performance across the eval stack, such as inference, capacity management, and asynchronous orchestration, while ensuring reliable,