Senior ML Platform Engineer — Automate RL Environments

Preference Model; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Competitive cash and equity

Ownership and autonomy

Health, vision, dental

401K match

Lunch onsite

Visa sponsorship & relocation

Job summary

Preference Model is seeking a senior ML Platform Engineer in San Francisco to own and scale the environments that power our RL research stack. You will lead integrations, packaging, and automated validation to ensure labs receive reliable environments for training capable, aligned superintelligences.

You will build and maintain Karotte, our open-source framework, and manage the platform that runs environments on CPU, GPU, and TPU while delivering them to partner labs with minimal hand work.

Qualifications

• 5+ years of experience building and owning ML Platform Engineering, ideally AI related ML Platform Engineering.
• Strong software engineering fundamentals and a track record of building and operating production infrastructure or developer platforms that other engineers depend on.
• Hands-on experience with containers and orchestration (Docker, Kubernetes, or similar) and at least one major cloud platform (GCP, AWS, or similar).
• A working understanding of how to run untrusted code safely: isolation, resource limits, and what happens when a process fills the disk or refuses to exit.
• You've owned something tedious end to end, such as a migration, a release process, or a delivery pipeline, and automated away as much of it as you could.
• Extensive hands-on use of coding agents (Claude Code, Cursor, Codex, or similar), and an intuition for where models cut corners.
• Proficiency in Python, including writing libraries and APIs that other people build on.

Responsibilities

• Own the integrations that deliver our environments to partner labs: build container images, match each lab's submission format, and automate the checks that confirm requirements are met before anything ships.
• Build and maintain Karotte, our open-source environment framework, from API design through releases and migrations that keep existing environments working.
• Run and scale the infrastructure that builds, runs, and grades environments, including containerized rollouts and grading across CPU, GPU, and TPU, VM-based sandboxing, and the resource limits and isolation that stop a model from gaming its own grader.
• Build the internal tools environment builders use every day (CLI, docs, data catalog, access controls, and run monitoring) so they spend their time on environment design instead of infrastructure.
• Partner with environment builders and post-training researchers to turn recurring pain points into platform features, and ship quickly in response to their feedback.

Skills

Python

ML Platform Engineering

Containers

Kubernetes

Cloud Platforms (GCP / AWS)

Tools

Docker

Kubernetes

GCP

AWS

Job description

Preference Model is seeking a senior ML Platform Engineer in San Francisco to own and scale the environments that power our RL research stack. You will lead integrations, packaging, and automated validation to ensure labs receive reliable environments for training capable, aligned superintelligences.

You will build and maintain Karotte, our open-source framework, and manage the platform that runs environments on CPU, GPU, and TPU while delivering them to partner labs with minimal hand work.