Member of Technical Staff

NextToken; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Stock options

Comprehensive health plan

Gym stipend

Job summary

NextToken is seeking a Member of Technical Staff in San Francisco to build and scale production AI systems. You will own end-to-end evaluation pipelines, harnesses for agents, post-training workflows, and interfaces that turn experimental results into deployable improvements.

The role emphasizes model-agnostic reasoning, metric-driven development, and hands-on work with leading AI tooling to optimize latency, cost, and quality at scale.

Qualifications

• 3+ years building production AI systems.
• Experience building eval harnesses from scratch.
• Ability to reason about prompts, context and models.
• Proactive problem solver with high initiative.

Responsibilities

• Own Evals as a first-class system, turning traces into datasets and graders.
• Develop agentic harnesses and environments for reproducible evaluation.
• Build post-training infrastructure for SFT/RL runs and deployment.
• Create reward signals and verifiers to enable hill-climbing improvements.
• Address latency and cost with efficient model routing and batching.

Skills

Production AI systems

Eval harnesses

Metrics-driven

Model-agnostic thinking

High agency

AI-native tooling

Claude/Cursor/Codex familiarity

Tools

Claude

Cursor

Codex

Job description

All roles

EngineeringSan Francisco

• Full-time

Member of Technical Staff

We turn the experience generated by production agents into evaluations, reward signals, and specialized models that improve continuously. The systems that make that loop real are yours to build and to raise the ceiling on: harnesses, pipelines, serving, and the agents themselves.

What you'll own

• Evals as a first-class system. Turn production traces into golden datasets, automated graders, and regression tests. Every change ships with a number.
• Agentic harnesses and environments: the tooling that runs agents reproducibly for evaluation, data generation, and RL.
• Post-training infrastructure. Data pipelines, SFT and RL runs, and the plumbing that takes a specialized model from experiment to deployment.
• Reward signals and verifiers. Make fuzzy notions of quality measurable and hill-climbable.
• Latency and cost. Model routing, caching, parallel tool calls, and serving specialized models economically alongside frontier ones.

About You

• 3+ years building production AI systems. You've shipped real systems to real users.
• Numbers over opinions. You've built eval harnesses from scratch and know the difference between a metric that hill-climbs and a metric that lies.
• Model-agnostic. You reason about prompt vs. context vs. model changes and know when each one wins.
• High agency. You find the highest-leverage problem and ship the fix without waiting for a spec.
• AI-native. You use Claude, Cursor, and Codex daily and have earned opinions on where they shine and where they mislead.

Bonus

• Hands-on with post-training: SFT, DPO, RLHF/RLAIF, LoRA.
• Built RL environments, verifier models, or synthetic data pipelines.
• Inference-level work: speculative decoding, KV cache, batching.
• Published research, OSS, or technical writing on agents, evals, or inference.
• 0-to-meaningful-scale experience with an AI product.