Senior AI Systems Engineer — Synthetic Data & Evaluation

Tonic.ai; San Francisco (CA); Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive salary and equity

Unlimited paid time off

401k plan with employer contribution

Medical, dental, and vision insurance

Generous parental leave policy

Remote-friendly work environment

Job summary

Tonic builds the data infrastructure behind modern AI. We create synthetic environments and de-identify real enterprise data for safe training and evaluation.

This role designs systems to generate longitudinally coherent synthetic environments for agent training, including persona modeling and verifiable ground truth. You will train and improve NER models, scale synthesis models, and build evaluation infrastructure to distinguish frontier models on real tasks.

Qualifications

• 8+ years (or PhD with 3+ years) building production ML systems, with real depth in some combination of LLMs, agents, RL, NER, or information extraction.
• Hands-on experience training and shipping models to production, and a pragmatic bar for quality: you know how to measure it, where it breaks, and when it's good enough to ship.
• Experience with generative or synthesis models where output fidelity and downstream utility both matter, not just plausibility.
• Strong software engineering fundamentals. You write code others build on.
• Fluency with modern training and eval stacks (PyTorch, distributed training, standard agent and benchmark frameworks).
• Comfort working with messy, sensitive, real-world data and the privacy constraints that come with it.
• A track record of framing ambiguous problems and driving them to measurable, shipped results.
• Bonus: synthetic data generation, data privacy or de-identification, or benchmark construction.

Responsibilities

• Design and build the systems that generate longitudinally coherent synthetic environments for agent training and evaluation, including persona modeling, task generators, and verifiable ground truth.
• Build and maintain synthesis models that generate realistic replacement values at very large scale, preserving format, statistical distribution, and semantic consistency so de-identified data stays useful downstream.
• Train and improve the NER models behind our entity detection, driving accuracy and recall across free text, structured fields, and mixed enterprise data at scale.
• Build evaluation infrastructure that grades agent outcomes, not just traces, and produces real discrimination between frontier models on real tasks.
• Fine-tune and evaluate open-weight models on Tonic-generated data, and turn benchmark results into product and research direction.
• Expand coverage into new domains, languages, and entity types, and handle the long tail of formats and edge cases that real customer data throws off.
• Own model evaluation across the board: precision and recall on detection, utility preservation on synthesis, and outcome-level grading for agents.
• Optimize inference so models run efficiently on large volumes of sensitive data inside customer environments.
• Partner directly with frontier labs and enterprise ML team to turn hard data problems into shipped model improvements.

Skills

Production ML

LLMs/Agents

NER/Info extraction

Software engineering

PyTorch

Privacy/data protection

Problem framing

Synthetic data

Tools

PyTorch

Distributed training

Agent/benchmark frameworks

Job description

Tonic builds the data infrastructure behind modern AI. We create synthetic environments and de-identify real enterprise data for safe training and evaluation.

This role designs systems to generate longitudinally coherent synthetic environments for agent training, including persona modeling and verifiable ground truth. You will train and improve NER models, scale synthesis models, and build evaluation infrastructure to distinguish frontier models on real tasks.