Senior ML Engineer, Nova AI — Build & Optimize Models

Iterable, Inc.; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Iterable is an AI customer engagement platform trusted by global brands. As a Senior Machine Learning Engineer on Nova, you will lead evaluation and automatic optimization for Nova's models and agents.

That means measuring how they perform, finding where they fail, and using those findings to automatically improve prompts, context, tools, and agent harnesses. You will work with the Nova retrieval and product teams to connect real interactions to measurable improvements.

Qualifications

• 5+ years of hands-on MLE experience putting complex ML systems into production at scale.
• Master's or PhD in Computer Science, Machine Learning, AI, or a related field.
• You keep up with the latest research and focus on applying it: shipping reliable changes that measurably improve the product experience for customers.
• Strong Python skills with an emphasis on readable, modular, and maintainable code, plus experience with modern ML frameworks.
• Deep experience evaluating LLMs or agents, including multi-turn behavior, tool use, and reasoning.
• Experience building automated evaluation, such as LLM judges, reward models, or benchmark infrastructure.
• Experience with at least one of: prompt or context optimization, post-training, preference optimization, or reinforcement learning.
• Strong communication and cross-team collaboration.

Responsibilities

• Build evaluation systems for models and agents: benchmarks, LLM judges, simulations, trajectory analysis, and regression tests.
• Create a loop that takes failures from diagnosis to fix to measured improvement.
• Turn evaluation results into automatic optimization of prompts, context, tools, rubrics, and agent harnesses.
• Build synthetic data pipelines for evaluation and post-training, with attention to quality, privacy, and reproducibility.
• Apply recent research (LLM-as-judge, reward modeling, test-time search, post-training) to production.
• Mentor engineers on the team and help set the technical direction for Nova.

Skills

Python

ML pipelines

LLM evaluation

Cross-team collaboration

ML frameworks

Education

Master's or PhD in Computer Science, Machine Learning, AI

Tools

PyTorch

LLM APIs (Anthropic, OpenAI, Google)

Job description

Iterable is an AI customer engagement platform trusted by global brands. As a Senior Machine Learning Engineer on Nova, you will lead evaluation and automatic optimization for Nova's models and agents.

That means measuring how they perform, finding where they fail, and using those findings to automatically improve prompts, context, tools, and agent harnesses. You will work with the Nova retrieval and product teams to connect real interactions to measurable improvements.