Post-Training AI Engineer: Behavior & Evaluation

Jobtailor; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Jobtailor is seeking an AI post-training specialist to design and run workflows that improve the behavior, reliability, and usefulness of enterprise AI systems.

You will develop datasets, reward models, evaluation suites, and fine-tuning workflows, partnering with researchers and engineers to apply methods in deployed systems.

You will analyze outputs, create repeatable processes for domain adaptation, and clearly communicate results, limitations, and tradeoffs to stakeholders.

Qualifications

• Experience with fine-tuning, preference optimization, RL, reward modeling, or post-training techniques.
• Ability to build training/evaluation pipelines and run controlled experiments.
• Curious researcher who cares about why behavior changes.
• Understanding that system behavior comes from data, prompts, tools, and deployment context.
• Daily use of AI tools to accelerate coding and research.
• Bias toward measurement with concrete evaluations and metrics.
• Able to balance ambitious goals with cost, latency, and reliability constraints.
• Takes ownership for real-world outcomes from post-training work.

Responsibilities

• Design and run post-training workflows to improve AI system behavior.
• Develop datasets, signals, evaluation suites, reward models, and fine-tuning workflows.
• Investigate effects of post-training techniques across workflows and constraints.
• Build infrastructure for experimentation, model comparison, regression testing, and analysis.
• Collaborate with AI Researchers and AI Engineers to apply techniques in deployed systems.
• Analyze model outputs, failures, feedback, and production traces for improvement.
• Create repeatable processes to adapt AI systems to customer domains while maintaining robustness.
• Communicate evaluation results, limitations, and tradeoffs to internal and customer stakeholders.

Skills

Model behavior

Programming & experiments

Research mindset

AI systems thinking

AI-native tooling

Measurement-driven

Constraints handling

Ownership mentality

Job description

Jobtailor is seeking an AI post-training specialist to design and run workflows that improve the behavior, reliability, and usefulness of enterprise AI systems.

You will develop datasets, reward models, evaluation suites, and fine-tuning workflows, partnering with researchers and engineers to apply methods in deployed systems.

You will analyze outputs, create repeatable processes for domain adaptation, and clearly communicate results, limitations, and tradeoffs to stakeholders.