Remote ML Systems Engineer - Prompting & Evaluation

DataAnnotation; Madison (WI); Remote; USD 170,000 - 243,000; Full time

Benefits offered by this job

Remote work

Flexible schedule

Project-based work

Job summary

DataAnnotation is seeking a Machine Learning Engineer to stress-test how models reason about ML systems, including training dynamics, evaluation design, data pipelines, and deployment trade-offs. You will write prompts that probe model reasoning and review AI outputs for subtle errors, then craft the correct solution grounded in practitioner experience.

The role emphasizes hands-on work with Python and the modern ML stack (PyTorch/JAX).

Qualifications

• Hands-on experience training, evaluating, or deploying models professionally or in serious personal work.
• Comfort with the modern ML stack; tasks assume Python and PyTorch or JAX.
• Clear written English; explanations are the training signal.
• No degree required; focus on what you can do.

Responsibilities

• Write prompts that probe how models reason about training, evaluation, debugging, and productionizing ML systems.
• Review AI output for subtle errors: leaky evaluations, wrong loss formulations, and misdiagnosed training failures.
• Write the correct solution when the model falls short, grounded in real practitioner experience.

Skills

Python

PyTorch

JAX

Clear English writing

Tools

PyTorch

JAX

Git

Job description

DataAnnotation is seeking a Machine Learning Engineer to stress-test how models reason about ML systems, including training dynamics, evaluation design, data pipelines, and deployment trade-offs. You will write prompts that probe model reasoning and review AI outputs for subtle errors, then craft the correct solution grounded in practitioner experience.

The role emphasizes hands-on work with Python and the modern ML stack (PyTorch/JAX).