Remote ML Systems QA & Evaluation Engineer

DataAnnotation; Oklahoma; Remote; USD 124,000 - 207,000; Full time

Benefits offered by this job

Flexible, remote work

Job summary

DataAnnotation is seeking a Machine Learning Engineer to stress-test how models reason about ML systems, including training dynamics, evaluation design, data pipelines, and deployment trade-offs. The role focuses on crafting prompts that reveal reasoning gaps and validating model outputs against real-world expectations.

You’ll review outputs, identify subtle errors, and write correct, principled solutions grounded in practical experience.

Qualifications

• Hands-on experience training, evaluating, or deploying models professionally or in serious personal work.
• Comfort with the modern ML stack; most tasks assume Python and PyTorch or JAX.
• Clear written English: your explanations are the training signal.
• No degree required. We care about what you can do, not where you learned it.

Responsibilities

• Write prompts that probe how models reason about training, evaluation, debugging, and productionizing ML systems.
• Review AI output for subtle errors: leaky evaluations, wrong loss formulations, and misdiagnosed training failures.
• Write the correct solution when the model falls short, grounded in real practitioner experience.

Skills

ML engineering

Python

PyTorch

JAX

Model evaluation

Productionizing ML

Tools

PyTorch

JAX

Job description

DataAnnotation is seeking a Machine Learning Engineer to stress-test how models reason about ML systems, including training dynamics, evaluation design, data pipelines, and deployment trade-offs. The role focuses on crafting prompts that reveal reasoning gaps and validating model outputs against real-world expectations.

You’ll review outputs, identify subtle errors, and write correct, principled solutions grounded in practical experience.