ML Systems Auditor & Prompt Engineer — Remote

DataAnnotation; Oregon; Remote; USD 124,000 - 207,000; Full time

Benefits offered by this job

Remote work

Project-based work

Job summary

DataAnnotation is seeking a Machine Learning Engineer to stress-test how models reason about ML systems, including training dynamics, evaluation design, data pipelines, and deployment trade-offs. You’ll write prompts that probe reasoning, review AI outputs for subtle errors, and craft correct solutions grounded in practitioner experience, with remote-flexible engagement and project-based work.

As part of DataAnnotation, you’ll work with a modern ML stack (Python, PyTorch or JAX), with no degree

Qualifications

• Hands-on experience training, evaluating, or deploying models in professional or serious personal projects.
• Comfort with Python and PyTorch or JAX.
• Clear written English; explanations as the training signal.
• No degree required; demonstrated ability matters more than where learned.

Responsibilities

• Write prompts that probe how models reason about training, evaluation, debugging, and productionizing ML systems.
• Review AI output for subtle errors: leaky evaluations, wrong loss formulations, and misdiagnosed training failures.
• Write the correct solution when the model falls short, grounded in real practitioner experience.

Job description

DataAnnotation is seeking a Machine Learning Engineer to stress-test how models reason about ML systems, including training dynamics, evaluation design, data pipelines, and deployment trade-offs. You’ll write prompts that probe reasoning, review AI outputs for subtle errors, and craft correct solutions grounded in practitioner experience, with remote-flexible engagement and project-based work.

As part of DataAnnotation, you’ll work with a modern ML stack (Python, PyTorch or JAX), with no degree